Bugzilla – Full Text Bug Listing
| Summary: | gef tests won't run on modeling.eclipse.org | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | [Tools] GEF | Reporter: | Nick Boldt <nboldt> | ||||||||||
| Component: | RelEng | Assignee: | Nick Boldt <nboldt> | ||||||||||
| Status: | RESOLVED FIXED | QA Contact: | |||||||||||
| Severity: | normal | ||||||||||||
| Priority: | P3 | CC: | ahunter.eclipse, david_williams, dennis.huebner, gheorghe, kim.moir, overholt | ||||||||||
| Version: | unspecified | ||||||||||||
| Target Milestone: | --- | ||||||||||||
| Hardware: | PC | ||||||||||||
| OS: | Linux | ||||||||||||
| Whiteboard: | |||||||||||||
| Bug Depends on: | |||||||||||||
| Bug Blocks: | 252423, 273485 | ||||||||||||
| Attachments: |
|
||||||||||||
|
Description
Nick Boldt
Why is the configuration of modeling.eclipse.org not 100% the same as emft.eclipse.org? Sorry if obvious, but ... does emft tests require a "UI"? I'm sure GEF would. For headless tests, that require a UI, an X virtual frame buffer must be created and used as a "virtual" display. You can google for Xvfb if this doesn't sound familar, but basically you need to start one for your own work (if not already running) and then set the DISPLAY of your process with something like # this variable must batch the screen number that Xvfb is using export DISPLAY=127.0.0.1:I.0 where 'I' is the display number used when you start Xvfb. (In reply to comment #1) > Why is the configuration of modeling.eclipse.org not 100% the same as > emft.eclipse.org? Because emft.eclipse.org runs Fedora Core 5, and modeling.eclipse.org runs SLES 10.2. I don't have control over what the WM puts on the vserver images -- I just asked for more space & RAM, and since Novell provides software to the EF, we get SLES. (In reply to comment #2) > Sorry if obvious, but ... does emft tests require a "UI"? I'm sure GEF would. We've been running headless UI tests using Xvfb since GEF adopted the Modeling build. So, yes, obvious. :) What's not obvious is why Debian/ubuntu and FC5 are all happily running these tests, but SLES is not. Evidently something is missing -- I was wondering if anyone had any bright ideas about what that something might be. So, I've tried... adding the swt.gtk jar to the classpath, to java.library.path, and to swt.library.path, and nothing works.
[exec] /opt/sun-java2-1.4/bin/java
[exec] -Xms256M
[exec] -Xmx256M
[exec] -enableassertions
[exec] -cp eclipse/plugins/org.eclipse.swt.gtk.linux.x86_3.5.0.v3525.jar:eclipse/plugins/org.eclipse.equinox.launcher_1.0.200.v20081201-1815.jar org.eclipse.equinox.launcher.Main
[exec] -ws gtk
[exec] -os linux
[exec] -arch x86
[exec] -application org.eclipse.ant.core.antRunner
[exec] -data ./eclipse/workspace
[exec] -file test.xml all
[exec] -Dosgi.bundlefile.limit=100
[exec] -Djava.library.path=/opt/sun-java2-1.4/jre/lib/i386:/opt/swt
[exec] -Dswt.library.path=/opt/sun-java2-1.4/jre/lib/i386:/opt/swt
[exec] -Dplatform=linux.gtk
[exec] -Dws=gtk
[exec] -Dos=linux
[exec] -Darch=x86
[exec] -Dclean=true
[exec] -logger org.apache.tools.ant.DefaultLogger
I've tried hardcoding "i386" instead of "x86".
I still get java.lang.UnsatisfiedLinkError: no swt-pi-gtk-3525 or swt-pi-gtk in swt.library.path, java.library.path or the jar file.
http://modeling.eclipse.org/gef/downloads/?project=gef&sortBy=date&hlbuild=0#latest
I'm out of ideas. Given that modeling.eclipse.org is a 64-bit box, do you think running these tests w/ a 64-bit Eclipse + Java would help? There's no 64-bit Sun Java 1.4, and I tried naively running the build+tests w/ Sun 6.0 64-bit but that too failed. Would building with 1.4 32-bit and testing with 6.0 64-bit be valid, much less reasonable?
Can you compare this command on SLES(modeling) and FC5 ? rpm -qa | grep -i gtk emft $ rpm -qa | grep -i gtk gtk2-2.6.10-2.fc4.4 modeling $ rpm -qa | grep -i gtk |sort gtk-1.2.10-907.2 gtk2-2.8.10-39.12 gtk2-32bit-2.8.10-39.12 gtk2-engines-2.6.7-17.2 gtk2-engines-32bit-2.6.7-17.2 gtk2-themes-0.1-653.2 gtk-32bit-1.2.10-907.2 gtk-engines-0.12-982.2 gtkhtml2-3.10.0-15.2 gtk-sharp2-2.8.2-15.3 gtk-sharp2-32bit-2.8.2-15.3 gtksourceview-1.5.6-18.2 gtkspell-2.0.11-20.2 python-gtk-2.8.2-21.2 Could this stem from the fact that on emft (FC5), there's only the 32-bit gtk2, but on modeling (SLES 10.2), the default gtk2 is 64-bit? Could this be an actual "running 32-bit headless UI tests w/ 32-bit SWT on 64-bit linux platform" bug? (Copying swt inbox and Kim Moir, in case this is a real problem in swt or in basebuilder.releng, though I kinda doubt it is.) Or, can you suggest what I might be doing wrong or what's missing on the server? Note that we're attempting to build w/ Eclipse 3.5M4 here, which includes eclipse/plugins/org.eclipse.swt.gtk.linux.x86_3.5.0.v3525.jar eclipse/plugins/org.eclipse.equinox.launcher_1.0.200.v20081201-1815.jar ... and that this works fine on emft.eclipse.org (which is Fedora Core 5) ... http://emft.eclipse.org/gef/downloads/?sortBy=date (note the GEF 3.5.0M4 build) ... but not on modeling.eclipse.org (SLES 10.2) ... http://modeling.eclipse.org/gef/downloads/?sortBy=date regarding comment #7, I haven't seen this issue before in our builds. Note that you will also get that message if the library is on the path but is unable to be loaded for some reason. If you are seeing this problem on a 64 bit machine, my guess is that you are missing some (or all) of the 32 bit versions of the libraries required by the swt-pi-gtk library. To verify (or refute) this, you can go into the configuration directory in the eclipse folder, locate the location of the extracted pi library (should be something like org.eclipse.osgi/bundles/[some number]/1/.cp) and perform a ldd on it. It looks like the 32-bit ppc .sos are present and linked properly: $ cd /opt/public/cbi/build/gef/downloads/drops/3.4.0/N200901161430/testing/N200901161430/testing/target/eclipse/configuration/org.eclipse.osgi/bundles/128/1/.cp $ ldd libswt-pi-gtk-3449.so ldd: warning: you do not have execution permission for `./libswt-pi-gtk-3449.so' linux-vdso32.so.1 => (0x00100000) libgtk-x11-2.0.so.0 => /opt/gnome/lib/libgtk-x11-2.0.so.0 (0x6fba6000) libgthread-2.0.so.0 => /opt/gnome/lib/libgthread-2.0.so.0 (0x6fb81000) [... no missing targets ...] $ file `readlink -f /opt/gnome/lib/libgtk-x11-2.0.so.0` $ ldd !$ [ output seems correct to me] (In reply to comment #9) > Note that you will also get that message if the library is on the path but is > unable to be loaded for some reason. If you are seeing this problem on a 64 bit > machine, my guess is that you are missing some (or all) of the 32 bit versions > of the libraries required by the swt-pi-gtk library. To verify (or refute) > this, you can go into the configuration directory in the eclipse folder, locate > the location of the extracted pi library (should be something like > org.eclipse.osgi/bundles/[some number]/1/.cp) and perform a ldd on it. When the tests run on build.eclipse.org (using the ppc version of Eclipse 3.4.1), I get these files: ./target/eclipse/plugins/org.eclipse.swt_3.4.1.v3449c.jar ./target/eclipse/plugins/org.eclipse.swt.gtk.linux.ppc.source_3.4.1.v3449c.jar ./target/eclipse/plugins/org.eclipse.swt.gtk.linux.ppc_3.4.1.v3449c.jar ./target/eclipse/plugins/org.eclipse.equinox.launcher.gtk.linux.ppc_1.0.101.R34x_v20080731/eclipse_1115.so ./target/eclipse/configuration/org.eclipse.osgi/bundles/23/1/.cp/os/linux/ppc/liblocalfile_1_0_0.so ./target/eclipse/configuration/org.eclipse.osgi/bundles/128/1/.cp/libswt-gtk-3449.so ./target/eclipse/configuration/org.eclipse.osgi/bundles/128/1/.cp/libswt-xpcominit-gtk-3449.so ./target/eclipse/configuration/org.eclipse.osgi/bundles/128/1/.cp/libswt-atk-gtk-3449.so ./target/eclipse/configuration/org.eclipse.osgi/bundles/128/1/.cp/libswt-pi-gtk-3449.so ./target/eclipse/configuration/org.eclipse.osgi/bundles/128/1/.cp/libswt-cairo-gtk-3449.so And indeed, the tests run: http://build.eclipse.org/cbi/build/gef/downloads/drops/3.4.0/N200901161430/testing/N200901161430/testing/results/html/org.eclipse.draw2d.test_linux.gtk.html (103/109 pass, 6 failures) http://build.eclipse.org/cbi/build/gef/downloads/drops/3.4.0/N200901161430/testing/N200901161430/testing/results/html/org.eclipse.gef.test_linux.gtk.html (6/6 pass) I'll try to see if GEF works on modeling.eclipse with the latest x86_64 Eclipse build instead of the x86 one. Created attachment 122846 [details] console out to compare to comment 10 Updating map used by GEF to latest, and building with 3.4.1 ppc, ALL tests pass! http://build.eclipse.org/cbi/build/gef/downloads/drops/3.4.0/N200901161647/testresults/html/org.eclipse.draw2d.test_linux.gtk.html http://build.eclipse.org/cbi/build/gef/downloads/drops/3.4.0/N200901161647/testresults/html/org.eclipse.gef.test_linux.gtk.html This is great news for Dash Athena project... but here's the bad news [1] for modeling.eclipse: [1] http://spreadsheets.google.com/ccc?key=pZbKyLy8DKkzJwJoztpR1Rg Using 32-bit Eclipse 3.4.1, I compile OK, but tests fail. java.lang.UnsatisfiedLinkError: no swt-pi-gtk-3449 or swt-pi-gtk in swt.library.path, java.library.path or the jar file at org.eclipse.swt.internal.Library.loadLibrary(Library.java:233) See attached file, 253114.console.out.ldd.txt for results of ldd. Using 64-bit Eclipse 3.4.1, I can't compile: [javac] 2. ERROR in /home/www-data/build/gef/downloads/drops/3.4.2/N200901161514/eclipse/plugins/org.eclipse.draw2d/src/org/eclipse/draw2d/AbstractLabeledBorder.java (at line 13) [javac] import org.eclipse.swt.graphics.Color; [javac] ^^^^^^^^^^^^^^^ [javac] The import org.eclipse.swt cannot be resolved Oh, and not surprisingly using PPC eclipse on modeling.eclipse didn't help either... but I figured I'd give it a shot. Compile was OK, but tests DNR: org.osgi.framework.BundleException: The activator org.eclipse.ui.internal.WorkbenchPlugin for bundle org.eclipse.ui.workbench is invalid Caused by: java.lang.NoClassDefFoundError: org/eclipse/swt/SWTError org.eclipse.core.runtime.CoreException: Plug-in org.eclipse.test was unable to load class org.eclipse.test.UITestApplication. Caused by: org.eclipse.core.runtime.internal.adaptor.EclipseLazyStarter$TerminatingClassNotFoundException: An error occurred while automatically activating bundle org.eclipse.ui.workbench (154). I updated the spreadsheet w/ the found *swt*.jar and *.so files from the tests. I'm exploring a solution to running tests on 64-bit linux here: https://jira.jboss.org/jira/browse/JBQA-2244 <for param="swtgtkjar"> <path> <fileset dir="${test-eclipse-root}/plugins" includes="org.eclipse.swt.gtk.*.jar" excludes="*source*" /> </path> <sequential> <unzip src="@{swtgtkjar}" dest="@{swtgtkjar}_" overwrite="true" /> <var name="sofiles" value="" /> <property name="LD_LIBRARY_PATH" value="@{swtgtkjar}_" /> <for param="sofile"> <path> <fileset dir="@{swtgtkjar}_" includes="*.so" /> </path> <sequential> <var name="sofiles" value="${sofiles}, @{sofile}" /> </sequential> </for> <propertyregex input="${sofiles}" defaultvalue="${sofiles}" regexp=", (.+)" select="\1"/> <echo>Set LD_LIBRARY_PATH = ${LD_LIBRARY_PATH} to resolve ${sofiles} </echo> </sequential> </for> and this to the <java> that runs the tests: <sysproperty key="LD_LIBRARY_PATH" path="${LD_LIBRARY_PATH}" /> It's also been suggested that we add: <jvmarg value="-Djava.library.path=${LD_LIBRARY_PATH}" /> The above solution requires ant-contrib, so it won't help as-is for GEF on modeling.eclipse... but with any luck we'll just start building on build.eclipse.org using Athena instead. :) (In reply to comment #14) > I'm exploring a solution to running tests on 64-bit linux here: Hi Nick, I am wondering if we really even want the GEF / GMF builds running on 64-bit Linux. Given that the JUnit tests are effectively a smoke test for the framework, do we really want these running on a platform that close to zero of the Eclipse population will actually be using? (In reply to comment #15) > (In reply to comment #14) > > I'm exploring a solution to running tests on 64-bit linux here: > Hi Nick, I am wondering if we really even want the GEF / GMF builds running on > 64-bit Linux. > Given that the JUnit tests are effectively a smoke test for the framework, do > we really want these running on a platform that close to zero of the Eclipse > population will actually be using? GEF is used by VE, by JBoss Tools, and others. All these run on Win32, Lin32, Lin64, and Mac32 (Carbon, soon to include Cocoa 64?). So... yes, there's a need for support on those platforms; but really, it's about being able to simply *run the tests* on those platforms. And really, if it turns out this is simply an SWT/classpath issue, then the box on which we run becomes irrelevant. Also, if I get the doc/javadoc stuff fixed in Athena, we can move GEF there, and the vservers become *way* less relevant. :) Sounds good Nick. I felt somewhat bad seeing you spending hours trying to fix the 64-bit Linux, when we are perfectly happy with the 32-bit Linux right now. No doubt you love this challenge :-) (In reply to comment #17) > Sounds good Nick. I felt somewhat bad seeing you spending hours trying to fix > the 64-bit Linux, when we are perfectly happy with the 32-bit Linux right now. > No doubt you love this challenge :-) Actually, I hate it... but if I can solve it for JBoss Tools we'll more than double the number of nodes on which we can build and thus speed up our cycle time. Still, this is irrelevant once we flip over to running on build.eclipse.org. :) (In reply to comment #18) > Still, this is irrelevant once we flip over to running on build.eclipse.org. :) Running *GEF* that is, not JBT. Created attachment 135361 [details]
patch to switch from x86 to x86_64 and JDK1.4 (32) to JDK6.0_64
Created attachment 135362 [details]
mylyn/context/zip
context (3 files)
Well, I've figured out why the 64-bit linux Eclipse doesn't work w/ Sun 1.4: "This build requires a 64-bit JVM, and will not run with a 32-bit JVM. You can, for example, use the Sun 64-bit 1.5 JVM for AMD64. Note that the Sun 1.4.2 JVM for AMD64 is 32-bit and therefore cannot be used to run this build." Spun a 3.5 build using Sun 6.0_64 and Eclipse x86_64 and got: !MESSAGE Platform filter did not match: (& (osgi.os=linux) (osgi.arch=x86_64)) So, I've hacked up the test script to switch to 6.0 JVM and arch=x86_64... and it works! 105 and 6 tests pass. Updated: http://spreadsheets.google.com/ccc?key=pZbKyLy8DKkzJwJoztpR1Rg ------------ So, going forward, here's how you build and run tests on modeling.eclipse.org (64-bit SLES): 1. "search your .releng project for 'arch=x86' and replace w/ 'arch=x86_64'" org.eclipse.gef.releng/builder/all/build.properties org.eclipse.gef.releng/builder/tests/build.properties org.eclipse.gef.releng/builder/tests/configs/local/relengbuildgtk.sh 2. replace existing code to add single x86 swt jar to classpath in org.eclipse.gef.releng/builder/tests/configs/local/relengbuildgtk.sh with # add swt jars cpAndMain=`find eclipse/ -name "org.eclipse.swt_*.jar" | sort | head -1`":"$cpAndMain; cpAndMain=`find eclipse/ -name "org.eclipse.swt.gtk.linux.${arch}_*.jar" | sort | head -1`":"$cpAndMain; 3. in build/_common.php, use a 6.0 JVM, and ensure your BREEs are correctly set in MANIFEST.MF to avoid accidentally forcing users to need a 6.0 JVM too "modeling.eclipse.org=------------,------------", "3.5.0=HEAD,/opt/sun-java2-6.0_64", 4. use the linux-gtk-x86_64 Eclipse SDK, not the normal linux-gtk one as input to the build. Created attachment 135389 [details]
modified patch
Should use JavaSE-1.6 not J2SE-1.5 (because Sun likes to rename things); also commenting out the 6.0 classpath stuff because the tests don't req 1.6 and shouldn't be setting javacSource and javacTarget to 1.6
Solution is actually simpler than reported in comment 22. I've wiki'd this so that it'll be easier to fix this phenomenal failure in the far-flung future. Say that five times fast. :) http://wiki.eclipse.org/Modeling_Project_Releng/Building#Cannot_run_tests_on_64-bit_OS Anthony said: I have smoke tested the GEF build GEF-ALL-N200905121721.zip with the platform M7 on three places: Ubuntu Linux with Sun JRE 1.4.2 SR10 Windows XP with Sun JRE 1.4.2 SR18 Windows XP with IBM JRE 1.4.2 Service Refresh 13 The only issue that I could find was that the javadoc in the GEF doc.isv was not working (bug 256211, bug 269290). The JavaDoc is working in GEF M7 so this is one remaining modeling.eclipse.org build issue. --- Closing; remaining doc issue will be addressed elsewhere (bug 256211, bug 269290). |