Bugzilla – Full Text Bug Listing
| Summary: | jetty-test-windows and jetty-test-mac | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Community | Reporter: | Jesse McConnell <jesse.mcconnell> | ||||||||
| Component: | CI-Jenkins | Assignee: | Eclipse Webmaster <webmaster> | ||||||||
| Status: | CLOSED FIXED | QA Contact: | |||||||||
| Severity: | enhancement | ||||||||||
| Priority: | P3 | CC: | denis.roy, d_a_carver, mgorovoy | ||||||||
| Version: | unspecified | ||||||||||
| Target Milestone: | --- | ||||||||||
| Hardware: | PC | ||||||||||
| OS: | Mac OS X - Carbon (unsup.) | ||||||||||
| Whiteboard: | |||||||||||
| Attachments: |
|
||||||||||
|
Description
Jesse McConnell
Done. -M. Sorry to bother but can we get maven installed on these boxes as well? neither the maven 3.0 or the 3.0.3 targets are working c:\hb\workspace\jetty-test-windows>set MAVEN_OPTS="-Xmx768m -XX:MaxPermSize=256m" c:\hb\workspace\jetty-test-windows>exit 0 FATAL: Couldn't find any executable in /shared/common/apache-maven-3.0 Recording test results > it would be very helpful if we can trigger a build on these
The Windows and Mac slaves are not meant for building. They should only be used for platform-specific testing which cannot be done on Linux, such as UI tests. We simply don't have the cycles to maintain every project's particular build stack across multiple platforms.
the build is how we launch all of our tests though ok, well if you can't do it you can't do it...feel free to smoke these build entries and I'll see if I can sort out another solution thanks jesse Done. -M. > the build is how we launch all of our tests though
As I understand it, the best way to do this is to have a separate jobs for the actual build process, and another for the tests. Job a would then call job b, and these jobs need not be tied or executed on the same slave.
Feel free to discuss this on the cross-project mailing list, I'm sure others who are running UI tests on the Windows and Mac slaves have this figured out.
Thanks.
Well, we have no UI tests...and no tests that are separate from the build. Its simply a matter that hudson infrastructure isn't setup to handle typical maven builds. Stock maven unit tests are grouped with their source so in each module you build the module source, then build test source, and then run the corresponding unit tests. Rinse and repeat for all other modules in the build. From here, I'm not sure what to do. The maven installs on /shared/common _should_ work since they should simply invoke java using $JAVA_HOME. I can see mvn.bat in /shared/common/apache-maven-3.0.3/bin which is clearly windows-specific. well the build entries are removed so I can't look at it anymore, but if you want to add just that one back I can try and spend time time to debug, at first blush the error looked like it couldn't find maven on a *nix path /foo/common (In reply to comment #9) > it couldn't find maven on a *nix path /foo/common Agh, that is right, we don't mount the *nix NFS exports on the Windows and Mac slaves. I really don't want to keep stonewalling your efforts, since my fear is that by opening the Windows and Mac slaves for building, they will easily be choked with build jobs when the actual build process doesn't need to be run here. However I do want to accommodate those projects who need to do cross-platform testing, so I've dumped maven in C:\Program Files\maven-3.0.3 and I've defined maven 3.0.0 in the Hudson tools for the Windows slave. Matt, can you re-establish the Windows job for Jesse? For some odd reason, I feel your build won't get very far on the Windows slave :/ The windows job is back. -M. Well, the maven bit seems to be being called now...something hinky with the java home though c:\hb\workspace\jetty-test-windows>exit 0 [jettytrunk] $ "c:\Program Files\maven-3.0.3\bin\mvn.bat" -fae clean install ERROR: JAVA_HOME is set to an invalid directory. JAVA_HOME = "c:\Java\jdk1.6.0_20\bin" Please set the JAVA_HOME variable in your environment to match the location of your Java installation All the JAVA_HOME entries pointed to \bin which I believe is incorrect. Please try now. I hope I didn't break any of Kim's stuff. nope, same deal.. and that is interesting, I have always pointed java_home to the top of the java installation, and then pointed the PATH at the bin directory oh, yes...that would be incorrect, sorry I read that first thinking that you were setting them all to /bin Created attachment 192219 [details]
Screenshot
Disclaimer: I hate Windows
I'm looking at the path and it seems correct... I'm not sure what more the build is lookig for. The bin/ directory does contain the usual suspects.
know windows you probably either need to reboot or restart the hudson process it magically sees the new env settings Hold the phone -- there was a separately defined JAVA_HOME which was set to c:\Java\jdk1.6.0_20\bin I've corrected it -- can you please entertain us one more time? https://hudson.eclipse.org/hudson/view/Jetty-RT/job/jetty-test-windows/22/console its doing something now... looks like its probably getting mixed up with the http proxy now what are those settings and I'll see if I can get them into the mvn cli I am using We've got JAVA_OPTS and JVM_ARGS set to this: -Dhttp.proxyHost=proxy.eclipse.org -Dhttp.proxyPort=9898 -Dhttps.proxyHost=proxy.eclipse.org -Dhttps.proxyPort=9898 -Dhttp.nonProxyHosts="*.eclipse.org" -Dhttps.nonProxyHosts="*.eclipse.org" -Dftp.proxyHost=proxy.eclipse.org -Dftp.proxyPort=9898 -Dftp.nonProxyHosts="*.eclipse.org" ok, I have it working from the command line.. I talked with David Carver a bit and a better solution would be if I could get you to add a mirror entry to the settings.xml for that maven instance.. in the conf/settings.xml of the maven install you could put this: <settings> <mirrors> <mirror> <!--This sends everything else to /public --> <id>eclipse</id> <mirrorOf>*</mirrorOf> <url>http://maven.eclipse.org/nexus/content/groups/central/</url> </mirror> </mirrors> then we will make use of the nice new nexus install and save a fair amount of download bandwith... is that something you could do? sorry to impose like this but thanks so much for helping get this windows box up and running like this (In reply to comment #18) > Hold the phone -- there was a separately defined JAVA_HOME which was set to > c:\Java\jdk1.6.0_20\bin > > I've corrected it -- can you please entertain us one more time? How are the Windows Java settings setup on the slave in the Hudson config. If they are setup right in the configuration, then just selecting the particular JDK to use during job configuration should get the appropriate JDK settings. The other thing you can do as well, is just let Hudson download maven and JDK, but maybe that isn't something you want to let it do, but would be less maintenance and setup for these types of things if Hudson can just do it. (In reply to comment #21) > ok, I have it working from the command line.. > > I talked with David Carver a bit and a better solution would be if I could get > you to add a mirror entry to the settings.xml for that maven instance.. > > in the conf/settings.xml of the maven install you could put this: > > <settings> > <mirrors> > <mirror> > <!--This sends everything else to /public --> > <id>eclipse</id> > <mirrorOf>*</mirrorOf> > > <url>http://maven.eclipse.org/nexus/content/groups/central/</url> > </mirror> > </mirrors> > > > then we will make use of the nice new nexus install and save a fair amount of > download bandwith... > > is that something you could do? sorry to impose like this but thanks so much > for helping get this windows box up and running like this We may want to hold off on adding the above just yet. Some url locations are changing soon. I'll create a separate bug for the settings.xml changes to use maven.eclipse.org Created attachment 192221 [details]
Screenshot of Java options
Here's how we define the java options. Let me know if there is a better way of doing it.
(In reply to comment #24) > Created attachment 192221 [details] > Screenshot of Java options > > Here's how we define the java options. Let me know if there is a better way of > doing it. No that is the correct way to do it if you aren't going to allow Hudson to download everything. Just need to make sure they point to the correct locations, which is more of a manual effort and maintenance. If you let hudson handle it, it'll download the appropriate JVM on the salve and JAVA_HOME will be setup. > The other thing you can do as well, is just let Hudson download maven and JDK,
> but maybe that isn't something you want to let it do
I still have mental blockage on that issue. Maybe some day it will pass, but for now I'm not emotionally ready to allow that :)
We had a successful jetty build on windows! this date should be marked on a calendar somewhere... anyway, there are different ways to get things configured but we have one sekrit sauce way in use now so that is fine... thanks a lot denis and matt (and david) for helping get this in place I have it building once a day instead of polling SCM, this is mostly just a compliance check for our unit tests and build processes on windows so we don't aim to suck up much in the way of resources on this build cheers, jesse That is good news. If all you need is Maven, do you want us to set up a job to run on the mac slave? That would be peachy keen...have to add back the jetty-test-mac build entry as well (In reply to comment #28) > That is good news. If all you need is Maven, do you want us to set up a job to > run on the mac slave? Denis, Could you guys please create jetty-test-mac job on the mac slave for us as discussed earlier? Thanks, Michael I've created the job and given Jesse full access to the config, so others can be added. I also set the 'restrict where this job can be run' to mac-tests. -M. I'm getting the following error when running the jetty-test-mac job. Please make sure that Maven 3.0.3 is accessible from the Mac slave machine(s). [jetty-test-mac] $ /bin/sh -xe /var/folders/ae/ae-8Kon+EhOvil7-mjEmok++piA/-Tmp-/hudson3429992287512867140.sh + export 'MAVEN_OPTS=-Xmx768m -XX:MaxPermSize=384m' + MAVEN_OPTS='-Xmx768m -XX:MaxPermSize=384m' FATAL: Couldn't find any executable in /shared/common/apache-maven-3.0.3 Recording test results Sending e-mails to: jetty-build@eclipse.org Finished: FAILURE (In reply to comment #32) Just wondering if my Friday post got buried in the weekend's worth of email... -Michael (In reply to comment #32) > I'm getting the following error when running the jetty-test-mac job. Please > make sure that Maven 3.0.3 is accessible from the Mac slave machine(s). > > [jetty-test-mac] $ /bin/sh -xe > /var/folders/ae/ae-8Kon+EhOvil7-mjEmok++piA/-Tmp-/hudson3429992287512867140.sh > + export 'MAVEN_OPTS=-Xmx768m -XX:MaxPermSize=384m' > + MAVEN_OPTS='-Xmx768m -XX:MaxPermSize=384m' > FATAL: Couldn't find any executable in /shared/common/apache-maven-3.0.3 > Recording test results > Sending e-mails to: jetty-build@eclipse.org > Finished: FAILURE Matt, looks like Maven isn't configured properly on the Mac slave. Can you check to make sure the mac slave can see that configuration? <bump> Could someone please respond to our request made 5 days ago today?? Sorry.. Matt is on vacation, so there is some backlog. /shared is not accessible to the Windows and Mac slaves. I've installed Maven 3.0.3 and configured your build to use it, but now it's complaining about a missing JAVA_HOME. 'java' (the command) runs from the commandline from some obscure location, so I'm not sure what it expects me to define. Any thoughts? I'm like a fish out of water on a Mac. (In reply to comment #37) > /shared is not accessible to the Windows and Mac slaves. I've installed Maven > 3.0.3 and configured your build to use it, but now it's complaining about a > missing JAVA_HOME. 'java' (the command) runs from the commandline from some > obscure location, so I'm not sure what it expects me to define. Any thoughts? Denis, Could you please add the following command to .bash_profile of the user account that is used to run Hudson build? This should get Maven to work correctly. JAVA_HOME="/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home" Thanks, -Michael Ooops, the command should be as follows. export JAVA_HOME="/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home" Thanks -- I've added that also in the Hudson config and restarted the slave, and that seems to work. It gets a bit further, now trying to download something from http://oss.sonatype.org. I've defined the proxy settings it should use just about everywhere I can think of and it still insists on ignoring them. Here are the settings for ANT_ARGS, ANT_OPTS, JVM_ARGS, JAVA_OPTS: -Dhttp.proxyHost=proxy.eclipse.org -Dhttp.proxyPort=9898 -Dhttps.proxyHost=proxy.eclipse.org -Dhttps.proxyPort=9898 -Dhttp.nonProxyHosts="*.eclipse.org|172.30.206.*" -Dhttps.nonProxyHosts="*.eclipse.org" -Dftp.proxyHost=proxy.eclipse.org -Dftp.proxyPort=9898 -Dftp.nonProxyHosts="*.eclipse.org" Created attachment 198914 [details]
Settings.xml to use maven.eclipse.org for artifacts
Add the settings.xml to the slaves .m2 directory, and this will force maven to try and get all it's artifacts from maven.eclipse.org instead of going out to other sites. We can add more proxies at maven.eclipse.org if we need to.
This should help with the proxy situation, as everything will go through maven.eclipse.org for downloading maven artifacts.
That worked! Thanks a bunch, Dave! (In reply to comment #42) > That worked! Thanks a bunch, Dave! I have an open issue to have that same settings.xml applied to all the hudson slave instances. The first step of bug 345792 is to add the same settings.xml file as it will be required to help enable that instance. The other advantage is that this forces all the hudson slaves to get their artifacts if a build uses maven from our maven.eclipse.org repo instead of going out to central. Could potentionally save a lot on external bandwidth and speed up build times as well. > The first step of bug 345792 is to add the same settings.xml file as it will be
> required to help enable that instance.
I've put your settings.xml file in /shared/common/maven-settings.xml
The .m2/settings.xml file on all the hudson slaves is now a symlink to that file. This should allow us to easily change the config on all the slaves with that single reference.
(In reply to comment #44) > > The first step of bug 345792 is to add the same settings.xml file as it will be > > required to help enable that instance. > > I've put your settings.xml file in /shared/common/maven-settings.xml > > The .m2/settings.xml file on all the hudson slaves is now a symlink to that > file. This should allow us to easily change the config on all the slaves with > that single reference. Okay, that will work for Mac and Linux, but you'll need to add the settings.xml to Windows machine as well. Jetty has been moved to Git, and with this comes the new chapter in the life of this bug, as there is apparently no Git distribution installed on the Windows slave machines. We would greatly appreciate if you could install it for us, please. Here is a link to the page that has information about installing Git on Windows. http://book.git-scm.com/2_installing_git.html -Michael jetty-test-mac is having problems building from Git as well. It is possible that the version of Git on the Mac slave machines is incompatible with Hudson. The link above has installation instructions for Mac as well. -Michael On the closer look, looks like Git works on the Windows Slaves, but not on the Mac ones... The problem with the Windows build appears to be that it is unable to connect to the Central repository via proxy, despite the fact that the proxy settings are specified in JAVA_OPTS environment variable. To summarize the issues we are having On Windows slaves in the interest of getting them resolved faster, we are getting the following errors. The JAVA_HOME and MAVEN_OPTS environment variables are explicitly setup as shown below. Yet we fail c:\hb\workspace\jetty-test-windows>set JAVA_HOME=C:\Java\jdk1.6.0_20 c:\hb\workspace\jetty-test-windows>set MAVEN_OPTS=-Xmx768m -XX:MaxPermSize=384m -Dhttp.proxyHost=proxy.eclipse.org -Dhttp.proxyPort=9898 -Dhttps.proxyHost=proxy.eclipse.org -Dhttps.proxyPort=9898 -Dhttp.nonProxyHosts="*.eclipse.org|172.30.206.*" -Dhttps.nonProxyHosts="*.eclipse.org" -Dftp.proxyHost=proxy.eclipse.org -Dftp.proxyPort=9898 -Dftp.nonProxyHosts="*.eclipse.org" c:\hb\workspace\jetty-test-windows>exit 0 [jetty-test-windows] $ "c:\Program Files\maven-3.0.3\bin\mvn.bat" -Dmaven.test.error.ignore=true -Dmaven.test.failure.ignore=true -fae clean install [INFO] Scanning for projects... Downloading: http://oss.sonatype.org/content/groups/jetty/org/eclipse/jetty/jetty-parent/18/jetty-parent-18.pom Downloading: http://repo1.maven.org/maven2/org/eclipse/jetty/jetty-parent/18/jetty-parent-18.pom [ERROR] The build could not read 1 project -> [Help 1] [ERROR] [ERROR] The project org.eclipse.jetty:jetty-project:7.5.0-SNAPSHOT (c:\hb\workspace\jetty-test-windows\pom.xml) has 1 error [ERROR] Non-resolvable parent POM: Could not transfer artifact org.eclipse.jetty:jetty-parent:pom:18 from/to sonatype-snapshots (http://oss.sonatype.org/content/groups/jetty): Error transferring file: Connection timed out: connect and 'parent.relativePath' points at wrong local POM @ line 3, column 11 -> [Help 2] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException [ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException To summarize the issues we are having on Mac slaves in the interest of resolving them faster, we are getting the following errors. According to Dave Carver, this is most likely caused by not having Git distribution installed on the slave machine. This page http://book.git-scm.com/2_installing_git.html has instructions on how to install Git on MacOS X. Checkout:jetty-test-mac / /Users/hudsonbuild/workspace/jetty-test-mac - hudson.remoting.Channel@64c7bac7:mac-tests Using strategy: Default Checkout:jetty-test-mac / /Users/hudsonbuild/workspace/jetty-test-mac - hudson.remoting.LocalChannel@279623 GitAPI created Cloning the remote Git repository Cloning repository origin $ git clone -o origin git://git.eclipse.org/gitroot/jetty/org.eclipse.jetty.project.git /Users/hudsonbuild/workspace/jetty-test-mac ERROR: Error cloning remote repo 'origin' : Could not clone git://git.eclipse.org/gitroot/jetty/org.eclipse.jetty.project.git ERROR: Cause: Error performing git clone -o origin git://git.eclipse.org/gitroot/jetty/org.eclipse.jetty.project.git /Users/hudsonbuild/workspace/jetty-test-mac Cannot run program "git": error=2, No such file or directory Trying next repository ERROR: Could not clone repository FATAL: Could not clone hudson.plugins.git.GitException: Could not clone at hudson.plugins.git.GitSCM$2.invoke(GitSCM.java:719) at hudson.plugins.git.GitSCM$2.invoke(GitSCM.java:658) at hudson.FilePath$FileCallableWrapper.call(FilePath.java:1899) at hudson.remoting.UserRequest.perform(UserRequest.java:114) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:270) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:637) It would be nice if we could get these jobs running again soon... I've installed Git on the Mac slave. I think the issue with the network was that the default gateway need to be updated. -M. I have just tried both of the jobs, and both of them fail in exactly same way as before... -Michael According to the following page http://wiki.hudson-ci.org/display/HUDSON/Git+Plugin#GitPlugin-CommonTasks, there's a bit more configuration that needs to be done on the Mac slaves. As far as the Windows slaves go, do they need to be rebooted in order for the default gateway settings to take effect? -Michael [BUMP] Why is it the platform team is able to run UI tests on these slaves? We originally set them up to run UI tests and _only_ UI tests. Is that what you're still doing? Why do they need a default gateway? What are they trying to accomplish? Lowering the severity of this since the machines are capable of running UI tests. If you have any available workarounds, it would be great to investigate them -- otherwise, we are rapidly running out of available cycles for this. (In reply to comment #57) > Why is it the platform team is able to run UI tests on these slaves? We > originally set them up to run UI tests and _only_ UI tests. Is that what > you're still doing? Why do they need a default gateway? What are they trying > to accomplish? > > Lowering the severity of this since the machines are capable of running UI > tests. If you have any available workarounds, it would be great to investigate > them -- otherwise, we are rapidly running out of available cycles for this. Dennis this is not a UI test issue, this is a git configuration issue. See the link that Michael provided about the cloning issue. The platform team as far as I know is still using CVS to check out and then run their tests. They aren't using git yet (I could be wrong). While I understand your cycles are limited, this has been an open issue for a while. Denis, we do understand that your cycles are limited, and appreciate your help. Please look back at comments 7 & 10 in this thread for the explanation why we need Maven in order to run our cross-platform tests. Right now on Windows platform Maven is unable to download the parent project's pom.xml file from Maven Central and thus the build is broken. And on Mac platform, the Hudson/Git integration appears to be not configured properly, that prevents us from being able to obtain the source code from our git repository. -Michael Even after updating the mac slave config, it still didn't seem to be able to find git, so I updated the build users shell settings and restarted the slave and at least from the 'script' section of my test build git seems happy. The other issue seemed to be the Mac was unable to actually reach git.eclipse.org, which should be fixed now. I've taken a look at the windows machine but but have any insight to offer. Firefox can browse the files you're trying to reach with maven, but I get the same error you do when I try to run the maven task on the command line. -M. Thanks, Matt. I can verify that the clone issue is now solved on the Mac. The build is executing at least now. |