Bugzilla – Full Text Bug Listing
| Summary: | Cannot sign with Buckminster on hudson.eclipse.org | ||
|---|---|---|---|
| Product: | Community | Reporter: | Nicolas Bros <nicolas.bros> |
| Component: | CI-Jenkins | Assignee: | Eclipse Webmaster <webmaster> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | normal | ||
| Priority: | P3 | CC: | thomas |
| Version: | unspecified | ||
| Target Milestone: | --- | ||
| Hardware: | All | ||
| OS: | All | ||
| Whiteboard: | |||
|
Description
Nicolas Bros
Are you sure this is being run inside Hudson? (In reply to comment #1) > Are you sure this is being run inside Hudson? Yes, I am running the build under Hudson, and calling Buckminster from a shell script defined in the Hudson job: buckminster="/opt/public/modeling/mdt/modisco/buckminster/install/buckminster --loglevel info -data ${WORKSPACE}/buildroot/buckminster.workspace -consolelog" $buckminster setpref targetPlatformPath="${WORKSPACE}/buildroot/target.platform" $buckminster import -Declipse.download="file:/home/data/httpd/download.eclipse.org" "${WORKSPACE}/org.eclipse.gmt.modisco.releng.buckminster/modisco.mspec" $buckminster resolve "${WORKSPACE}/org.eclipse.gmt.modisco.releng.buckminster/modisco.cquery" $buckminster build $buckminster junit -DX.display="${DISPLAY}" --launch "org.eclipse.gmt.modisco.tests/AllTests.launch" --flatXML --output "${WORKSPACE}/buildroot/test.results/xml/org.eclipse.mdt.modisco.tests.xml" $buckminster perform --properties "${WORKSPACE}/buildroot/build.properties" "org.eclipse.gmt.modisco.all#site.p2" Which job is this? Can you add a link to the build log? Or better still can you have this buckminster instance dump it's environment so we can check the user/group? -M. (In reply to comment #3) > Which job is this? https://hudson.eclipse.org/hudson/job/modisco-integration/ > Can you add a link to the build log? I deleted the job artifacts recently to try and reduce disk space usage. I'll attach a build log next time. > Or better still can you have this buckminster instance dump its > environment so we can check the user/group? Here's the result of running "env" from the Job. I assume Buckminster inherits this environment: LESSKEY=/etc/lesskey.bin NNTPSERVER=news MANPATH=/usr/local/man:/usr/local/share/man:/usr/share/man INFODIR=/usr/local/info:/usr/share/info:/usr/info BUILD_URL=https://hudson.eclipse.org/hudson/job/modisco-integration/67/ HOSTNAME=build XKEYSYMDB=/usr/share/X11/XKeysymDB SHELL=/bin/bash HOST=hudson TERM=xterm PROFILEREAD=true HISTSIZE=1000 BUILD_TAG=hudson-modisco-integration-67 PROJRELENGROOT=svn://dev.eclipse.org/svnroot/modeling/org.eclipse.mdt.modisco/releng/trunk QT_HOME_DIR=/usr/share/desktop-data MORE=-sl WORKSPACE=/opt/users/hudsonbuild/.hudson/jobs/modisco-integration/workspace JOB_URL=https://hudson.eclipse.org/hudson/job/modisco-integration/ ANT_OPTS=-Dhttp.proxyHost=206.191.52.34 -Dhttp.proxyPort=9898 -Dhttps.proxyHost=206.191.52.34 -Dhttps.proxyPort=9898 -Dhttp.nonProxyHosts=*.eclipse.org -Dhttps.nonProxyHosts=*.eclipse.org USER=hudsonbuild http_proxy=http://206.191.52.34:9898 LS_COLORS=no=00:fi=00:di=01;34:ln=00;36:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=41;33;01:ex=00;32:*.cmd=00;32:*.exe=01;32:*.com=01;32:*.bat=01;32:*.btm=01;32:*.dll=01;32:*.tar=00;31:*.tbz=00;31:*.tgz=00;31:*.rpm=00;31:*.deb=00;31:*.arj=00;31:*.taz=00;31:*.lzh=00;31:*.lzma=00;31:*.zip=00;31:*.zoo=00;31:*.z=00;31:*.Z=00;31:*.gz=00;31:*.bz2=00;31:*.tb2=00;31:*.tz2=00;31:*.tbz2=00;31:*.avi=01;35:*.bmp=01;35:*.fli=01;35:*.gif=01;35:*.jpg=01;35:*.jpeg=01;35:*.mng=01;35:*.mov=01;35:*.mpg=01;35:*.pcx=01;35:*.pbm=01;35:*.pgm=01;35:*.png=01;35:*.ppm=01;35:*.tga=01;35:*.tif=01;35:*.xbm=01;35:*.xpm=01;35:*.dl=01;35:*.gl=01;35:*.wmv=01;35:*.aiff=00;32:*.au=00;32:*.mid=00;32:*.mp3=00;32:*.ogg=00;32:*.voc=00;32:*.wav=00;32: LD_LIBRARY_PATH=/opt/public/common/sun-jdk1.6.0_21_x64/jre/lib/amd64/server:/opt/public/common/sun-jdk1.6.0_21_x64/jre/lib/amd64:/opt/public/common/sun-jdk1.6.0_21_x64/jre/../lib/amd64 XNLSPATH=/usr/share/X11/nls ENV=/etc/bash.bashrc HOSTTYPE=x86_64 USERNAME=hudsonbuild PAGER=less CSHEDIT=emacs JVM_OPTS=-Dhttp.proxyHost=206.191.52.34 -Dhttp.proxyPort=9898 -Dhttps.proxyHost=206.191.52.34 -Dhttps.proxyPort=9898 -Dhttp.nonProxyHosts=*.eclipse.org -Dhttps.nonProxyHosts=*.eclipse.org XDG_CONFIG_DIRS=/etc/xdg MINICOM=-c on NLSPATH=/usr/dt/lib/nls/msg/%L/%N.cat PATH=/shared/common/jdk-1.5.0-22.x86_64/bin:/shared/common/apache-ant-1.7.1/bin:/shared/common/sun-jdk1.6.0_21_x64/bin/:/opt/users/hudsonbuild/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/opt/kde3/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin MAIL=/var/spool/mail/hudsonbuild CPU=x86_64 _=/usr/bin/env INPUTRC=/opt/users/hudsonbuild/.inputrc PWD=/opt/public/jobs/modisco-integration/workspace JAVA_HOME=/shared/common/sun-jdk1.6.0_21_x64 HUDSON_URL=https://hudson.eclipse.org/hudson/ LANG=en_US.UTF-8 VERSION=0.9.0 JOB_NAME=modisco-integration PYTHONSTARTUP=/etc/pythonstart XFILESEARCHPATH=/usr/dt/app-defaults/%L/Dt BUILD_TYPE=I BUILD_ID=2010-09-09_10-44-32 https_proxy=http://206.191.52.34:9898 JAVA_ARGS=-Dhttp.proxyHost=206.191.52.34 -Dhttp.proxyPort=9898 -Dhttps.proxyHost=206.191.52.34 -Dhttps.proxyPort=9898 -Dhttp.nonProxyHosts=*.eclipse.org -Dhttps.nonProxyHosts=*.eclipse.org HOME=/opt/users/hudsonbuild SHLVL=3 OSTYPE=linux LESS_ADVANCED_PREPROCESSOR=no no_proxy=localhost, 127.0.0.1, 172.30.206.0, eclipse.org LS_OPTIONS=-N --color=tty -T 0 XCURSOR_THEME=DMZ ANT_ARGS=-Dhttp.proxyHost=206.191.52.34 -Dhttp.proxyPort=9898 -Dhttps.proxyHost=206.191.52.34 -Dhttps.proxyPort=9898 -Dhttp.nonProxyHosts=*.eclipse.org -Dhttps.nonProxyHosts=*.eclipse.org EXECUTOR_NUMBER=1 WINDOWMANAGER=/usr/bin/gnome BASH_ENV=/opt/users/hudsonbuild/.bashrc LOGNAME=hudsonbuild LESS=-M -I MACHTYPE=x86_64-suse-linux G_FILENAME_ENCODING=@locale,UTF-8,ISO-8859-15,CP1252 CVS_RSH=ssh XDG_DATA_DIRS=/usr/share:/etc/opt/kde3/share:/opt/kde3/share SIGN_UPDATE_SITE=true HUDSON_HOME=/opt/users/hudsonbuild/.hudson LESSOPEN=lessopen.sh %s BUILD_NUMBER=67 INFOPATH=/usr/local/info:/usr/share/info:/usr/info HUDSON_COOKIE=9e12d60a-cd0a-4fde-811f-fe5db0fc3cf2 DISPLAY=:10 LESSCLOSE=lessclose.sh %s %s G_BROKEN_FILENAMES=1 COLORTERM=1 > Here's the result of running "env" from the Job. I assume Buckminster inherits > this environment: I would presume so. > USERNAME=hudsonbuild Well that certainly implies it's running with the correct user id. I think the issue may actually be bad group ownership. I'll try a better fix for this by restarting hudson when the current jobs end. -M. Signing still doesn't work for me. When /usr/bin/sign is called by Buckminster from Hudson, I get: java.io.IOException: /usr/bin/sign: cannot execute But I tried running "/usr/bin/sign" directly from Hudson (in a Job's shell script) and it started with no error: Usage: /usr/bin/sign <file> <mail|nomail> [outputDir] Signs JAR or ZIP file <file>, placing the resulting file in optional [outputDir] and mails notification that signing is complete, if requested. I don't understand why. Do you have an idea? Could it be caused by this old Java bug? http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4052517 In /etc/passwd: hudsonbuild:x:55011:8411:Hudson:/opt/users/hudsonbuild:/bin/bash So hudsonbuild's primary group is 8411 (callisto-dev). Confirmed by /etc/group: callisto-dev:x:8411:hudsonbuild And when running "groups hudsonbuild" from Hudson, I get: hudsonbuild : callisto-dev Shouldn't I see the "signers" group here too? Indeed. This happened to us in bug 313722. I've applied the same fix -- can you verify if it works? It is getting further, though I don't know whether it's thanks to your fix or mine (I just tried modifying the ant script to call /bin/sh on /usr/bin/sign instead of calling it directly)
/usr/bin/sign starts, but I receive an error (in /home/data/httpd/download-staging.priv/arch/signer.log):
2010-09-24 11:26:01 build QUEUE: Begin processing queue: 7819
1285341904:hudsonbuild:/opt/users/hudsonbuild/workspace/modisco-nightly/buildroot/eclipse.staging/site_608056616.zip:nomail:/opt/users/hudsonbuild/workspace/modisco-nightly/buildroot/eclipse.staging/out20100924152501
======================= end of (7819) contents
2010-09-24 11:26:03: build QUEUE(7819): calling jarprocessor.jar for /opt/users/hudsonbuild/workspace/modisco-nightly/buildroot/eclipse.staging/site_608056616.zip
java.io.FileNotFoundException
at org.eclipse.update.internal.jarprocessor.JarProcessor.process(JarProcessor.java:80)
at org.eclipse.update.internal.jarprocessor.Main.runJarProcessor(Main.java:168)
at org.eclipse.update.internal.jarprocessor.Main.main(Main.java:184)
2010-09-24 11:26:03: build QUEUE(7819): Finished processing queue.
When I look at my workspace through Hudson, I see site_608056616.zip:
https://hudson.eclipse.org/hudson/job/modisco-nightly/ws/buildroot/eclipse.staging/
You have to move/copy your build artifacts to the download staging area, since the signing occurs on the build server and cannot access the (local) hudson workspaces. On any Hudson node, simply copy/move files to be signed to /shared/download-staging.priv/(your project directory) I've updated /usr/bin/sign on all the hudson nodes so that it fails if you're signing from the hudson workspace. It will provide a hint as to where to sign. (In reply to comment #11) > On any Hudson node, simply copy/move files to be signed to > /shared/download-staging.priv/(your project directory) There doesn't seem to be a project directory for MoDisco. Could you please create "/shared/download-staging.priv/modeling/mdt/modisco/", owned by group "modeling.mdt.modisco"? I've created the directory for you. I can't wait to see your builds succeed! I get an error: ERROR: File /shared/download-staging.priv/modeling/mdt/modisco/site_134341880.zip is not in the downloads staging area. Not signing from unknown location. "/shared/download-staging.priv" doesn't appear in the valid locations in /usr/bin/sign: VALIDLOCATION=$(echo "$DIR" | egrep -c "(\/opt\/public\/download-staging.priv\/|\/home\/data\/httpd\/download-staging.priv|\/opt\/users\/hudsonbuild\/)") I've changed my staging directory to "/home/data/httpd/download-staging.priv/...", and it works now. Thanks for your help! I've added /shared/download-staging.priv to the list of valid locations. |