Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 319012 - [releng] specify different user for signing than initiating build id
Summary: [releng] specify different user for signing than initiating build id
Status: RESOLVED FIXED
Alias: None
Product: e4
Classification: Eclipse Project
Component: UI (show other bugs)
Version: unspecified   Edit
Hardware: PC Windows XP
: P3 normal (vote)
Target Milestone: 1.0 RC3   Edit
Assignee: Kim Moir CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-07-06 10:25 EDT by Kim Moir CLA
Modified: 2010-07-28 10:11 EDT (History)
4 users (show)

See Also:


Attachments
patch (7.59 KB, patch)
2010-07-06 10:43 EDT, Kim Moir CLA
no flags Details | Diff
patch (1.15 KB, patch)
2010-07-12 10:20 EDT, Kim Moir CLA
no flags Details | Diff
patch (1.18 KB, patch)
2010-07-12 11:40 EDT, Kim Moir CLA
no flags Details | Diff
patch (1.44 KB, patch)
2010-07-13 11:20 EDT, Kim Moir CLA
no flags Details | Diff
patch (1.72 KB, patch)
2010-07-19 08:39 EDT, Kim Moir CLA
no flags Details | Diff
patch (1.72 KB, patch)
2010-07-26 11:49 EDT, Kim Moir CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Kim Moir CLA 2010-07-06 10:25:53 EDT
Paul would like to specify a different user for the signing process than the id used to initiate the build.  Currently the build assumes that the build id and signing user are identical.
Comment 1 Kim Moir CLA 2010-07-06 10:43:11 EDT
Created attachment 173563 [details]
patch

Patch to specify signing user. Alternatively, we could specify the user in startup script and extract it to pass to the signing task.
Comment 2 Kim Moir CLA 2010-07-12 08:55:21 EDT
Paul, I've applied the patch. I don't know if you need me to tag the builder or if you just run it from the R4_HEAD branch.
Comment 3 Paul Webster CLA 2010-07-12 08:57:08 EDT
(In reply to comment #2)
> Paul, I've applied the patch. I don't know if you need me to tag the builder or
> if you just run it from the R4_HEAD branch.

Thanx Kim.

I just run eclipsebuilder from R4_HEAD
PW
Comment 4 Paul Webster CLA 2010-07-12 09:45:29 EDT
Kim, I get part way and then it stops asking for password:, because I don't specify hudson.

Is it trying to check out something from CVS as someone other than anonymous if I don't specify hudson?

I realize I'm the worst case scenario  :-)  Running on build.eclipse.org so that I can use local as much as possible, as e4Build, but he can take advantage of ssh to pwebster for anything that needs a committer id.

PW
Comment 5 Paul Webster CLA 2010-07-12 09:51:22 EDT
[CVS - org.eclipse.sdk.examples] cvs export: Updating org.eclipse.sdk.examples
[CVS - org.eclipse.sdk.examples] cvs export: Updating org.eclipse.sdk.examples/META-INF
      [cvs] cvs [export aborted]: received termination signal

This was as far as it got.

PW
Comment 6 Kim Moir CLA 2010-07-12 09:59:16 EDT
Yes, if you don't specify hudson, the build assumes that you are using extssh to check out code.

Right now it's 

<property name="cvsuser" value="kmoir" />

I can change this to pwebster in R4_HEAD.  Are the ssh keys set up for this?
Comment 7 Paul Webster CLA 2010-07-12 10:03:19 EDT
Yes, e4Build has pwebster setup.

If you commit the change I can continue to use bootstrapHudsone4.sh 

Thanx,
PW
Comment 8 Kim Moir CLA 2010-07-12 10:20:19 EDT
Created attachment 174024 [details]
patch
Comment 9 Kim Moir CLA 2010-07-12 10:22:29 EDT
patch applied to change cvsuser to pwebster
Comment 10 Paul Webster CLA 2010-07-12 11:25:21 EDT
Hi Kim, could we try and break up the running on build.eclipse.org from the userid used for signing?

without -hudson, I get
I201007121100/src/fetch_master-ecf.xml:50: Error occurred while transforming repository: Unknown Host: http://fullmoon.ottawa.ibm.com/tools/orbit/downloads/drops/R20100519200754/repository/artifacts.xml

but with -hudson it won't use ssh/scp for signing.

PW
Comment 11 Kim Moir CLA 2010-07-12 11:40:32 EDT
Created attachment 174050 [details]
patch

Patch to avoid fetching from ibm mirror when running as a non-hudson build has been applied.
Comment 12 Paul Webster CLA 2010-07-12 19:10:54 EDT
I get a lot of: 
     [echo] original:  ${originalAttribs}
     [echo] polled:  Error: Can't open display: :0.0
     [echo] Error: Can't open display: :0.0
     [echo] Error: Can't open display: :0.0
     [echo] Error: Can't open display: :0.0
     [echo] Permission denied, please try again.
     [echo] Error: Can't open display: :0.0
     [echo] Permission denied, please try again.
     [echo] Error: Can't open display: :0.0
     [echo] Received disconnect from 206.191.52.34: 2: Too many authentication failures for e4Build
     [exec] Result: 255

Kim, could you please check out my build:
http://build.eclipse.org/eclipse/e4/build/e4/downloads/drops/4.0.0/workspace/I201007121521/

Console log:
http://build.eclipse.org/eclipse/e4/build/e4/downloads/drops/4.0.0/workspace/sdk1521.txt

Technically e4Build has :8.0 available, if that will make it run better.

PW
Comment 13 Kim Moir CLA 2010-07-13 10:56:00 EDT
I'm running my own command line build to try to determine the source of the problem.  Not sure why this is happening.
Comment 14 Kim Moir CLA 2010-07-13 11:20:30 EDT
Created attachment 174164 [details]
patch

A patch to remove setting xhost and display and removing \ in front of @ sign for signing user.  I tested this with my userid on build.eclipse.org and it seems to work. The patch has been applied to R4_HEAD.
Comment 15 Paul Webster CLA 2010-07-14 12:24:44 EDT
This stopped again, asking for a password:

     [exec]     zip warning: name not matched: hpux**
     [exec]     zip warning: name not matched: macosx**
     [exec]     zip warning: name not matched: qnx**
     [exec]     zip warning: eclipse-master-I20100714-0926.zip not found or empty
     [exec] 
     [exec] zip error: Nothing to do! (eclipse-master-I20100714-0926.zip)
     [exec] Result: 12
Duplicated project name in import. Project Build specific targets and properties defined first in /opt/public/eclipse/e4/build/e4/downloads/drops/4.0.0/workspace/builds/I201007140926/src/plugins/org.eclipse.pde.doc.user/customBuildCallbacks.xml and again in /opt/public/eclipse/e4/build/e4/downloads/drops/4.0.0/workspace/builds/I201007140926/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.6.0.v20100603/templates/plugins/customBuildCallbacks.xml
Duplicated project name in import. Project Build specific targets and properties defined first in /opt/public/eclipse/e4/build/e4/downloads/drops/4.0.0/workspace/builds/I201007140926/src/plugins/org.eclipse.pde.doc.user/customBuildCallbacks.xml and again in /opt/public/eclipse/e4/build/e4/downloads/drops/4.0.0/workspace/builds/I201007140926/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.6.0.v20100603/templates/plugins/customBuildCallbacks.xml
     [echo] push drop to staging directory
Password:

Build directory http://build.eclipse.org/eclipse/e4/build/e4/downloads/drops/4.0.0/workspace/builds/I201007140926

PW
Comment 16 Paul Webster CLA 2010-07-14 12:32:12 EDT
it looks like it was run with builderTag=r4_v20100712 which doesn't include your buildAll.xml changes.

PW
Comment 17 Kim Moir CLA 2010-07-14 13:44:23 EDT
Sorry Paul, I have tagged it.
Comment 18 Paul Webster CLA 2010-07-15 08:05:29 EDT
Today's test failed with:

/opt/public/eclipse/e4/build/e4/downloads/drops/4.0.0/workspace/builds/I201007150750/src/fetch_master-ecf.xml:10: The following error occurred while executing this line:
/opt/public/eclipse/e4/build/e4/downloads/drops/4.0.0/workspace/builds/I201007150750/src/fetch_master-ecf.xml:27: Can't get http://www.eclipse.org/external/rt/ecf/integration/platform/v2010-05-29_00-32-38/plugins/org.eclipse.ecf_3.1.0.v20100529-0735.jar to /opt/public/eclipse/e4/build/e4/downloads/drops/4.0.0/workspace/builds/I201007150750/src/plugins/org.eclipse.ecf_3.1.0.v20100529-0735.jar

Is it because ECF has cleaned up something?

PW
Comment 19 Kim Moir CLA 2010-07-15 08:12:11 EDT
I can fetch that ECF jar now.  I find that sometimes eclipse.org times out and then I can't fetch the jars.  The only solution is to restart the build where it failed....
Comment 20 Paul Webster CLA 2010-07-15 15:23:30 EDT
I re-ran /opt/public/eclipse/e4/build/e4/downloads/drops/4.0.0/workspace/builds/I201007151054

It got up to the signing step again, but then failed.

     [exec] zip error: Nothing to do! (eclipse-master-I20100715-1054.zip)
     [exec] Result: 12
Duplicated project name in import. Project Build specific targets and properties defined first in /opt/public/eclipse/e4/build/e4/downloads/drops/4.0.0/workspace/builds/I201007151054/src/plugins/org.eclipse.pde.doc.user/customBuildCallbacks.xml and again in /opt/public/eclipse/e4/build/e4/downloads/drops/4.0.0/workspace/builds/I201007151054/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.6.0.v20100603/templates/plugins/customBuildCallbacks.xml
Duplicated project name in import. Project Build specific targets and properties defined first in /opt/public/eclipse/e4/build/e4/downloads/drops/4.0.0/workspace/builds/I201007151054/src/plugins/org.eclipse.pde.doc.user/customBuildCallbacks.xml and again in /opt/public/eclipse/e4/build/e4/downloads/drops/4.0.0/workspace/builds/I201007151054/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.6.0.v20100603/templates/plugins/customBuildCallbacks.xml
     [echo] push drop to staging directory
     [exec] Result: 1
     [exec] Result: 1
     [echo] invoke sign script and wait
Password: 


PW
Comment 21 Paul Webster CLA 2010-07-15 15:30:09 EDT
Where will it try and place the zips for signing?

/home/data/httpd/download-staging.priv/eclipse ?

I don't have write permission for that directory (neither pwebster nor e4Build) but I do for /home/data/httpd/download-staging.priv/eclipse/e4

PW
Comment 22 Kim Moir CLA 2010-07-15 16:24:59 EDT
I think we need to update stagingDirectoryOutput to point to this directory. I'll do that and attach a patch. You can just hack the existing buildAll.xml to point to call to the signing task so you don't have to recompile everything.  Then run command.txt again.
Comment 23 Kim Moir CLA 2010-07-19 08:39:38 EDT
Created attachment 174609 [details]
patch

patch released and tagged
Comment 24 Paul Webster CLA 2010-07-19 17:59:39 EDT
The zip has been signed in :
/home/data/httpd/download-staging.priv/eclipse/e4/I20100719-1400-out


But my build is busy compiling tests and complaining about:

     [exec] Result: 255
     [echo] original:  ${originalAttribs}
     [echo] polled:  Permission denied, please try again.
     [echo] Permission denied, please try again.
     [echo] Received disconnect from 206.191.52.34: 2: Too many authentication failures for e4Build

Build directory:

/opt/public/eclipse/e4/build/e4/downloads/drops/4.0.0/workspace/builds/I201007191400
Comment 25 Paul Webster CLA 2010-07-19 18:06:42 EDT
This is in the sign call:

${signUser}${signhost}

Does that have to be in the signpolline as well?

PW
Comment 26 Paul Webster CLA 2010-07-23 06:03:58 EDT
I ran the eclipsebuilder 4.0 last night, and it appears http://download.eclipse.org/e4/sdk/drops/I20100722-2038/index.php is signed.

But the zips are about 27M light.  For some reason 

org.eclipse.platform.doc.isv 35315809
org.eclipse.platform.doc.isv 6434627

PW
Comment 27 Paul Webster CLA 2010-07-23 15:55:02 EDT
Chris, when we ship doc have we pre-processed or pre-indexed it?

PW
Comment 28 Chris Goldthorpe CLA 2010-07-26 11:13:09 EDT
(In reply to comment #27)
> Chris, when we ship doc have we pre-processed or pre-indexed it?
> 
> PW

The docs are preindexed at build time and there is a JUnit test in org.eclipse.ua.tests.doc  which verifies that the index is build to it is easy to verify if that step has happened.
Comment 29 Kim Moir CLA 2010-07-26 11:35:33 EDT
It looks like the new platform.doc.isv bundles are missing the reference/api directory.
Comment 30 Kim Moir CLA 2010-07-26 11:49:03 EDT
Created attachment 175230 [details]
patch

The path to the javadoc binary the 4.0 build's bootstrap script was incorrect. I've fixed this and tagged it for the next build.
Comment 31 Paul Webster CLA 2010-07-28 10:11:02 EDT
It's all good.

PW