Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 357056

Summary: jar files in maintenance repo (3.3.1) appear not "packed"
Product: [WebTools] WTP Releng Reporter: David Williams <david_williams>
Component: relengAssignee: Carl Anderson <ccc>
Status: RESOLVED FIXED QA Contact: Carl Anderson <ccc>
Severity: major    
Priority: P3    
Version: unspecified   
Target Milestone: ---   
Hardware: PC   
OS: Linux   
Whiteboard:
Attachments:
Description Flags
console output from test promote run
none
Carl's log from promote attempt with permissions problems. none

Description David Williams CLA 2011-09-08 06:50:14 EDT
While investigating another bug, I noticed the jar files in webtools maintenance repositories on downloads have not been processed with pack200. For example, 

/home/data/httpd/download.eclipse.org/webtools/downloads/drops/R3.3.1/M-3.3.1-20110906122954/repository/plugins


I think maybe you've been forgetting the '-z' options in the ./promote.sh command?
Comment 1 Carl Anderson CLA 2011-09-08 09:19:37 EDT
From /shared/webtools/releng.control I run

./promote.sh -vdsczap wtp-R3.3.1-M

So unless the order is wrong, I am not forgetting the -z parameter.
Comment 2 David Williams CLA 2011-09-08 09:54:59 EDT
Order of commands shouldn't matter, but something is amiss. Maybe a bug in 'promote.sh'? You run these from canderson id, I assume (I think it'd fail completely if not). 

Maybe you could run one and capture output (say, with ... 2>&1 | tee ~/promoteout.txt ) attach it, and I could look at output and/or compare with one from when I run it, to see if there's something in my "path" that causes a difference in behaviour? 

Or, maybe promote.sh is more obviously broken ... I think I'll try a run (with no -s) and see if results are any different. 

Thanks,
Comment 3 David Williams CLA 2011-09-08 13:10:59 EDT
Created attachment 203012 [details]
console output from test promote run

I tried "promote.sh" from my login, and it seems to have worked fine. So, I suspect there is something about 'canderson' setup that's not right. Of course, the script should actually fail if it can no do what's requested, so I'll see if there's any each "return code" and "file existence" checks that can be done.
Comment 4 David Williams CLA 2011-09-08 15:30:45 EDT
I've added some checking and more output for these files involved in promoting: 


addRepoProperties.sh    
promote.sh 
runAntRunner.sh 

But ... I really do not see anything that could go wrong, that would not show up already as some sort of error in the output, so, I (still) recommend running to capture the output. 

Since there are builds running, we can not "get-relengTools" like usual, to get these three new files ... but, I will "surgically" put them on build server, still under releng.control, so you can be sure to use them next time you promote. 

Let me know what output you get ... or, if errors now show up.
Comment 5 David Williams CLA 2011-09-09 02:43:21 EDT
Carl and I resolved this via IM tonight. He had an extra parameter file on this path that was throughing things off .. we never were quite sure how ... but all works without that file (though, he'll need to test patch builds, and make sure there's not something those older buiders need ... but, I don not think so.
Comment 6 David Williams CLA 2011-09-15 13:02:12 EDT
reopening, since still doesn't seem right, looking at latest "pack200" report on indigo site

http://build.eclipse.org/indigo/simrel/
or more specifically
http://build.eclipse.org/indigo/simrel/reports/pack200data.txt

This time, seems for completely different reasons, having to do with "permission" problems ... so, I opened bug 357834 for that. 

But ... I'm reopening for two reasons 1) once permission problem is fixed, then we should try to re-run, but 2) I was surprised the "promote.sh" script did not fail on the "eclipse error" that was reported from p2. ... so, I think should investigate that too. See if that script can't be made more bullet proof.
Comment 7 David Williams CLA 2011-09-15 13:05:12 EDT
Created attachment 203425 [details]
Carl's log from promote attempt with permissions problems.

To fix correctly, might have to change the "promote task" in Java code (to return non zero error code) ... in which case that effort should be "branched" to new bug, once permission problems are fixed.
Comment 8 David Williams CLA 2011-09-15 20:32:09 EDT
Appears our promote script now works correctly, again, now that some file permission problems have been corrected. See bug 357834. I opened bug 357896 to track improvements to promote script to more obviously fail in conditions like this, where requested p2 operations fail.
Comment 9 David Williams CLA 2011-09-15 21:46:42 EDT
I don't know what is it ... but seems "final bits" still are not getting pack.gz files correctly, according to "indigo report" at 

http://build.eclipse.org/indigo/simrel/reports/pack200data.txt
Comment 10 David Williams CLA 2011-09-15 22:05:26 EDT

I looked at several locations "manually" and see they each have consistent number of /plugins/*.wst.*.jar files and /plugins/*.wst.*.jar.pack.gz files ... so ... I suspect something else is now the problem. (109 or 197 depending of if 'source' in repo). 

Ah, and re-looking, I see now the report no longer lists numerous 
*.wst.* pack.gz files missing ... so, I think I was looking at an old or cached copy.