Community
Participate
Working Groups
We only recently started to "pack200" our jar files in Orbit repos ... previously thinking it wasn't that important, since 80% of time (in theory) these jars are consumed (and packed) by Eclipse projects using them. But, some other changes made it easy to do (scripts use in WTP were put in place to do promotion and pack200 at same time). Just today, I thought wise to "verify" all signatures (of jars and pack.gz files) were correct. Turns out there were a number of problems reported. Most, but not all, were the "prebuilt" jars. Will attach "report" file soon. Need to investigate if I am "testing" these wrong (using a Java 6 JRE? ... which is best way to test, I think) or if something is wrong with our scripts or if something wrong with the specific jars involved. While not captured in error report (to be attached) they wrote the message "EOF reading archive magic number" to STDERR.
Created attachment 207504 [details] output from my "verifydir" script, ran on the most recent I-build repo
Created attachment 207506 [details] jars and pack.gz files that verified correctly In case a "comparison" might come in handy, here is the output of all files that verified correctly (during same run).
to cross-reference, bug 178723 is the original one where we discussed pros and cons of how to handle "prebuilt" bundles. Good reading for back ground, but no obvious solutions to fundamental issues. Bug 343416 and bug 191870 also talk about the procedure and a possible problem (that turned out not to be a problem).
# These three are not "prebuilt" bundles, so must have some special issue. # And, they do have error messages different than the prebuilt cases below, # java.lang.SecurityException: SHA1 digest error ... com.google.gwt.servlet_2.1.0.v20101206-1430.jar.pack.gz com.google.gwt.user_2.0.4.v20100709-0658.jar.pack.gz org.apache.jasper.glassfish_2.2.2.v201111151223.jar.pack.gz # This set of ICU bundles are match exactly the list of ICU bundles that are "pre built" com.ibm.icu.base.source_4.0.1.v20090822.jar.pack.gz com.ibm.icu.base.source_4.2.1.v20100412.jar.pack.gz com.ibm.icu.base.source_4.4.2.v20110831.jar.pack.gz com.ibm.icu.base_4.0.1.v20090822.jar.pack.gz com.ibm.icu.base_4.2.1.v20100412.jar.pack.gz com.ibm.icu.base_4.4.2.v20110831.jar.pack.gz com.ibm.icu.source_4.0.1.v20100112.jar.pack.gz com.ibm.icu.source_4.2.1.v20100412.jar.pack.gz com.ibm.icu.source_4.4.2.v20110823.jar.pack.gz com.ibm.icu_4.0.1.v20100112.jar.pack.gz com.ibm.icu_4.2.1.v20100412.jar.pack.gz com.ibm.icu_4.4.2.v20110823.jar.pack.gz # This is the interesting case ... all of these are indeed "prebuilt", but not # all of the prebuild sat4j bundles are problematic. Namely, the 2.1.0 and 2.3.0 # versions are "ok". org.sat4j.core_2.0.3.v20081021.jar.pack.gz org.sat4j.core_2.1.1.v20090825.jar.pack.gz org.sat4j.core_2.2.0.v20100429.jar.pack.gz org.sat4j.pb_2.0.3.v20081021.jar.pack.gz org.sat4j.pb_2.1.1.v20090825.jar.pack.gz org.sat4j.pb_2.2.0.v20100429.jar.pack.gz So, my theory about the "prebuilt" cases is that they are not actually "conditioned" as they should be. I thought we had a strong statement that prebuilt bundles must be, but all I could find was in http://wiki.eclipse.org/Orbit/Adding_Bundles_to_Orbit#Adding_a_library_that_is_already_a_bundle where we say (only) "we hope they are already conditioned". And, as corollary, my guess is the "sat4j" bundles that do not show up here as having problems, were in fact correctly preconditioned, so we can still sign them and still pack200 them.
DJ, Pascal, any insights? Advice? On the one hand, we could code up enough script logic that we not "pack200" them ... but ... then everyone would have to a) know that and b) do that. Another option ... if these are not contributed as already "conditioned", then IMHO we should NOT treat them as "pre-built". Would it hurt anything for use to explode them and add to to repo like usual? [Granted, at this point, I'm half guessing the problem is that they are not conditioned, we do not condition them, on purpose, but we still sign them, on purpose ... and if no one conditions them, then they can very be pack200'd. (And they do not have any "internal" info to say that ... people just have to know not to).
(In reply to comment #5) > .. then they can very be pack200'd. should be "... then they can never be pack200'd."
To be sure cross-references, I opened bug 364785 as a clone of this one, to address the three cases that are not "prebuilt". They do have different error messages and my guess is their "solution" will be different than how we solve these "prebuilt" cases.
For what its worth, I checked the sat4j core bundle, at 2.3.0 level, and it is not obviously "conditioned" ... meaning, it does not contain an eclipse.inf file saying it was ... though, there is no reason it'd have to if someone used a different procedure to condition it ... so, only original builder would know for sure. And ... might just be a coincidence that it packs and unpacks ok after eclipse signing (that is, "preconditioning" may not change it anyway so signing and packing would still work, by "chance"). Pascal, do you know original author well enough to find out if they "precondition" it?
After several hours of trying to get the <p2.remove.iu> task to work, to remove these, without success, I finally discovered that these "prebuilt" bundles in question have zero size on the file system AND (the significant part) are not part of any metadata. I'm using Eclipse 3.6.1 to do the p2 tasks ... maybe its smart enough to know zero size files don't need to go into meta data? The "verifydir" script I have, just literally works on the file system to verify signatures of jars and jar.pack.gz files. So, I don't think this problem is quite as bad as I first thought ... essentially, I'd just have to remove zero size files (and not worry about meta data). Of course, anyone else making the mistake of trying to "pack" these will have the same problem ... so still not sure we are using the best procedure for "prebuilt" bundles ... seems they don't accomplish much by "not touching them". I don't understand why we just don't explode them, and treat them like usual?
It was relatively easy to remove zero length pack.gz files from the plugins directory, in our "promote" script ... I did not even have to use p2.remove.ui since these zero length artifacts are not mentioned there in artifacts.jar/xml (nor content.jar/xml file). So ... let's count this issue, for these "prebuilt" bundles are resolved. We already have bug 310183 to discuss changing the way we handle "prebuilt" bundles ... whether or not to condition, etc. ... so we'll continue the high level discussion there.
