Community
Participate
Working Groups
admittedly, I was using an old version, and have not tried a more recent version ... but, wanted to capture this in case still an issue. In bug 348028 we in WTP found we had a large jar file, that was causing pack200 to through an out of memory exception. But, there was not error or "build failure" from ant, as I would have thought (we use the p2.process.artifacts task, with the pack="true" attribute. From the dump file left on the file system, the "command under the covers" was IBM_JAVA_COMMAND_LINE=/opt/public/webtools/apps/ibm-java2-sdk-5.0-12.1-linux-i386/jre/bin/pack200 -E4 /opt/public/webtools/committers/wtp-R3.3.0-S/20110527214303/S-3.3.0RC3-2011052 7214303/repository/plugins/org.eclipse.jpt.doc.isv_2.0.0.v201105240000.jar.pack.gz /opt/public/webtools/committers/wtp-R3.3.0-S/20110527214303/S-3.3.0RC3-20110527214303/repository/plugins/temp.or g.eclipse.jpt.doc.isv_2.0.0.v201105240000.jar/org.eclipse.jpt.doc.isv_2.0.0.v201105240000.jar My guess is that pack200 does not return a proper error code ... but, wanted to enter this bug before I forgot, in case there is some improvements in error processing that can be made. The best outcome would be to return a "build failed" or at least let the ant task capture the error code. Apologies in advance if this has changed since 3.6.1.
Andrew could you please look into this?
Move all 3.8 bugs to Juno.
Still seems valid, and simple. This was not fixed for Juno. I'll put in on Kepler since there is a pretty simple path forward for this.
Created attachment 225159 [details] beginning of one possible approach to address this issue These are changes that better handle the case where the external pack200 call fails. I haven't really dealt with the ant case ... having trouble remembering how to run newly written ant tasks that are also in the runtime target (i.e. by attempts to run them, just runs the old version, instead of the one in my workspace, with changes). Anyway, hoping this much of a patch will get some attention on this bug and maybe we can fix this and bug 361628 at the same time. I say "same time", meaning, once we get something new we are happy with, we should get Dennis to set up a "testsign" script, instead of "sign" then we (and others) can use it a few weeks to make sure no surprises before we make it official, to be used in releases, etc.). The goal (my goal) in what it means to "fix" this is to a) if there is a bad error, like "out of memory", to not hide that just because verbose is false. Or, as I solved it, give a brief message there was a problem, and to run with -verbose, for more details. Second, I'd like the ability to say if something like that occurs to have the ability to "fail the build" and just stop everything until investigated and decision made on how to handle. (The default would still have to be "failonerror='false'" to be compatible with current behavior). run instructions and example output in next post.
Created attachment 225160 [details] 13 Meg file that started this whole thing I don't see the "huge attachment" option anymore, so assume its automatic? But likely be be removed at some point if over 10 Megs?
So, to run the code to see the changes, so far, in action, from workbench, you can run org.eclipse.equinox.p2.jarprocessor with options similar to (use your own path, for where you put the huge jar). -processAll -repack /home/davidw/testHugeBuildArtifacts/olddoc/org.eclipse.jpt.doc.isv_2.0.0.v201105240000.jar If you run it before applying the patch, you'd see that "nothing happens" ... no output. No indication of errors. If you run it after the patch, with same arguments above, you'd see as output in console: An error occured during pack200 processing. Run with -verbose for more details. Now, if you run with -verbose, you'd see output similar to what you would have seen before: Running Repack on /home/davidw/testHugeBuildArtifacts/olddoc/org.eclipse.jpt.doc.isv_2.0.0.v201105240000.jar org.eclipse.jpt.doc.isv_2.0.0.v201105240000.jar is included in pack200 by default for main jars. STDERR: JVMDUMP006I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" - please wait. STDERR: JVMDUMP032I JVM requested Snap dump using '/home/davidw/git/rt.equinox.p2/bundles/org.eclipse.equinox.p2.jarprocessor/Snap.20130103.022901.3213.0001.trc' in response to an event STDERR: JVMDUMP010I Snap dump written to /home/davidw/git/rt.equinox.p2/bundles/org.eclipse.equinox.p2.jarprocessor/Snap.20130103.022901.3213.0001.trc STDERR: JVMDUMP032I JVM requested Heap dump using '/home/davidw/git/rt.equinox.p2/bundles/org.eclipse.equinox.p2.jarprocessor/heapdump.20130103.022901.3213.0002.phd' in response to an event STDERR: JVMDUMP010I Heap dump written to /home/davidw/git/rt.equinox.p2/bundles/org.eclipse.equinox.p2.jarprocessor/heapdump.20130103.022901.3213.0002.phd STDERR: JVMDUMP032I JVM requested Java dump using '/home/davidw/git/rt.equinox.p2/bundles/org.eclipse.equinox.p2.jarprocessor/javacore.20130103.022901.3213.0003.txt' in response to an event STDERR: JVMDUMP010I Java dump written to /home/davidw/git/rt.equinox.p2/bundles/org.eclipse.equinox.p2.jarprocessor/javacore.20130103.022901.3213.0003.txt STDERR: JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError". STDERR: Exception in thread "main" java.lang.OutOfMemoryError STDERR: at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:116) STDERR: at java.io.ByteArrayOutputStream.writeTo(ByteArrayOutputStream.java:133) STDERR: at com.sun.java.util.jar.pack.Package$File.writeTo(Package.java:784) STDERR: at com.sun.java.util.jar.pack.PackageWriter.writeFiles(PackageWriter.java:701) STDERR: at com.sun.java.util.jar.pack.PackageWriter.write(PackageWriter.java:66) STDERR: at com.sun.java.util.jar.pack.PackerImpl$DoPack.flushPackage(PackerImpl.java:596) STDERR: at com.sun.java.util.jar.pack.PackerImpl$DoPack.flushAll(PackerImpl.java:550) STDERR: at com.sun.java.util.jar.pack.PackerImpl$DoPack.run(PackerImpl.java:498) STDERR: at com.sun.java.util.jar.pack.PackerImpl.pack(PackerImpl.java:91) STDERR: at com.sun.java.util.jar.pack.Driver.main(Driver.java:279) Error: 1 was returned from command: /usr/local/jdks/ibm-java2-x86_64-50-SR13FP1/jre/bin/pack200 -r -E4 /home/davidw/git/rt.equinox.p2/bundles/org.eclipse.equinox.p2.jarprocessor/temp_org.eclipse.jpt.doc.isv_2.0.0.v201105240000.jar /home/davidw/testHugeBuildArtifacts/olddoc/org.eclipse.jpt.doc.isv_2.0.0.v201105240000.jar At this point, if you had a bunch of jars, processing would continue, and you'd have to know to go look at "signer log" to see the error. If you use -failonerror (new parameter) then the build would stop, and throw a RuntimeException. -- oh, I see that's not working. In my haste, must have forgotten something. If its obvious, I'll reattach working patch.
Created attachment 225198 [details] new patch, with simpler solution I've changed my mind ... or, given up? :) ... on trying to add a "failonerror" option. I'm sure its possible, but ... its become obvious to me its would take more work than I'm able to give it. The current "options" are handled in confusing, inconsistent ways and given priorities, I'd be happy with a tiny improvement here. The tiny improvement provided by this patch is that whether or not "verbose" is set, and error in the external command will also produce some message to log (or, "system.out" to be exact) and if verbose is not set, a reminder than -verbose will provide more details. So, for this test case, where this huge jar runs out of memory, running without -verbose, gives following output: Error: 1 was returned from command: /usr/local/jdks/ibm-java2-x86_64-50-SR13FP1/jre/bin/pack200 -r -E4 /home/davidw/git/rt.equinox.p2/bundles/org.eclipse.equinox.p2.jarprocessor/temp_org.eclipse.jpt.doc.isv_2.0.0.v201105240000.jar /home/davidw/testHugeBuildArtifacts/olddoc/org.eclipse.jpt.doc.isv_2.0.0.v201105240000.jar Run with -verbose for more details. If verbose is set, then user would get what they'd have gotten before: Running Repack on /home/davidw/testHugeBuildArtifacts/olddoc/org.eclipse.jpt.doc.isv_2.0.0.v201105240000.jar org.eclipse.jpt.doc.isv_2.0.0.v201105240000.jar is included in pack200 by default for main jars. STDERR: JVMDUMP006I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" - please wait. STDERR: JVMDUMP032I JVM requested Snap dump using '/home/davidw/git/rt.equinox.p2/bundles/org.eclipse.equinox.p2.jarprocessor/Snap.20130103.195743.24605.0001.trc' in response to an event STDERR: JVMDUMP010I Snap dump written to /home/davidw/git/rt.equinox.p2/bundles/org.eclipse.equinox.p2.jarprocessor/Snap.20130103.195743.24605.0001.trc STDERR: JVMDUMP032I JVM requested Heap dump using '/home/davidw/git/rt.equinox.p2/bundles/org.eclipse.equinox.p2.jarprocessor/heapdump.20130103.195743.24605.0002.phd' in response to an event STDERR: JVMDUMP010I Heap dump written to /home/davidw/git/rt.equinox.p2/bundles/org.eclipse.equinox.p2.jarprocessor/heapdump.20130103.195743.24605.0002.phd STDERR: JVMDUMP032I JVM requested Java dump using '/home/davidw/git/rt.equinox.p2/bundles/org.eclipse.equinox.p2.jarprocessor/javacore.20130103.195743.24605.0003.txt' in response to an event STDERR: JVMDUMP010I Java dump written to /home/davidw/git/rt.equinox.p2/bundles/org.eclipse.equinox.p2.jarprocessor/javacore.20130103.195743.24605.0003.txt STDERR: JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError". STDERR: Exception in thread "main" java.lang.OutOfMemoryError STDERR: at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:116) STDERR: at java.io.ByteArrayOutputStream.writeTo(ByteArrayOutputStream.java:133) STDERR: at com.sun.java.util.jar.pack.Package$File.writeTo(Package.java:784) STDERR: at com.sun.java.util.jar.pack.PackageWriter.writeFiles(PackageWriter.java:701) STDERR: at com.sun.java.util.jar.pack.PackageWriter.write(PackageWriter.java:66) STDERR: at com.sun.java.util.jar.pack.PackerImpl$DoPack.flushPackage(PackerImpl.java:596) STDERR: at com.sun.java.util.jar.pack.PackerImpl$DoPack.flushAll(PackerImpl.java:550) STDERR: at com.sun.java.util.jar.pack.PackerImpl$DoPack.run(PackerImpl.java:498) STDERR: at com.sun.java.util.jar.pack.PackerImpl.pack(PackerImpl.java:91) STDERR: at com.sun.java.util.jar.pack.Driver.main(Driver.java:279) Error: 1 was returned from command: /usr/local/jdks/ibm-java2-x86_64-50-SR13FP1/jre/bin/pack200 -r -E4 /home/davidw/git/rt.equinox.p2/bundles/org.eclipse.equinox.p2.jarprocessor/temp_org.eclipse.jpt.doc.isv_2.0.0.v201105240000.jar /home/davidw/testHugeBuildArtifacts/olddoc/org.eclipse.jpt.doc.isv_2.0.0.v201105240000.jar
There's probably a few other places, where the code catches IOException, where something similar could be done, but I've not included changes there. Just to keep this bug focused on the specific problem I originally had. As a general rule, though, I will say, I think "verbose" means "give extra information about normal processing", its absence, in my experience, does not mean "suppress error and exception messages". But, I like the attached patch because at least its not completely silent about errors and gives a hint on how to find out more. HTH
Will not be addressed in Luna.
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant. -- The automated Eclipse Genie.
