Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 340704 - [tools] Improve jar processor
Summary: [tools] Improve jar processor
Status: CLOSED WONTFIX
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: p2 (show other bugs)
Version: 3.7   Edit
Hardware: PC Windows XP
: P3 enhancement (vote)
Target Milestone: ---   Edit
Assignee: P2 Inbox CLA
QA Contact:
URL:
Whiteboard: stalebug
Keywords:
Depends on:
Blocks:
 
Reported: 2011-03-22 15:59 EDT by Kim Moir CLA
Modified: 2019-11-14 03:45 EST (History)
5 users (show)

See Also:


Attachments
non packed version of pde.core (1.12 MB, application/octet-stream)
2011-03-22 16:03 EDT, Kim Moir CLA
no flags Details
corrupt packed jar (379.40 KB, application/octet-stream)
2011-03-22 16:18 EDT, Kim Moir CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Kim Moir CLA 2011-03-22 15:59:51 EDT
We pack the jars in our repo in the following manner

The pack.properties just specifies the list of orbit bundles to exclude from the packing process.

<copy file="${eclipse.build.configs}/../../extras/pack.properties" todir="${reposource}" />
<p2.process.artifacts repositoryPath="file://${reposource}" pack="true" />
<delete>
	<fileset dir="${reposource}">
	     <include name="pack.properties" />
        </fileset>
</delete>
	<p2.remove.iu>
		<repository location="file://${reposource}" />
		<iu id="com.ibm.icu" artifacts="(format=packed)" />
		<iu id="com.ibm.icu.base" artifacts="(format=packed)" />
		<iu id="com.ibm.icu.source" artifacts="(format=packed)" />
		<iu id="com.ibm.icu.base.source" artifacts="(format=packed)" />
		<iu id="org.eclipse.jdt.debug.ui" artifacts="(format=packed)" />
		<iu id="org.eclipse.pde.api.tools" artifacts="(format=packed)" />
	</p2.remove.iu>

During the last two I-builds, we've had a problem with a corrupt pde.core jar.pack.gz file.  See bug 340165.  

I've determined that this corrupt pack.gz file occurs when running the p2.process.artifacts task against the repo.  However, it doesn't appear when I use the native pack200 jar that is included in the JDK.  

I can put a zip of a packed repo somewhere for a p2 committer to reproduce this problem.  It's too large to attach to a bug report.
Comment 1 Kim Moir CLA 2011-03-22 16:03:22 EDT
Created attachment 191709 [details]
non packed version of pde.core
Comment 2 Kim Moir CLA 2011-03-22 16:18:08 EDT
Created attachment 191710 [details]
corrupt packed jar
Comment 3 DJ Houghton CLA 2011-03-22 17:35:16 EDT
Andrew and I have looked at this problem and believe that it is related to packing bundles which have nested JARs. We were able to reproduce the problem using only the command line and java tools. (no Eclipse involved)

Recommendation is for the pde.core bundle to include a META-INF/eclipse.inf file which directs us to exclude the child JAR. (jarprocessor.exclude.children=true)

Going forward we might consider changing the default behaviour to ignore child nested JARs when packing.
Comment 4 Kim Moir CLA 2011-03-22 21:24:33 EDT
Thanks DJ and Andrew.  I'll open a bug against pde with a patch for the eclipse.inf
Comment 5 Thomas Hallgren CLA 2011-03-23 02:57:00 EDT
(In reply to comment #3)
> Going forward we might consider changing the default behaviour to ignore child
> nested JARs when packing.

When I wrote the Buckminster jarprocessor, I found that it was often more efficient to pack the nested jars and then pack the containing jar with level 1. This level is essentially a no-op and can be used to make the tooling aware that the file needs to be unpacked. Nested packed jars wouldn't be found otherwise.

I think the most optimal result is obtained when jars containing .class files are packed. Checking their presence is probably a better discriminator than just ignoring the child or the parent.

The Buckminster jarprocessor has more smarts and is intended as a slot-in replacement for the p2 one. It will however require Java 1.5. Feel free to copy whatever you find interesting. You can find it here:

http://dev.eclipse.org/svnroot/tools/org.eclipse.buckminster/trunk/org.eclipse.buckminster.jarprocessor/

and a test suite (uses an older version of the file discussed in this bugzilla):

http://dev.eclipse.org/svnroot/tools/org.eclipse.buckminster/trunk/org.eclipse.buckminster.jarprocessor.test/
Comment 6 Andrew Niefer CLA 2011-03-23 10:24:35 EDT
(In reply to comment #5)
> (In reply to comment #3)
> > Going forward we might consider changing the default behaviour to ignore child
> > nested JARs when packing.
> 
> When I wrote the Buckminster jarprocessor, I found that it was often more
> efficient to pack the nested jars and then pack the containing jar with level
> 1. This level is essentially a no-op and can be used to make the tooling aware
> that the file needs to be unpacked. Nested packed jars wouldn't be found
> otherwise.
> 

This would be a good strategy for bundles like org.eclipse.pde.build where all the java code is in nested jars.  However, for org.eclipse.pde.core, the majority of the .class files are in the outer jar and only a few classes for ant tasks are in the nested jar.
Comment 7 Kim Moir CLA 2011-03-23 11:31:01 EDT
I opened bug 340776 to add an eclipse.inf to the pde core bundle.
Comment 8 Lars Vogel CLA 2019-11-14 03:45:23 EST
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

If the bug is still relevant, please remove the "stalebug" whiteboard tag.