Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 367098

Summary: bundle classpath extraction too slow for big web bundles with lots of nested jars
Product: z_Archived Reporter: Jan Sievers <jan.sievers>
Component: TychoAssignee: Jan Sievers <jan.sievers>
Status: RESOLVED FIXED QA Contact:
Severity: major    
Priority: P3 CC: p.petrov
Version: unspecified   
Target Milestone: ---   
Hardware: All   
OS: All   
Whiteboard:
Attachments:
Description Flags
JIRA web archive bundle OSGi MANIFEST none

Description Jan Sievers CLA 2011-12-19 11:38:21 EST 
in an internal application, we assemble a bundle-ized (RFC66 WAB) version of the atlassian JIRA WAR [1].
This is a 110 MB web bundle with ~45 MB worth of nested jars on the Bundle-Classpath (WEB-INF/lib/*.jar).

A tycho build for a feature including this bundle takes ~15min every time just to extract the nested jars.

The bug is that nested jar extraction, which is only needed for compilation, should not happen at all if the bundle is referenced from a feature only (or update site/repository for that matter).

An enhancement which could make this bug less severe is that extraction could be optimized (extracted released artifacts could be assumed to be immutable, thus reused across builds)


[1] http://www.atlassian.com/software/jira/download
Comment 1 Jan Sievers CLA 2011-12-19 17:44:52 EST 
I measured that the plexus ZipUnarchiver is much slower (at least the way we are using it with extraction paths) than plain java.util.ZipFile

Will go back to java.util.ZipFile in DefaultBundleReader#getEntry()
Comment 2 Tobias Oberlies CLA 2011-12-20 04:10:56 EST 
(In reply to comment #0)
> The bug is that nested jar extraction, which is only needed for compilation,
> should not happen at all if the bundle is referenced from a feature only (or
> update site/repository for that matter).
Bug 364134 strives for a separation of concerns in the afterProjectsRead code, and this change would fit into this work.
Comment 4 Jan Sievers CLA 2011-12-20 07:55:00 EST 
just for the record: a typical stacktrace taken during ~15 min of zip extraction:

"main" prio=6 tid=0x00eb9800 nid=0x2a64 runnable [0x0027f000]
   java.lang.Thread.State: RUNNABLE
        at java.io.RandomAccessFile.length(Native Method)
        at java.io.RandomAccessFile.skipBytes(RandomAccessFile.java:429)
        at org.codehaus.plexus.archiver.zip.ZipFile.resolveLocalFileHeaderData(ZipFile.java:440)
        at org.codehaus.plexus.archiver.zip.ZipFile.<init>(ZipFile.java:151)
        at org.codehaus.plexus.archiver.zip.AbstractZipUnArchiver.execute(AbstractZipUnArchiver.java:269)
        at org.codehaus.plexus.archiver.AbstractUnArchiver.extract(AbstractUnArchiver.java:127)
        at org.eclipse.tycho.core.osgitools.DefaultBundleReader.getEntry(DefaultBundleReader.java:208)
        at org.eclipse.tycho.core.maven.MavenDependencyInjector.newExternalDependencies(MavenDependencyInjector.java:116)
        at org.eclipse.tycho.core.maven.MavenDependencyInjector.addDependency(MavenDependencyInjector.java:95)
        at org.eclipse.tycho.core.maven.MavenDependencyInjector.injectMavenDependencies(MavenDependencyInjector.java:51)
        at org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.injectDependenciesIntoMavenModel(P2TargetPlatformResolver.java:447)
        at org.eclipse.tycho.core.resolver.DefaultTychoDependencyResolver.resolveProject(DefaultTychoDependencyResolver.java:98)
        at org.eclipse.tycho.core.maven.TychoMavenLifecycleParticipant.afterProjectsRead(TychoMavenLifecycleParticipant.java:91)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:273)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
Comment 5 Jan Sievers CLA 2011-12-20 08:00:10 EST 
Created attachment 208607 [details]
JIRA web archive bundle OSGi MANIFEST