Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 355256 - Out of memory errors during PDE build with large export-package list
Summary: Out of memory errors during PDE build with large export-package list
Status: RESOLVED WONTFIX
Alias: None
Product: PDE
Classification: Eclipse Project
Component: Build (show other bugs)
Version: 3.7   Edit
Hardware: All All
: P3 normal with 15 votes (vote)
Target Milestone: ---   Edit
Assignee: pde-build-inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-08-19 15:00 EDT by Nadeem Aboobaker CLA
Modified: 2018-12-03 09:11 EST (History)
4 users (show)

See Also:


Attachments
The exception/stacktrace when doing the build on the command line with the PDE antRunner build. (1.84 KB, text/plain)
2011-08-19 15:01 EDT, Nadeem Aboobaker CLA
no flags Details
The exception/stacktrace when doing the build within Eclipse/PDE. (1.84 KB, text/plain)
2011-08-19 15:02 EDT, Nadeem Aboobaker CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Nadeem Aboobaker CLA 2011-08-19 15:00:14 EDT
Build Identifier: I20110613-1736

We recently switched from Eclipse 3.6.2 to Eclipse 3.7 and as a result had to change the way we referenced library jar files in our plugins.


We used to do this in the plugin.xml:

<runtime>
<library name="../../somefolder/somejar.jar"><export name="*"/></library>
...
<library name="../anotherfolder/anotherjar.jar"><export name="*"/></library>
</runtime>

It turns out that the "feature" of being able to refer to jar files outside of a plugin was fixed in Bug 342114. So, we changed the way our library jar files get referenced.


Now we use MANIFEST.MF and explicitly list every package in the jar files:

Bundle-Classpath: external:c:/somefolder/somejar.jar,
external:c:/folder/anotherfolder/anotherjar.jar
Export-Package: com.example.some.package,
com.example.another.package,
...
com.example.package


Now when we run the command line PDE build or the build in our development Eclipse, we get "OutOfMemoryError: Java heap space" errors even after significantly increasing our VM memory. We've gone from everything working fine at -Xmx512m to now running out of memory at -Xmx1024m. If increased much past that, then the Eclipse/VM doesn't launch on our 32 bit Windows due to the memory number being too high. I've tried -Xmx2048 on 64 bit Windows and Linux and still hit the out of memory error, but less frequently.

At least for the command line build problem, I believe it's an issue with the large Export-Package list in MANIFEST.MF.  If I remove all 10,000 entries from Export-Package, the command line build works perfectly fine.  I need to add the Export-Package list back to the MANIFEST.MF to get things working at runtime.  My current workaround at buildtime to build the plugins is to remove the Export-Package list at build and then add it back afterwards.

I have no workaround during dev time, except to use my 64 bit OS and keep re-launching my dev Eclipse until it's the one time I don't run into the issue.

Just a few notes:
- It's not a matter of a few jars and a few exported packages, it's 500+ jar files and 10,000 packages. I understand that's a very high number, but it worked when using plugin.xml and export *.
- The PDE still builds correctly when using the old plugin.xml way of referring the external jars and * to export the packages, but the runtime does not work (leads to class not found problems).
- There are no problems at runtime with using the new MANIFEST.MF approach. Eclipse uses the same memory VM settings as before and things work just as before.


Reproducible: Always

Steps to Reproduce:
1. Have plugin.xml reference hundreds of jar files and export *
2. Convert plugin.xml jar file references to MANIFEST.MF Bundle-Classpath entries and explicitly list all 10,000 packages in Export-Package.
3. Run build on command line or in UI, see OutOfMemory error.
Comment 1 Nadeem Aboobaker CLA 2011-08-19 15:01:41 EDT
Created attachment 201829 [details]
The exception/stacktrace when doing the build on the command line with the PDE antRunner build.
Comment 2 Nadeem Aboobaker CLA 2011-08-19 15:02:05 EDT
Created attachment 201830 [details]
The exception/stacktrace when doing the build within Eclipse/PDE.
Comment 3 Curtis Windatt CLA 2011-10-25 11:38:48 EDT
Duplicate of Bug 346730
Comment 4 Curtis Windatt CLA 2011-10-25 11:39:25 EDT
That was intended as a question.

Is this a dupe of bug 346730
Comment 5 Nadeem Aboobaker CLA 2011-10-26 12:16:23 EDT
This bug looks similar to Bug 346730, but the stack traces are different.  I'm not sure if that means the underlying cause is different or not.

This bug:
java.lang.OutOfMemoryError: Java heap space
at org.eclipse.jdt.internal.core.ClasspathEntry.<init>(ClasspathEntry.java:285)
at org.eclipse.jdt.internal.core.ClasspathEntry.<init>(ClasspathEntry.java:255)
at org.eclipse.jdt.core.JavaCore.newLibraryEntry(JavaCore.java:4188)
...



Bug 346730:
java.lang.OutOfMemoryError
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.<init>(ZipFile.java:203)
at java.util.zip.ZipFile.<init>(ZipFile.java:234)
at org.eclipse.jdt.internal.compiler.batch.ClasspathJar.initialize(ClasspathJar.java:143)
...
Comment 6 Lars Vogel CLA 2018-12-03 09:11:01 EST
Currently we are not actively enhancing PDE build anymore. Therefore, I close this bug as WONTFIX. 

Please reopen, if you plan to provide a fix.
Comment 7 Lars Vogel CLA 2018-12-03 09:11:15 EST
Currently we are not actively enhancing PDE build anymore. Therefore, I close this bug as WONTFIX. 

Please reopen, if you plan to provide a fix.