Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 212075 - java.lang.OutOfMemoryError: Java heap space when attempting to load specific publish0.dat file
Summary: java.lang.OutOfMemoryError: Java heap space when attempting to load specific ...
Status: CLOSED FIXED
Alias: None
Product: WTP ServerTools
Classification: WebTools
Component: wst.server (show other bugs)
Version: 3.0   Edit
Hardware: PC Windows XP
: P3 major (vote)
Target Milestone: 3.0 M4   Edit
Assignee: Tim deBoer CLA
QA Contact: Tim deBoer CLA
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2007-12-05 16:46 EST by Rob Frost CLA
Modified: 2017-10-11 16:19 EDT (History)
0 users

See Also:


Attachments
Problematic file (682 bytes, application/octet-stream)
2007-12-05 16:47 EST, Rob Frost CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Rob Frost CLA 2007-12-05 16:46:23 EST
During my dev/testing of some new classpath publish/export support I all of a sudden got stuck in a situation where every attempt to execute publish (against a Tomcat 6.0 server in this case) hit the following OutOfMemoryError:

!ENTRY org.eclipse.core.jobs 4 2 2007-12-05 16:33:37.921
!MESSAGE An internal error occurred during: "Publishing to Tomcat v6.0 Server at localhost...".
!STACK 0
java.lang.OutOfMemoryError: Java heap space
	at org.eclipse.wst.server.core.internal.ModulePublishInfo.loadResource(ModulePublishInfo.java:191)
	at org.eclipse.wst.server.core.internal.ModulePublishInfo.load(ModulePublishInfo.java:186)
	at org.eclipse.wst.server.core.internal.ModulePublishInfo.<init>(ModulePublishInfo.java:86)
	at org.eclipse.wst.server.core.internal.ServerPublishInfo.load(ServerPublishInfo.java:274)
	at org.eclipse.wst.server.core.internal.ServerPublishInfo.<init>(ServerPublishInfo.java:56)
	at org.eclipse.wst.server.core.internal.PublishInfo.getServerPublishInfo(PublishInfo.java:75)
	at org.eclipse.wst.server.core.internal.Server.getServerPublishInfo(Server.java:856)
	at org.eclipse.wst.server.core.internal.Server.doPublish(Server.java:936)
	at org.eclipse.wst.server.core.internal.Server.publish(Server.java:887)
	at org.eclipse.wst.server.core.internal.PublishServerJob.run(PublishServerJob.java:73)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)

I was getting this regardless of whether or not there was a module associated with the server instance and found that if I simply removed the current .metadata/.plugins/org.eclipse.wst.server.core/publish/publish0.dat file from that directory everything worked (a new publish0.dat file was generated).

I'm attaching the problematic file.

Depending on how likely this is to occur under standard use cases, this may be a higher severity (my testing could have triggered a publish failure that corrupted that file...)
Comment 1 Rob Frost CLA 2007-12-05 16:47:08 EST
Created attachment 84570 [details]
Problematic file
Comment 2 Tim deBoer CLA 2007-12-05 16:50:53 EST
What are your memory settings in eclipse.ini? Did you get the same error after a restart?

Do you have any reason to think that this process was using a lot of memory, or could this just have been the tipping point over the edge?
Comment 3 Rob Frost CLA 2007-12-05 17:02:49 EST
The issue was odd, I launching that Eclipse from my dev instance using Sun's JVM (1.6.0_02) with the following memory settings:

-Xms384m
-Xmx768m
-XX:MaxPermSize=256m
-XX:PermSize=192m

right before the error the javaw process was consuming about 120M of roughly 600M allocated to the process - so I was very surprised to be hitting this...

however, it is definitely related to loading that publish0.dat file - removing it completely eliminated the issue.
Comment 4 Rob Frost CLA 2007-12-05 17:04:27 EST
and yes, I did get the same error after restarting that Eclipse instance

what do you see if you try dropping that file into .metadata/.plugins/org.eclipse.wst.server.core/publish/?
Comment 5 Tim deBoer CLA 2007-12-05 21:11:47 EST
Yup, I've reproduced. The JVM I'm using gives a slightly more helpful error message (java.lang.OutOfMemoryError: Requested array size exceeds VM limit), and tracking it down to the method below and debugging is looks like the publish0.dat is reporting that it contains a folder with 1181708078 resources.

I need to do more debugging. Do you have any idea how to reproduce, or can you tell me about the 3 projects that were deployed to this server? Right now I don't know if it's just a corrupt file, or there is a case where it saves incorrectly.
Comment 6 Tim deBoer CLA 2007-12-05 22:09:14 EST
*wow*. I found a bug in the saving of the file, and tracing back it has been there since we switched to a binary format for this file in mid WTP-2.0. Based on where the bug is, publish history has almost always been lost after workbench restart, causing a load failure in almost all cases when you publish to that server again.

In almost every case, the load failure would result in an IOException and be handled acceptably (any server that takes advantage of incremental publishing wouldn't get an advantage until the second publish), but you've hit an odd case where the bits just happened to line up where an array size will be 1181708078 long and causes an OOME.

I've fixed the bug, and any new files will be saved correctly. I also put a limit of 1,000,000 files per directory - this hopefully shouldn't block anyone :), and hitting this trigger will avoid an OOME and throw an IOException, reverting to a full publish the first time.

Fix released to 3.0 M4 builds.
Comment 7 Rob Frost CLA 2007-12-06 08:36:16 EST
Note: no idea how to reproduce the original issue

I'll sync to your fixes, continue testing with those and let you know if I see anything else.

thanks for the quick response, Tim

Comment 8 Tim deBoer CLA 2008-05-01 16:59:58 EDT
Closing old bugs.
Comment 9 Eclipse Genie CLA 2017-10-11 16:19:22 EDT
New Gerrit change created: https://git.eclipse.org/r/108472
Comment 10 Eclipse Genie CLA 2017-10-11 16:19:24 EDT
New Gerrit change created: https://git.eclipse.org/r/108471
Comment 11 Eclipse Genie CLA 2017-10-11 16:19:31 EDT
New Gerrit change created: https://git.eclipse.org/r/108473