Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 357025

Summary: [RDT] Dstore crashes during indexing due to out of memory condition
Product: [Tools] PTP Reporter: Corey Ashford <cjashfor>
Component: Remote ToolsAssignee: Project Inbox <ptp-inbox>
Status: CLOSED WONTFIX QA Contact:
Severity: normal    
Priority: P3 CC: g.watson, recoskie, rsjoao, sdix, wainersm
Version: 5.0.2   
Target Milestone: ---   
Hardware: PC   
OS: Linux   
Whiteboard:
Attachments:
Description Flags
Video showing the dstore crash at about 49% complete
none
Log file exported from the Error Log view
none
Tar file containing the .txt and .trc files that resulted from the dstore crash none

Description Corey Ashford CLA 2011-09-07 20:31:47 EDT
Build Identifier: I20110613-1736

During indexing of the Linux kernel source on a remote Power-based machine, the Dstore daemon is crashing due to an out-of-memory condition.

Reproducible: Always

Steps to Reproduce:
1. Install Linux kernel source on remote Power-based machine
2. Create an RDT/Remote Tools project whose home directory is the kernel source on the remote machine
3. Change the configuration to filter out the source from the arch/* directories other than arch/powerpc
4. Start an indexing operation.
5. At 49% complete, the dstore server will crash with an out of memory condition
Comment 1 Corey Ashford CLA 2011-09-07 20:34:15 EDT
Created attachment 202948 [details]
Video showing the dstore crash at about 49% complete
Comment 2 Corey Ashford CLA 2011-09-07 20:38:25 EDT
Created attachment 202949 [details]
Log file exported from the Error Log view
Comment 3 Corey Ashford CLA 2011-09-07 20:40:14 EDT
Created attachment 202950 [details]
Tar file containing the .txt and .trc files that resulted from the dstore crash
Comment 4 Corey Ashford CLA 2011-09-07 20:49:29 EDT
I can attach the .phd (hex dump) files, if those would be useful.  I haven't attached them yet because they are big (about 20MB each)
Comment 5 Chris Recoskie CLA 2011-09-08 11:04:40 EDT
You need to set the heap size yourself to something that's big enough to handle parsing your code.  The heap size set in the server launch script is just a default.  Unless you find that there's some kind of runaway condition that is allocating memory in an endless loop or something, it's almost certainly just that one of the files in your project requires a lot of memory to parse and you need to update the heap maximum.

Try setting the heap max to 512 MB, 1 GB, etc., and see where that gets you.  Generally 1 GB is enough for just about anything.  If after that you discover something is still amiss, then we might want to reopen this.
Comment 6 Corey Ashford CLA 2011-09-09 21:15:23 EDT
(In reply to comment #5)
> You need to set the heap size yourself to something that's big enough to handle
> parsing your code.  The heap size set in the server launch script is just a
> default.  Unless you find that there's some kind of runaway condition that is
> allocating memory in an endless loop or something, it's almost certainly just
> that one of the files in your project requires a lot of memory to parse and you
> need to update the heap maximum.
> 
> Try setting the heap max to 512 MB, 1 GB, etc., and see where that gets you. 
> Generally 1 GB is enough for just about anything.  If after that you discover
> something is still amiss, then we might want to reopen this.

After digging for awhile, I found that the java command that is used is not specified in a script, but rather in an extension declared by org.eclipse.ptp.rdt.server.dstore:

<plugin>
        <extension
           point="org.eclipse.ptp.remote.core.remoteServer">
        <remoteServer
              class="org.eclipse.ptp.rdt.server.dstore.core.DStoreServer"
              continuous="true"
              id="org.eclipse.ptp.rdt.server.dstore.RemoteToolsDStoreServer"
              launchCommand="java -Xms64m -Xmx128m -Xss2m -DA_PLUGIN_PATH=. -DDSTORE_TRACING_ON=false -Dclient.username=$USER -DDSTORE_SPIRIT_ON=true -jar ${payload} 0 60000"
              name="%RemoteToolsDStoreServer.name"
              payload="rdt-server.jar"
              verifyLaunchCommand="java -version"
              verifyLaunchFailMessage="{0}: Invalid java version or java not installed on &quot;{1}&quot;"
              verifyLaunchPattern="^(java version )(.)(1\.[5-9]|[2-9]\.).*$">
        </remoteServer>
...

It doesn't look to me like this is designed to be modified by users, especially without using a plug-in.  Even if they used a plug-in, it's not clear which one of the extensions would get processed first and override the other one.

Am I missing something here?
Comment 7 Chris Recoskie CLA 2011-09-12 08:29:54 EDT
Good point... I forgot that the launch parameters are somewhat locked when you are using Remote Tools.

Greg... not sure how you want to address that.  You can never really pick one size that fits all.  Maybe the heap size needs to be a command line parameter to the server launch script.
Comment 8 Greg Watson CLA 2011-09-12 08:39:54 EDT
This is something that could be done in a future release. 

For SR1, I could increase the heap size to say, 1gb. At least that would reduce the incidence of the problem. Let me know if you would like me to do this.
Comment 9 Greg Watson CLA 2011-09-14 08:54:13 EDT
I would need to do this today if it is to go into SR1.
Comment 10 Corey Ashford CLA 2011-09-14 12:20:02 EDT
(In reply to comment #9)
> I would need to do this today if it is to go into SR1.

I tested changing the max heap limit to 1024m yesterday, and found that Eclipse was able to complete the indexing operation, though there still are a number of other anomalies.

Making this change would be an improvement.
Comment 11 Greg Watson CLA 2011-09-14 14:06:29 EDT
Done.
Comment 12 Wainer dos Santos Moschetta CLA 2011-09-19 13:06:50 EDT
(In reply to comment #11)
> Done.

Fix will be on 5.0.2?
Comment 13 Greg Watson CLA 2011-09-19 15:48:50 EDT
Yes, I committed to ptp_5_0 branch.
Comment 14 Steven CLA 2012-07-16 15:46:58 EDT
This problem appears to have slipped back into the latest version.
Comment 15 Greg Watson CLA 2014-05-29 15:35:39 EDT
Assuming this will not be fixed.