Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 355542

Summary: Massive memory spikes from solr
Product: [ECD] Orion Reporter: John Arthorne <john.arthorne>
Component: ServerAssignee: John Arthorne <john.arthorne>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P3    
Version: unspecified   
Target Milestone: 0.3 M2   
Hardware: PC   
OS: Windows 7   
Whiteboard:

Description John Arthorne CLA 2011-08-23 13:42:58 EDT 
While profiling an orion server I noticed large spikes in heap usage for brief periods (300MB). None of this is retained. This is coming from a background job that is looking for stale index entries.
Comment 1 John Arthorne CLA 2011-08-23 14:41:00 EDT 
The problem was the index purge job did a query for "everything" (in chunks of 1000 results at once), and then checked if each file was still present. By default solr returns all fields, including the contents of the files. So, we had the contents of 1000 files in memory during the query. In this case all we needed was the ID of each solr document, which vastly reduces the memory overhead. With this fix I am seeing a spike of 5MB rather than 300MB during index operations.

http://git.eclipse.org/c/orion/org.eclipse.orion.server.git/commit/?id=7a062e8c082fa90b42da20353f00c203a7790b91