Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 364412 - SVG charts with large number of data points exhaust heap space very quickly
Summary: SVG charts with large number of data points exhaust heap space very quickly
Status: VERIFIED WORKSFORME
Alias: None
Product: z_Archived
Classification: Eclipse Foundation
Component: BIRT (show other bugs)
Version: 3.7.1   Edit
Hardware: PC Linux
: P3 major (vote)
Target Milestone: 4.4.1   Edit
Assignee: Yulin Wang CLA
QA Contact: Galina Derenshteyn CLA
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-11-21 14:26 EST by akshay.s86 CLA
Modified: 2015-05-12 17:47 EDT (History)
4 users (show)

See Also:


Attachments
A Patch (3.40 KB, patch)
2015-05-12 17:44 EDT, Joshua DeFord CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description akshay.s86 CLA 2011-11-21 14:26:23 EST
I noticed that SVG charts take up huge amount of heap space and this memory does not get released quick enough. I have examples (Unable to disclose them as they are confidential) where 5-10 scatter charts containing 20-30k points each can exhaust over 300-400MB of heap memory easily when they are all generated in the same template. So with about 1GB of maximum heap, if you run this template a couple of times without forcing garbage collection, you can run out of heap space very quickly. I ran BIRT runtime with JMX extension and JConsole to monitor the heap usage and memory usage explodes exponentially. Ofcourse, this problem does not persist with PNG or JPG.
Comment 1 Hao Zhou CLA 2011-12-29 03:30:35 EST
We have done performance test like this,no such issue.
Could you provide a sample report we can test with?
Comment 2 Yulin Wang CLA 2014-08-13 14:09:50 EDT
can not reproduce.
Comment 3 Galina Derenshteyn CLA 2014-09-17 17:06:38 EDT
Based on Yulin Wang and Hao Zhou latest comments, this is being moved to Verified for the upcoming 4.4.1 release.

Should there be additional justifications for this issue to be reconsidered, please update with those comments, and it can be reopened for the next release.
Comment 4 Joshua DeFord CLA 2015-05-12 17:44:29 EDT
Created attachment 253424 [details]
A Patch
Comment 5 Joshua DeFord CLA 2015-05-12 17:47:04 EDT
We have a bubble chart report with 3000 bubbles and are seeing out of memory errors.  The problem occurs because each of the 3000 bubbles has a reference to a unique SeriesRenderingHints object. Each of which, in turn have 3000 unique reference to a DataPointHints object.  The retained heap size of each DataPointHints object is 544 bytes.  Since there are 9,000,000 of those we run out of memory pretty fast.

We dug into this issue to try to determine why neither the SeriesRenderingHints nor the DataPointHints objects are being reused.  We determined that, after all the rendering hints are created, they are then passed to a rendering function which determines wether to render each bubble by making a call to Bubble.isValidBubbleEntry which performs a null check on the DataPointRenderingHints object's orthogonalValue.

Once we determined that DataPoinHints objects are considered invalid if they have a null orthogonalValue we scoured the rest of the code and determined that no DataPointHints are even used if they don't have their orthogonalValue set.

If this assumption is correct then a solution to our out of memory issue is to prevent the creation of invalid DataPointRenderingHints object that will not be used when rendering a Plot2DAxes plot.

The code that constructs the DataPointHints objects lies in PlotWith2DAxes.java.  This appeared to be a natural place to add a null check and avoid creating the DataPointRenderingHints if their is no orthogonalValue to assign to it.  However, introducing a bunch of null values into an array that, in previous releases, has had no null values, seemed a bit risky.

Our approach was to create one DataPointRenderHints object with a null orthogonalValue.  All DataPointHints objects that have a valid orthogonalValue are created as they always have been.  But, when a null orthogonalValue is encountered, the one that has a null orthogonalValue is reused rather than created again.  

This reuse of invalid DataPointHints for PlotWith2DAxes significantly reduces the memory footprint created by bubble charts.

I've attached a patch containing our fix.