Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 325152 - [builds] improve scalability of model
Summary: [builds] improve scalability of model
Status: CLOSED MOVED
Alias: None
Product: z_Archived
Classification: Eclipse Foundation
Component: Mylyn (show other bugs)
Version: unspecified   Edit
Hardware: PC Linux
: P3 enhancement (vote)
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 323877
  Show dependency tree
 
Reported: 2010-09-13 14:33 EDT by Steffen Pingel CLA
Modified: 2010-10-26 09:38 EDT (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Steffen Pingel CLA 2010-09-13 14:33:34 EDT
The builds framework currently maintains a single model in memory that holds information about servers, plans and builds. This can lead to high memory consumption if builds reference thousands of tests, large change sets etc. or if several builds are cached per plan. To improve this a more scalable model is needed for persisting the model, e.g. by using CDO or distributing the model over several resources.

In Memory (information that is displayed in Builds view and tooltip):
* Servers
* Subscribed Plans
* Synchronization information for efficient refresh

On Demand (when editor is opened or details are accessed otherwise):
* Build details (test results, change set information, build artifacts...)
Comment 1 Kenn Hussey CLA 2010-09-13 15:31:12 EDT
Here are some changes that should have an immediate (positive) impact on memory footprint:

- containment proxies

Setting the 'Containment Proxies' generator model property to "true" will allow you to serialize build models, build servers, build plans, builds, change sets, and test results in separate resources.

- "ultra slim diet"

Setting the 'Root Extends Class' generator model property to "org.eclipse.emf.ecore.impl.MinimalEObjectImpl$Container" will reduce the overhead required to represent build elements in memory.


To improve performance and on-disk footprint, I would recommend using the binary resource implementation to persist build model elements. Also, with minor changes to the resource implementation (to persist attribute values for proxies), it would be possible to render information about build elements (e.g., in tooltips) without having to actually load them, which would in turn further improve memory footprint.
Comment 2 Eike Stepper CLA 2010-09-13 23:56:15 EDT
Reducing the footprint per object is not bad but does not really increase scalability. Splitting out more resources with containment proxies is also a good idea, but only creates more scalability if there's a mechanism that kicks them out of main memory at some point.

I've blogged about model scalability a while ago: http://thegordian.blogspot.com/2008/11/how-scalable-are-my-models.html . It explains the difficulties with EMF and how CDO solves the problem.
Comment 3 Kenn Hussey CLA 2010-09-15 13:36:59 EDT
(In reply to comment #2)

Yes, CDO certainly provides a more comprehensive (i.e., true) scalability solution, especially in collaboration scenarios, but may be overkill if you're simply querying and persisting data locally. Ultimately, it depends on what the application requirements/constraints are...
Comment 4 Eclipse Webmaster CLA 2022-11-15 11:45:08 EST
Mylyn has been restructured, and our issue tracking has moved to GitHub [1].

We are closing ~14K Bugzilla issues to give the new team a fresh start. If you feel that this issue is still relevant, please create a new one on GitHub.

[1] https://github.com/orgs/eclipse-mylyn