Description Knut Wannheden 2011-08-26 02:12:21 EDT

The current clustering builder works with a fixed (but configurable) cluster size. Working with larger cluster sizes can considerably improve the build performance and is now also more readily possible as the node model memory footprint was reduced drastically with Xtext 2.0.

Another clustering strategy would be to continue to process resources as part of the same cluster until a certain memory threshold is reached. E.g. until there is less than 20 MB or 10% free heap memory available to the JVM. This would allow users to increase the build performance by giving more memory to the JVM.

Experience has shown that a dynamic clustering strategy as described works very well in practice. But it should be completed with - what may seem a bit strange - a minimum cluster size of e.g. 20. Without this the JVM's garbage collector may decide that there is still plenty of memory available (e.g. 10% of 5GB) and not kick in. Then, without a minimum cluster size, the builder would continue to work with clusters of size 1 until the garbage collector finally kicks in.

There are probably other very interesting clustering strategies so I think there should be an interface for this. Then I think the described dynamic clustering strategy should be the default of the Xtext builder.