Community
Participate
Working Groups
With the increase in plug-ins coming from and going to third parties the development cycle needs to be more incremental. In the Eclipse project itself we have started to consume pre-built plugins such as ICU4J and SSH2. This trend is likely to continue as, for example, Apache seeks to deliver their libraries as bundles and more people look to consume individual bundles/components from Equinox and similar projects. As such, the build process should support the incremental building of plug-ins (that is building plugins individually) based on a set of pre-built plugins kept in some sort of provisioning repository. Plug-ins coming out of the build should feed into said repository to be used as a base for future builds. This ability will in turn enable the more frequent building of whole systems (since you build only what has changed) and simplify incremental, milestone and release build cycles.
My initial thoughts on incremental build were that it comes down to a kind of initial provisioning of the build. Basically you have 2 sources for a given bundle: a previous built binary version, or the source from a repository and a decision must be made about which to use. When using .qualifiers and tags then it can be easy to decide if binary bundle is the version you want or if you need to build a new one. However, there are other cases where the decision is not so simple. More investigation is needed, but it seems that maybe something like http://www.eclipse.org/buckminster/ could be used to make these kinds of decisions. Ideally, this kind of thing could be handled in the fetch phase of the build and even implemented as a extension to the existing fetch task factory. One other complicating factor that comes to mind is assembling source features and finding the source for any binary bundles that were used.
Hey Andrew, I'm assuming this won't make 3.3? My team is currently very interested in this behavior and willing to have a resource to help out get this done. Is there willingness from the PDE Build team to help out a but in guidance of where to start? Our usecase is that we would like to point to say a target platform and have PDE build automatically just grab the binary if the plug-in exists instead of building everything from scratch.
not happening for 3.3. There is some interest in making it happen in coordination with the new provisioning story (e.g., managing the target is basically a provisioning operation)
3.3 isn't really an issue for us as its late in the cycle anyway, we just need a solution and willing to contribute with what we come up with.
One of the reason why we have not addressed the incremental build problem in 3.3 is because the problem can not be reduced to check which plug-ins have changed in the repo and just rebuild those. By doing such a simplification, any structural change to a class from a given plug-in could potentially break dependent plug-ins if they are not recompiled, which could cause weird errors at runtime. A complete / real incremental build solution consists in reimplementing the java builder implemented in JDT (or refactoring it to make it work without workspace). Is this really what we want to do? So, in the absence of a full incremental builder, given that the only way we could guarantee the validity of the output is by recompiling all plug-ins depending on the one that changed, I'm wondering if there are often situations where this could be valuable... Anyway, here is what could be done to implement a more incremental approach. It relies on the premise that qualifiers control the input and are kept in the output: - in the fetch script generator: - build a PDEState containing the content of the previous build and what we are building against. - for each plug-in that needs to be built, if the plug-in is in the PDEState then check the value of its qualifier. If it has changed, then fetch the new content and fetch the content of all plug-ins that depended on it. - in the build script generators, the difficulty is in making sure that the version that has been fetched will be used rather than the version being on disk. For that there are two possibilities: a) in the fetch script generator, modify the PDEState as things are being fetched, persist the state and reload it in the build script generator b) find a way to favor the plug-ins available in source over the binary ones The other problem, that we have today anyway, is how do we version a plug-in that has been recompiled but whose qualifier has not changed? Does the qualifier of the plug-in needs to contain a hash of all the plug-ins it has been compiled against?
The other complicating issue will be source feature generation. If only some of the plugins of the source feature changed, then we will need to be able to assemble a source feature with portions coming from the source feature of the previous build.
How about tossing the eclipse pde/launching out of the equation and treating it like any other build process. If a dependency has changed, then the files is rebuilt. Break it down into plugin, features, product phases. If the source file for the plugin has changed, and the feature uses the plugin the feature needs to change, which will cause the product to recompile. As it stands right now, I think the build process needs to be ripped out and completely re-structured. Incremental builds are aleardy handled by any other java application run with ant or any make type process, eclipse shouldn't be any different. As for source featuers, If you don't do a clean build and have the sources checked out to the same directory every time, the prior compiled classes will still exist. Some of this is all on how you architect the build, and in alot of ways we've gone away for the KISS principle when it comes to builds. Some of the Eclipse specific features and configuration problems can be addressed by specialized tasks to read eclipse configuration files without need eclipse to be run to acces them: http://ant4eclipse.sourceforge.net/
Removing plan keyword for bugs that are not linked to the 3.5 (Galileo) plan.
*** Bug 148074 has been marked as a duplicate of this bug. ***
Currently we are not actively enhancing PDE build anymore. Therefore, I close this bug as WONTFIX. Please reopen, if you plan to provide a fix.
