Description Pascal Rapicault 2005-08-10 00:42:09 EDT

Before going any further on how PDE could be changed to be workpsace aware, a little bit of 
background. When PDE Build started (many years ago), the state of eclipse was not such good looking 
as it is today: PDE UI was not so fancy, the workspace was always local (no linked folder), there was only 
one syntax for java, the compiler was minimal, and Eclipse wanted to be like any other open source 
project in that it wanted to provide its community with ways to reproduce its build (later referred as 
source builds) from minimal tooling (Ant and Javac).
As years passed by, eclipse improved: the workspace model evolved (and is evolving as we speak), the 
compiler added abilities to report more warnings, support for project based and file based preferences 
got added, but more importantly the eclipse eco-system grew and new builders got created, which 
ended up ended creating this gap between pde build and eclipse. But still after all those years we want 
to be good OS citizens and still provide source builds.

The workspace integration problem, even though realized as early as 3.0, got never solved because it 
was not too much in people's face (only a few people had complained about linked resources back then) 
and more important things had to be done in other areas of eclipse (of course all those decisions have 
been taken under PMC guidance).

Since then, the situation has slightly evolved: in 3.1 it is more easy for the user to be faced with this 
integration problem mainly due to the support for java 5 (bugs 55353, 78218).

The reminder of the bug classify the problems caused by pde build as in 3.1, proposes a tractable 
solution for 3.2, and explore what it would mean to make pde revolve around the workspace.

Problem classification as in 3.1
===================

Preferences set in the workspace are ignored
- 81918  encoding used in projects is ignored
- 55353, 76218 project or workspace compiler settings are ignored
- 82414, 103645 source filters are ignored
 
PDE build is workspace agnostic - IResource awareness
101966  "Export as deployable plugins" fails when prerequisite plugin is linked binary project 
What happens here with respect to the remote resource work going on
 
PDE Build only compiles java code - Builder awareness (35371)
104156  PDE build.xml should use all project builders 
74880   PDE build does not use project class libraries 
101241  Plugin exporter should support exporting plugins from their output folder


Tractable 3.2 solution
=================
Given the priority, time and resource constraints known as I write this, for 3.2 we are planning on using 
a band aid approach that will satisfy / heal a large set of people (I would say around 80%). However this 
is without counting the evolution coming in 3.2 in the resource model (remote resources).

Redoing pde build
===============
NOTE: Before going down this path, we would first need to get the community feedback with respect to 
the issues that are generated, understand how far we can go and in which timeframe, and get help from 
the community.

If we had more time and resources, PDE build could be re-written to fully to be workspace based. It 
would become a generic build mechanism.
The benefits for the end users would be:
- if it compiles in the workspace it will build in batch mode
- all the preferences and settings are reused
- reuse all the builders
- support all the repo that are integrated in eclipse and support PSF files (potential incremental fetch)
- no problem for cyclic dependencies
- incremental building (at least for java)

For the PDE Build maintenance
- same code base than pde core (no need to duplicate things)

Problems that could happen:
- Less transparence in the build mechanism (a lot of steps occur in java)
- Scalability issues:
    - can we build large products all at once?
    - builder scalability (I'm afraid of bad builders that needs to load a huge model to do any operation)
- Harder to provide a source build to community (mainly a bootstrap pb, however we could  provide ant 
scripts to build the builder, and then tell people to use the builder to build other projects).

On the surface it sounds easy, however scratching the surface discover challenges
since creating a product drop is not as simple as "fetch the code in the workspace and let the builders 
build". This actually only represents half of the process, since the "binary version of the plugin" needs 
to be created (and it is not only the .class files, it is also the plugin.xml, the manifest, the images, etc... 
basically what is listed today in the build.properties under bin.includes) -this is known as assemble in 
pde build terminology-, and finally the final output needs to be created (zip, tar, folder) which is known 
as packaging.

Moreover, there are all those steps that "just" happen and that would have to be run only while 
exporting the product:
- how do we get the compilation problems?
- how do we sign?
- how do we force version numbers in manifests without making a change in the workspace?
- how do we force the version numbers in features?
- how do we generate source features and plugins?
- how do we provide hooks in the process?
- how do we brand the executable?
- how do we do jnlp file generation?
- how do we make sure that the libraries and JRE referred to by various projects are correct (not 
hardcoded path)?
- how do we control what goes into the target?
- how would the packager functionality be implemented?

The list shows clearly that thinking "it is building in the workspace" so it will just build in batch mode is 
not as straight as it may look and not everyone in the team may be able to guarantee that the build will 
just work by running it.

Here is a just a list of other problems that needs to be considered: 
- do we want to write everything in java?
- do we keep bin.includes in the build.properties or can everybody write its own build.xml file for the 
equivalent of the gather.bin.parts
- what about backward compatibility?