Community
Participate
Working Groups
As a release engineer, when building a bugfix release of a project, I'd like to be able to only ship bundles and features that have changes since the original release of the project and keep the original version of all artifacts thats did not change. Currently, tycho unconditionally generates new bundle qualifier and therefore new version of bundles and features. The new qualifier gets included in feature.xml files and generated p2 metadata, which makes redelivery of otherwise unchanged code the only practically feasible option. There are two fundamentally different approaches to solve this problem (note, though, that implementation will likely be very similar in both cases). It is possible for projects to either use fully qualified versions and change the version manually whenever there is a code change. There are two problems with this approach. First, manual maintenance of build qualifier is error prone. Second, generated artifact can change even when there was no code change, i.e. build configuration changed, build tools have changed, build environment has changed, etc. To avoid both of these problems we need to add ability to compare generated build output with baseline version and fail the build if the two versions differ. Alternatively, it is possible to build project output with new generated qualifier, but add an extra step in the build lifecycle that will compare the new build output with the baseline version ignoring the qualifier. If the two versions are the same, the new version is discarded and the old one is used instead.
For me the second option about filtering sounds like the best and most manageable option.
What about feeding a "version.properties" (or something like that) to Tycho which contains a mapping of artifact ids (bundle symbolic names) to qualifiers? That can be used as a base table. Later when mirroring artifacts to the repository, filtering may be used to prevent mirroring of artifacts with the same content. Well, "same" is interesting as sometimes class files may change as a result of a re-compile. There is some interesting point about the "baseline" comparison, though. It may be suitable to integrate API tooling which aids in detecting missing version increases.
(In reply to comment #2) > What about feeding a "version.properties" (or something like that) to Tycho > which contains a mapping of artifact ids (bundle symbolic names) to qualifiers? > That can be used as a base table. I actually find the second option described in comment #0 more promising. I even believe that this should become doable once I'm done with bug 364134: I'm currently working towards introducing a single object representing the build output of a module, e.g. the bundle with its p2 metadata. Once we have this, we only need to compare this to the predecessor (e.g. from a separately configured p2 repository) and replace the entire module output with the old result in case the jars are the same. > There is some interesting point about the "baseline" comparison, though. It may > be suitable to integrate API tooling which aids in detecting missing version > increases. For this bug, we are not talking about API comparison, but rather comparing that the binaries are identical (module time-stamps and qualifier). AFAIK there is a tool for this in p2 that we could re-use.
I believe there are two related usecases. "Patch release", is when build is expected to produce some delta compared to some baseline, with unchanged bundles/features carried over from the original build. "Rebuild of historical version", is when build needs to recreate bundles/features and their version qualifiers as they were originally built. The enhancement request tracked in this bugzilla addresses "Patch release" usecase. "version.properties" maybe useful to implement "Rebuild of historical version", although I'd personally prefer to capture build qualifier in pom.xml (which is already possible, btw).
I was thinking of a third case, when qualifiers are generated on the fly, i.e. the ".qualifier" string is used in the manifest and replaced at build time with a generated timestamp. The resulting bundle is then compared to a baseline and thrown away if it didn't change, i.e. no additional/removed content and the class files are the same. The "version.properties" approach is also useful if the ".qualifier" replacement should not be driven by timestamps but by some other data (eg. SCM tags). For example, the Eclipse build has a script that analyses the Git history of a project ("module"). If the history indicates changes in the project the repository is tagged and the tag is recorded in such a version file which is feed to the build.
(In reply to comment #5) > I was thinking of a third case, when qualifiers are generated on the fly, i.e. > the ".qualifier" string is used in the manifest and replaced at build time with > a generated timestamp. The resulting bundle is then compared to a baseline and > thrown away if it didn't change, i.e. no additional/removed content and the > class files are the same. > How is it difference from "Patch release"? > The "version.properties" approach is also useful if the ".qualifier" > replacement should not be driven by timestamps but by some other data (eg. SCM > tags). For example, the Eclipse build has a script that analyses the Git > history of a project ("module"). If the history indicates changes in the > project the repository is tagged and the tag is recorded in such a version file > which is feed to the build. Is this versions file generated for each build from the local checkout? In any case, as already mentioned in comment #3, this enhancement request is about shipping bundles/features from an earlier baseline build. Please open a separate bugzilla if you need non-timestamp based version qualifiers and we'll discuss it there.
(In reply to comment #6) > How is it difference from "Patch release"? I wasn't sure from your description that "Patch release" also includes a comparison down to the class level, i.e. discover bundles which are "equal" but have different qualifiers. > Is this versions file generated for each build from the local checkout? It's updated incrementally. After the checkout the Git history of the modules is analyzed. If there are relevant commits "after" the recorded tag a new tag is created and the entry in the file is updated and committed and pushed. > In any case, as already mentioned in comment #3, this enhancement request is > about shipping bundles/features from an earlier baseline build. Please open a > separate bugzilla if you need non-timestamp based version qualifiers and we'll > discuss it there. I'll think more about it before I'll do so. IMHO with Tycho the script could be better implemented with a Maven/Tycho plugin that implements a custom logic for generating a module's qualifier.
I believe this is ready for testing. See http://wiki.eclipse.org/Tycho/Reproducible_Version_Qualifiers for description of how to set it up and how it works.
