Community
Participate
Working Groups
Today I started to build using 3.7RC1 only to find that nothing was actually built any more. In the problematic case the build.xml generated into our feature has an empty "all.plugins" target. Experiments seem to show that the problem occurs if a plugin is built that already exists with the same version in the base. (for the curious: we've been using this setup for many years in order to bootstrap our build: first create a fresh compiler that shall replace the original compiler, before starting a second build that uses the compiler from phase 1). This issue pretty much seems to match with a recent change in pde build: bug 330006, where Andrew commented (bug 330006 comment 7): "I'm not comfortable changing the equals/hashCode for FeatureEntry at this point in the release because it is widely used throughout PDE/Build and p2.publisher." Actually, the comment mentions a change that "was required in PDE/Build's FeatureGenerator for testing." It seems that change has been committed. If that was intended for testing only, could this just be reverted??
OK, with Indigo RC1 approaching I couldn't wait, so I spend time to work around this breakage. Reducing severity because I can now build again.
Can you give more details on your build setup. Is it a product build? What was the last version of PDE/Build that was working for you? I don't believe that 330006 is the source of your problems, it affects the FeatureGenerator which is only used to generate a feature based on a .product file. Specifically, the only difference should be when adding additional plugins/features to the generated feature using the "pluginList" or "featureList" properties and when setting specific attributes (unpack,os,arch,etc) on that entry. You could compare the ${buildDirectory}/features/org.eclipse.pde.build.container.feature/feature.xml from your failed build to a previous successful one to see if there is any difference. More generally, when two instances of the same version of a plugin exist for PDE/Build, which one gets chosen is essentially random based on the results of resolving an OSGi state. PDE/Build makes no effort to choose source over binary. The "all.plugins" target will only include the plugins that were compiled from source, if a binary version was chosen instead, it will not show up here.
thanks for looking into this, Andrew, (In reply to comment #2) > Can you give more details on your build setup. Is it a product build? the build is feature based. > What was the last version of PDE/Build that was working for you? 3.7 M7. > You could compare the > ${buildDirectory}/features/org.eclipse.pde.build.container.feature/feature.xml > from your failed build to a previous successful one to see if there is any > difference. I don't have features/org.eclipse.pde.build.container.feature I'm building from my own features fetched from SVN. > More generally, when two instances of the same version of a plugin exist for > PDE/Build, which one gets chosen is essentially random based on the results of > resolving an OSGi state. PDE/Build makes no effort to choose source over > binary. > > The "all.plugins" target will only include the plugins that were compiled from > source, if a binary version was chosen instead, it will not show up here. That could well explain. So in previous years I was just lucky that the OSGi state would favor the source plugin and the fact that my plugin was compiled by PDE/Build was only due to this lucky chance? In that case *any* change in equinox could have flipped that chance from source to binary? Are you saying, PDE/Build cannot reliably be used to re-build a plugin of the previous version (which already exists in base)? I kind-of remember a discussion where some relengs emphasized that re-creating an existing version was very important to them, no? Wouldn't it be better to prefer source over binary instead of a random selection (which can really drive you nuts :) ...)? As mentioned, I've changed my build so neither does this issue impact me any more, nor can I easily make experiments to investigate further.
(In reply to comment #3) > the build is feature based. > I don't have features/org.eclipse.pde.build.container.feature In this case, bug 330006 is definitely not related. Baring any other differences, it is possible that bug 344631 is the source of the change > That could well explain. So in previous years I was just lucky that the OSGi > state > would favor the source plugin and the fact that my plugin was compiled by > PDE/Build was only due to this lucky chance? > In that case *any* change in equinox could have flipped that chance from > source to binary? As a general guideline, I believe the osgi resolver will tend (but is not guaranteed) to prefer bundles with a lower bundle-id (a unique number assigned when the bundle is installed into osgi), for PDE/Build this is bundles that were added to the state first. PDE/Build adds bundles in the order they were returned by File.listFiles (vm/os dependent, not guaranteed in any order) first for pluginPath, then buildDirectory, then baseLocation. Generally this puts source (buildDirectory) before binary (baseLocation). However with the advent of p2 (using transformedRepoLocation) we are more often finding binary content in pluginPath. > > Are you saying, PDE/Build cannot reliably be used to re-build a plugin of > the previous version (which already exists in base)? > > I kind-of remember a discussion where some relengs emphasized that > re-creating an existing version was very important to them, no? Yes, releng setup want to be able re-create a version, but they are not generally doing this in the presence of that same binary version they are recreating. Generally you want to do this in as close as possible the same setup that was used originally. Note however, that with p2 there is a strong assertion that bundle symbolic name + bundle version === one unique set of bytes. Republishing new content under the same name + version is discouraged, p2 won't update a bundle if the version did not change. It is difficult to support a user when you don't know which content he has. I would suggest that if you are recompiling the compiler for your own use, then you could increment the version by appending "\nqualifier=z<TIMESTAMP>\n" to the build.properties file for that bundle. OSGi would then favour it over the other binary.
(In reply to comment #4) > I would suggest that if you are recompiling the compiler for your own use, then > you could increment the version by appending "\nqualifier=z<TIMESTAMP>\n" to > the build.properties file for that bundle. OSGi would then favour it over the > other binary. When we started migrating our pre-p2 build to a p2-based setup, we had to use the exact same plugin version&qualifier for the replacement to be picked up during the 2nd phase build. Everywhere else we do use qualifiers, and my fix in this issue was to use a patch feature so we can indeed inject a bundle with higher version number into the builder, and the p2 director has matured to reliably perform this installation at build time. So, if nobody joins in I'm fine with closing as WONTFIX. Anyway I'm glad this random selection has been documented here.
Closing this as WORKSFORME. Please raise a new feature enhancement if we want to do something to prefer source over binary.
