Community
Participate
Working Groups
Packaging of eclipse-repository RCP application fails when indirect dependencies are introduced with p2.inf file. The attached example project has the following simplified dependencies product -> feature p2.inf -> org.eclipse.m2e.maven.runtime As an observation, everything works when the same dependency is defined in feature.xml directly.
Created attachment 208296 [details] sample project that demostrates the problem
The missing dependency is coming from outside the reactor, right? Then the root cause is that eclipse-repository uses the resolved project dependencies instead of working on the target platform directly. The difficult part of this switch is to make sure that includeAllRequirements doesn't include too much afterwards. Are you planning to work on the root cause?
I have problems both with reactor and outside-of-the-reactor dependencies and I think there is more that one underlying cause First, dependency-only metadata generated from feature.xml and .product files did not include information from p2.inf file. This is fixed now with http://git.eclipse.org/c/tycho/org.eclipse.tycho.git/commit/?id=4c6b0294b6f0493064a33ace490dec8556fe71da . This made the attached project work. Second, I am not sure if packaged eclipse-repository uses dependency-only or full metadata of other reactor projects. It looks like it's the full metadata, but I don't understand how this works yet. There is also a problem with .qualifier expansion in .product file, i.e. current code appears to use qualifier of eclipse-repository project instead of the included dependency and fails because of .qualifier mismatch. I did not look into this yet and don't know if this is a separate issue or not. You are probably right that there is also an issue with resolved dependencies vs (dumb) target platform, but this did not cause me any trouble yet so I can't comment on it.
Within the reactor, the problem is that some module build results do not become part of the (full metadata) target platform for the eclipse-repository module. (In the code, the full-metadata target platform is called "visible repositories" - cleaning this up is a pending task for bug 364134.) From the reactor only those modules are added to that target platform to which there is a dependency in the Maven model. The Maven dependencies are injected from the p2 dependency resolution result, and that again uses the dependency-only metadata. So if you fixed the dependency-only metadata, the symptom will also go away. Still, I think that this concept is hard to keep up: It requires to know with a high fidelity all potential dependencies in the reactor before we have the real data to do dependency resolution. This is why I came up with the idea to base the full-metadata target platform on the build order: everything which has been built earlier in the reactor becomes part of the full-metadata target platform. Then we only need a rough automatic ordering of modules, which could be based on a approximate dependency-only publishing, and if that isn't enough, there is always to option to manually tweak the build order.
(FYI, there are more problems with p2.inf, I am still working on this) (In reply to comment #4) > Within the reactor, the problem is that some module build results do not become > part of the (full metadata) target platform for the eclipse-repository module. > (In the code, the full-metadata target platform is called "visible > repositories" - cleaning this up is a pending task for bug 364134.) From the > reactor only those modules are added to that target platform to which there is > a dependency in the Maven model. The Maven dependencies are injected from the > p2 dependency resolution result, and that again uses the dependency-only > metadata. So if you fixed the dependency-only metadata, the symptom will also > go away. > > Still, I think that this concept is hard to keep up: It requires to know with a > high fidelity all potential dependencies in the reactor before we have the real > data to do dependency resolution. > This is why I came up with the idea to base the full-metadata target platform > on the build order: everything which has been built earlier in the reactor > becomes part of the full-metadata target platform. Then we only need a rough > automatic ordering of modules, which could be based on a approximate > dependency-only publishing, and if that isn't enough, there is always to option > to manually tweak the build order. Reactor build order is not a problem. I am more concerned about .qualifier expansion and other changes to project metadata that happen during the build. We need to make sure that eclipse-repository uses the full metadata, this already appears to be case, but I want to make sure the code actually guarantees this.
eclipse-repository uses the full metadata - this is ensured today (although this is not entirely obvious) - but its build can fail because modules from the reactor are missing altogether. The way that we calculate which modules are to be used by an eclipse-repository module is IMHO too complicated (see comment 4). A better solution would be to make all modules "visible" which have been built before the eclipse-repository module. What do you think about this idea? This should also fix your intra-reactor p2.inf problems. Known drawbacks: - Parallel build would cause problems (because the "module build happens before" relationship is equivalent to "is dependency for" relationship in that mode - so there is no gain) - I have doubts that the Maven API provides the "built before" information, so we would to compute this ourselves, e.g. by flagging projects through a context value.
I pushed another set of changes [1] and now Tycho is able to correctly build RCP applications with p2.inf files during eclipse-repository project build. There is still a problem with packaged repository itself, which does not contain IUs defined in RCP's p2.inf file. Unless I am mistaken, this means the repository won't be usable as install/upgrade source for the RCP. There is a TODO comment in PublisherServiceImpl#publishProduct, which I believe explains the root cause of this problem but I did not have time to look any closer. There is also commented out assertion of expected repository IUs in MultienvP2infTest. I am not sure when I will be able to look into this, so if somebody wants to fix this, feel free to reassign the bug to yourself. [1] http://git.eclipse.org/c/tycho/org.eclipse.tycho.git/commit/?id=bbc050b37a4c11b7a00996fd578bf5da79021ee4
(In reply to comment #7) > There is still a problem with packaged repository itself, which does not > contain IUs defined in RCP's p2.inf file. Unless I am mistaken, this means the > repository won't be usable as install/upgrade source for the RCP. There is a > TODO comment in PublisherServiceImpl#publishProduct, which I believe explains > the root cause of this problem but I did not have time to look any closer. > There is also commented out assertion of expected repository IUs in > MultienvP2infTest. I am not sure when I will be able to look into this, so if > somebody wants to fix this, feel free to reassign the bug to yourself. Here are my findings regarding the commented out section in MultienvP2infTest [1]: By default, IUs (regardless whether declared in p2.inf or elsewhere) will only be assembled in the resulting p2 repository if they are (transitively) required *with a perfect version match* from one of the "root IUs" (i.e. product or feature in category.xml). IIRC a perfect version match requirement is how p2 translates an "include" type of dependency. For the concrete example, if I change requires.3.range in [2] from "0.0.0" to requires.3.range=[1.0.0.qualifier,1.0.0.qualifier] , the IU tychotest.product.macosx ends up in the packaged repository. I also verified that another option to make the inclusion work is to switch on the "includeAllDependencies" flag [3] which will drag in all transitive dependencies regardless whether perfect version match or not. Same goes for the p2.inf in the feature module. I fixed the test for now with commit [4]. Not sure if we should treat this as a bug; looks to me like it works as it should but Tobias is more familiar with this part of the code and may be able to explain better when he returns. BTW I also noticed that .qualifier is not replaced in p2.inf. Is this a separate bug? [1] http://git.eclipse.org/c/tycho/org.eclipse.tycho.git/tree/tycho-its/src/test/java/org/eclipse/tycho/test/bug366456_multienv_p2inf/MultienvP2infTest.java#n57 [2] http://git.eclipse.org/c/tycho/org.eclipse.tycho.git/tree/tycho-its/projects/366456_multienv_p2inf/product/product.p2.inf#n14 [3] https://hudson.eclipse.org/hudson/job/tycho-sitedocs-nightly/ws/target/staging/tycho-p2/tycho-p2-repository-plugin/assemble-repository-mojo.html [4] http://git.eclipse.org/c/tycho/org.eclipse.tycho.git/commit/?id=4c8d9809865e0689d8be1ae94719b8c1102eda37
(In reply to comment #8) > > BTW I also noticed that .qualifier is not replaced in p2.inf. Is this a > separate bug? > This is a good question. We currently replace .qualifier in definitions of bundle/feature/etc, but not in references (i.e. dependencies). If apply this approach to p2.inf, then .qualifier should be replaced in iu.<#>.version entries, but not in provides/requires/etc entries.
move all bugs to post 0.14.0
(In reply to comment #9) > (In reply to comment #8) > > > > BTW I also noticed that .qualifier is not replaced in p2.inf. Is this a > > separate bug? > This is a good question. We currently replace .qualifier in definitions of > bundle/feature/etc, but not in references (i.e. dependencies). If apply this > approach to p2.inf, then .qualifier should be replaced in iu.<#>.version > entries, but not in provides/requires/etc entries. I don't believe that this will be necessary. AFAIK, the magic strings $version$ and $qualifier$ are replaced by p2 with the version of the "main" artifact being published. Can't remeber if I have ever tried this though.
This problem seems to have been fixed in the 0.14.0 timeframe - the sample project builds with that version and the current snapshot.
