Bugzilla – Full Text Bug Listing
| Summary: | Target platform filtering | ||
|---|---|---|---|
| Product: | z_Archived | Reporter: | Tobias Oberlies <t-oberlies> |
| Component: | Tycho | Assignee: | Tobias Oberlies <t-oberlies> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | enhancement | ||
| Priority: | P3 | CC: | igor, jan.sievers, kane.mx, kane.zhu, pwebster, slewis |
| Version: | unspecified | ||
| Target Milestone: | --- | ||
| Hardware: | All | ||
| OS: | All | ||
| Whiteboard: | |||
|
Description
Tobias Oberlies
The patch of bug 363025 implements supporting specified features and plug-ins for the installation type of .target file. (In reply to comment #1) > The patch of bug 363025 implements supporting specified features and plug-ins > for the installation type of .target file. It is correct, that target files also have an option to remove bundles from the target file content. However this enhancement aims for a somewhat more general approach: - It shall apply to Tycho's overall target platform, and not only to the target file content - It shall allow filtering on both bundle and package level, i.e. apart from removing bundles, it shall also be possible to select one provider for a package/bundleId This doesn't mean that we could/should also support the bundle selection in target files. If you are planning to contribute this, please open a new enhancement request. IIRC, the bundle selection is independant of the location type, and should therefore not be mixed with support for location type="Profile" (bug 363025) I'll implement at least some filtering support within the next couple of days. With commit 201655d, there is now support for restricting versions of bundles/packages/units, removing bundles/units altogether and restricting the providers of a package. I still need to add log messages and write documentation, but if anyone is super-keen on trying this out, look at the integration test [1] and further syntax examples [2]. [1] http://git.eclipse.org/c/tycho/org.eclipse.tycho.git/tree/tycho-its/projects/target.restriction.filter?id=201655ddf41b17015bec0f6ec3cb425cdd59a472 [2] http://git.eclipse.org/c/tycho/org.eclipse.tycho.git/tree/tycho-core/src/test/resources/targetfilter?id=201655ddf41b17015bec0f6ec3cb425cdd59a472 My main user story for this feature is this: as a Tycho user, I want to configure my project build to use specific provider of a certain capability.
Although the filtering syntax looks kinda clumsy, I think it allows me do what I need. Using the following two examples, can you confirm I got the syntax right?
If I want "Import-Package:javax.servlet" to resolve into javax.servlet bundle version 2.5.0 (i.e. not the servletbridge crap), the filter should be
<filter>
<type>java-package</type>
<id>javax.servlet</id>
<restrictTo>
<type>osgi-bundle</type>
<id>javax.servlet</id>
<versionRange>[2.5.0,2.6.0)</versionRange>
</restrictTo>
</filter>
If I want "Require-Bundle:org.eclipse.jdt.core" to resolve into org.eclipse.jdt.core bundle version 3.7.1.v_B76_R37x (i.e. not the OTDT crap), the filter should be
<filter>
<type>eclipse-plugin</type>
<id>org.eclipse.jdt.core</id>
<restrictTo>
<type>eclipse-plugin</type>
<id>org.eclipse.jdt.core</id>
<versionRange>[3.7.1.v_B76_R37x,3.7.1.v_B76_R37x]</versionRange>
</restrictTo>
</filter>
Did I get this right?
(In reply to comment #5) > Did I get this right? Yes, both examples should do exactly what you described. The second filter can also be written as <filter> <type>eclipse-plugin</type> <id>org.eclipse.jdt.core</id> <restrictTo> <version>3.7.1.v_B76_R37x</version> </restrictTo> </filter> The <restrictTo> section "inherits" type and id if they are not specified, and you can either use version or versionRange. I have couple of comments/observations about proposed syntax.
First, the syntax looks too complicated and I think it will be hard to explain how to use it properly.
Second, I believe it is too easy to express contradicting filtering conditions using the proposed syntax.
I wonder if filter exclude/keep configuration, something like below, is more appropriate here. I believe it is functionally equivalent to the proposed syntax, i.e. it can express everything that can be expressed in the originally proposed syntax and vise versa, but I think it will be easier to explain, understand and ultimately use properly.
<filter>
<excludes>
<exclude>
<!-- exclude IUs that provide this capability -->
<ns></ns>
<id></id>
<versionRange></versionRange>
</exclude>
...
</excludes>
<keeps>
<keep>
<!-- keeps these IUs regardless of excludes -->
<id></id>
<versionRange></versionRange>
</keep>
...
</keeps>
</filter>
(In reply to comment #7) > First, the syntax looks too complicated and I think it will be hard to explain > how to use it properly. I checked with Jan before implementing it, and he understood the syntax for the common cases (version restriction, package provider selection) without further explanations. I believe that people will just copy the two templates (which I still need to provide in the wiki) and be happy. > Second, I believe it is too easy to express contradicting filtering conditions > using the proposed syntax. This can happen, but I plan to issue a warning in this case. There will also be debug output which explains what happens in more details (e.g. "filter xyz applies to 3 units; restrictTo pattern matches 1 unit"). > I wonder if filter exclude/keep configuration, something like below, is more > appropriate here. I believe it is functionally equivalent to the proposed > syntax, i.e. it can express everything that can be expressed in the originally > proposed syntax and vise versa, but I think it will be easier to explain, > understand and ultimately use properly. If I understand it correctly, the exclude/keep syntax is more powerful, because it would allow retaining multiple providers of a package. This is not possible with the proposed syntax (and I'm not sure if we really need this). Still, I prefer my original proposal: - I suppose that the exclude/keep syntax is more verbose for the bundle version restriction case, because the bundle namespace&id has to be repeated each in exclude and a keep. - IMHO it will become even easier to remove all providers of a package, because it is far from evident which one of the "keep"s is supposed to provide the package which was excluded. (In reply to comment #8) > - IMHO it will become even easier to remove all providers of a package, because > it is far from evident which one of the "keep"s is supposed to provide the > package which was excluded. Overly aggressive filter configuration will result in fast failure with obvious error message, so I do not believe this is a big concern. I sent an email to tycho-dev and tycho-users, so lets wait for some feedback before we decide. (sadly, eclipse mailing lists appear to be down for last few days, but hopefully things get back to normal after the holidays are over). I played with proposed <restrictTo> syntax for a fairly large multimodule project. <restrictTo> syntax does not seem to work well with maven configuration inheritance. I wanted to define a bunch of essentially "excludes" in the parent pom, and then override some of these excludes for individual projects (one of my projects actually needs servletbridge.extensionbundle, believe it or not). I think this will be a common usage scenario and there does not seem to be a good (any?) way to achieve this with <restrictTo>. Excludes/keeps syntax works much better in this case. Actually, neither of the filtering syntax works well bug 348045 (i.e. the servletbridge crap). The root cause of the problem are the extra provided capabilities of servletbridge, so the right solution would be to tell Tycho to ignore those provided capabilities, i.e. "do not consider servletbridge as a provider of javax.servlet package". All other approaches are bound to have undesired side effects. I am not sure if target platform filtering actually provides any useful behaviour, at least I do not see a usecase well supported by it. Even the original user story from comment #0 "I want to add a newer version [of eclipse platform] to the target platform and ensure that only that newer version can be used" suggests version constraints on "resolved" dependencies, not target platform filtering. Maybe we should really keep the target platform as low-level "dumb" bucket of installable units and apply all filtering, version constraining, etc, at higher-levels. (In reply to comment #10) > I played with proposed <restrictTo> syntax for a fairly large multimodule > project. <restrictTo> syntax does not seem to work well with maven configuration > inheritance. I wanted to define a bunch of essentially "excludes" in the parent > pom, and then override some of these excludes for individual projects (one of my > projects actually needs servletbridge.extensionbundle, believe it or not). I am assuming that you worked with combine.children="append", right? So we would need a way to override a filter from further up in the list of filters. IMHO this is doable: <filter override="true"> <type>eclipse-plugin</type> <id>org.eclipse.osgi</id> <keepAll/> </filter> could disable any inherited type=eclipse-plugin, id=org.eclipse.osgi filters. And in any case, there is always the possibility to override the whole <filters> element. [1] http://www.sonatype.com/people/2007/06/how-to-merge-sub-items-from-parent-pom-to-child-pom-in-a-maven-plugin-configuration-2/ (In reply to comment #11) > I am not sure if target platform filtering actually provides any useful > behaviour, at least I do not see a usecase well supported by it. Even the > original user story from comment #0 "I want to add a newer version [of eclipse > platform] to the target platform and ensure that only that newer version can be > used" suggests version constraints on "resolved" dependencies, not target > platform filtering. Removing all but one version from the target platform entails the same constraint on the resolved dependencies. (In reply to comment #11) > Maybe we should really keep the target platform as low-level "dumb" bucket of > installable units and apply all filtering, version constraining, etc, at > higher-levels. There will be many different mechanisms that work on the target platform content: the p2 resolver for the compile class path and test runtime, direct lookup of the latest artifact for features, the p2 director for products and a p2 slicer for eclipse-repository. For something with effect everywhere, we need to control over the target platform content. But feel free to implement additional parameters/filtering/etc. for some of the users. As long as they don't imply a global effect by being part of the target-platform-configuration. (In reply to comment #13) > (In reply to comment #11) > > Maybe we should really keep the target platform as low-level "dumb" bucket of > > installable units and apply all filtering, version constraining, etc, at > > higher-levels. > There will be many different mechanisms that work on the target platform > content: the p2 resolver for the compile class path and test runtime, direct > lookup of the latest artifact for features, the p2 director for products and a > p2 slicer for eclipse-repository. For something with effect everywhere, we need > to control over the target platform content. > But feel free to implement additional parameters/filtering/etc. for some of the > users. As long as they don't imply a global effect by being part of the > target-platform-configuration. Yes, I agree, target platform filtering is orthogonal to dependency resolution and should be configured separately. I've changed the summary to better reflect what is being implemented under this enhancement request and I will use bug 363331 to cover the work on dependency resolution enhancements and corresponding configuration (bug 363331 needs new summary, too). move all open bugs to post 0.14.0 Target platform filtering is available in 0.14.0 (see documentation here [1]). I had kept this open because I wanted to add more debugging in 0.14.0, but I didn't manage to do this on time. There is some more debug output in 0.15.0-SNAPSHOT, but it turned out to be more complicated than I thought, so there is a follow-up for this: bug 372035. [1] http://wiki.eclipse.org/index.php?title=Tycho/Target_Platform#Filtering (In reply to Tobias Oberlies from comment #16) > Target platform filtering is available in 0.14.0 (see documentation here > [1]). > > I had kept this open because I wanted to add more debugging in 0.14.0, but I > didn't manage to do this on time. There is some more debug output in > 0.15.0-SNAPSHOT, but it turned out to be more complicated than I thought, so > there is a follow-up for this: bug 372035. > > [1] http://wiki.eclipse.org/index.php?title=Tycho/Target_Platform#Filtering After consulting this web page, something is unclear for me: Does the filtering syntax include support for wildcards?...e.g. <filter> <type>p2-installable-unit</type> <id>org.eclipse.*</id> <removeAll /> </filter> (In reply to Scott Lewis from comment #17) > After consulting this web page, something is unclear for me: Does the > filtering syntax include support for wildcards?...e.g. > > <filter> > <type>p2-installable-unit</type> > <id>org.eclipse.*</id> > <removeAll /> > </filter> looks like right now the id filter checks equality https://github.com/eclipse/tycho/blob/9e72e1394c5d80139be3378e5231668b2b429a8b/tycho-bundles/org.eclipse.tycho.p2.resolver.impl/src/main/java/org/eclipse/tycho/p2/target/filters/TargetPlatformFilterEvaluator.java#L231 extending this to support wildcards/regex would be an enhancement (In reply to Jan Sievers from comment #18) > (In reply to Scott Lewis from comment #17) > > > After consulting this web page, something is unclear for me: Does the > > filtering syntax include support for wildcards?...e.g. > > > > <filter> > > <type>p2-installable-unit</type> > > <id>org.eclipse.*</id> > > <removeAll /> > > </filter> > > looks like right now the id filter checks equality > https://github.com/eclipse/tycho/blob/ > 9e72e1394c5d80139be3378e5231668b2b429a8b/tycho-bundles/org.eclipse.tycho.p2. > resolver.impl/src/main/java/org/eclipse/tycho/p2/target/filters/ > TargetPlatformFilterEvaluator.java#L231 > > extending this to support wildcards/regex would be an enhancement as a side note, if you need wildcards for removing bundles for you target platform, maybe the "blacklist" approach where you add everything from a repo to your target platform and then remove many bundles again is not the most straightforward solution. If you want tight "whitelist" control over which bundles are available in your target platform, consider using a .target file [1] [1] https://wiki.eclipse.org/Tycho/Target_Platform#Target_files |