Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 348428 - [publisher] Allow platform-independent product publishing
Summary: [publisher] Allow platform-independent product publishing
Status: RESOLVED FIXED
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: p2 (show other bugs)
Version: 3.7   Edit
Hardware: PC Windows Vista
: P3 normal (vote)
Target Milestone: Juno M1   Edit
Assignee: Shenol Yousouf CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-06-06 11:50 EDT by Shenol Yousouf CLA
Modified: 2014-12-23 14:08 EST (History)
7 users (show)

See Also:


Attachments
Patch for publisher (1.99 KB, patch)
2011-06-23 10:51 EDT, Shenol Yousouf CLA
pascal: iplog+
Details | Diff
Tests for "-configs ANY" publishing (16.79 KB, patch)
2011-06-23 10:56 EDT, Shenol Yousouf CLA
pascal: iplog+
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Shenol Yousouf CLA 2011-06-06 11:50:13 EDT
Build Identifier: I20110519-1138

At the moment, the startup settings for bundles in the <configurations> section of the .product file creates configuration units (CUs) with filters. The filters are related to the platforms, specified by the "-configs" parameter of the product publisher application. Installation with a p2 profile, which does not match these filters, will discard the CUs. If no "-configs" parameter is specified, the CUs are not created and again the startup settings are not applied during installation. It turns out that the product is always supposed to be published and installed for a pre-defined set of environments.

Although there are use cases, which require such behavior, this assumption does not always hold true. The user should be allowed to publish its products for any platform, without any  workarounds, if the product has no special requirements to the environment (e.g. no native parts).

What are the current options to utilize ?

The example on http://wiki.eclipse.org/Equinox/p2/Setting_Start_Levels shows one way to define a filter-less CU for a particular plug-in with p2.inf file. No filters means that the touchpoint instructions will be executed on any installation. Unfortunately, the workaround is very user-unfriendly to apply for the configuration of many units at a time.

Some people use the "-configs all" option during product publishing but it again generates the CUs with filter osgi.ws=all which requires a special configuration of the p2 profile (e.g. to execute the installation with p2 director with option "-p2.ws all") and is not a straightforward solution.

Reproducible: Always
Comment 1 Shenol Yousouf CLA 2011-06-06 11:52:39 EDT
There are possibly many approaches to solve the problem but probably the most direct one is to enable the generation of CUs without filters by the product publisher (and not via p2.inf, for example).

We could offer several ways to the user to specify the creation of such CUs:
- the "-configs all" can be modified to truly allow publishing "for all platforms"
- ...or when no "-configs" parameter is passed to the publisher
- an option with a finer granularity is to allow the .product author to point out for which plugins in the <configurations> section the filters should not be generated; for example, this can be achieved with an additional attribute in the <plugin> element:
<plugin id="my.bundle" autoStart="true" startLevel="0" nofilters="true"/>

What direction would you suggest to follow ?
Comment 2 Pascal Rapicault CLA 2011-06-07 00:44:14 EDT
Having thought about it for about 5 min I would go with the proposal of implementing the new behaviour when -configs is not specified, but internally use a special token (something similar is used in PDE build with a token called ANY) to differentiate this case.

As for changes to the .product file format itself, I would stay away from it unless you also want to change the editor in PDE UI to read / write this file. That said if some changes are really needed, it is always possible and if we are going down that path it may be an opportunity to do a large cleanup, but this is outside of p2 scope :)
Comment 3 Shenol Yousouf CLA 2011-06-09 12:37:46 EDT
"internally use a special token (something similar is used in PDE build with a token called ANY) to differentiate this case"

Could you please elaborate on your idea ? I am afraid that I do not understand it fully. An example would be nice.

The default behavior of no "-configs" parameter is no configuration units published. Would you want it to stay the same way by default but still allow the users to specify that they want these CUs without filters ? For example, this could be a combination of no "-configs" parameter and a special system environment variable, which, when set, will notify the publisher not to create the filters. I would avoid introducing a property which is hidden for the publisher users (if not adequately documented) but, on the other hand, it would require no changes in the publisher's arguments, if this is the purpose, and is fully backward compatible.

Trying to guess what you mean by "token called ANY" :), I encountered such string constant in AbstractPublisherAction which replaces the missing and "*" slots from the <os><ws><arch> triplet (e.g. "-configs all" or "-configs all.*.*" is translated to "-configs all.ANY.ANY"); in the later phases of publishing, a configuration spec ANY will not generate a filter for the corresponding environment parameter. However, such transformations are still triggered by something specific in the input - something which, as far as I understand, we'd want to avoid.

As a side note, in theory, "-configs *" and "-configs *.*.*" should generate the configuration units without filters. Unfortunately, "*", without any alphanumeric characters beside it and passed as an input to an Eclipse application, is expanded to the file list in the current directory, so it never gets as a value to the publisher application and defeats the purpose of the whole show. :)
Comment 4 Pascal Rapicault CLA 2011-06-09 23:21:31 EDT
After reading you answer and looking at the code some more I suggest the following: 
- From the end user point of view, when -configs is missing, we keep doing what we do today (keep "backward compatibility"). And then, when -configs * is used, the CUs are generated without any filter.
- From the code point of view it means changing the code in ConfigCUsAction to do the right thing when * is specified and not use the * as the value of the filer.

Makes sense?
Comment 5 Shenol Yousouf CLA 2011-06-10 03:15:54 EDT
Uhm, as noted above, the technical problem with "-configs *" and with the "*" symbol as a standalone argument, in general, is that it is expanded to the file list in the current directory long before AbstractPublisherApplication.run() is called. So if you call the application with "-configs *", it sees in its args[] array something like this:
[..., -configs, <file name>, <file name>,...]
The "*" character is preserved only if it is combined with other alphanumeric characters, e.g. as in "-configs *.win32.*".

In short, the publisher application cannot guess that it has been called with "-configs *" unless applying some exotic solution - for example, detect that -configs is followed by file names in args[]. From that point of view, I don't think that your proposal can be implemented, unless I am missing something.

Looking at the AbstractPublisherAction.createFilterSpec() method and how CONFIG_ANY is interpreted there, I am tempted to propose "-configs ANY" as a candidate for creating filterless CUs - the implementation will require just to replace the comparisons by reference with comparisons by value. Unfortunately, this is not backward compatible (currently, "-configs ANY" will generate filter "osgi.ws=ANY") but, on the other hand, I see no point why a user would want to publish his/her product with a configuration that would not match any conventional system environment.
Comment 6 Pascal Rapicault CLA 2011-06-10 10:13:00 EDT
(In reply to comment #5)
> Uhm, as noted above, the technical problem with "-configs *" and with the "*"
> symbol as a standalone argument, in general, is that it is expanded to the file
   d'oh! That's what happens when you watch a movie and answer bug at the same time :p

Let's go with ANY as the keyword.
Comment 7 Shenol Yousouf CLA 2011-06-23 10:51:17 EDT
Created attachment 198470 [details]
Patch for publisher

The simplest implementation of the proposed bugfix as we discussed it - when creating the LDAP filters for configuration units, the comparisons with the CONFIG_ANY constant by reference are replaced with comparisons by value (case insensitive). Thus, when the input config spec is "ANY", "ANY.ANY.ANY" and so on, the configuration units will be created without filters. Nothing is changed if the specs are different.

It is important to note that this minimal modification in publisher targets only the removal of filters from CUs in general (or, it is more proper to say, not to assign any filters to them) but it does not affect the publishing flow in any other aspect, most notably the decisions in the code whether to create the CUs at all. The latter is most relevant for EquinoxLauncherCUAction and EquinoxExecutableAction which handle native artifacts and it is proper that their behavior is not affected.

In any case, the idea is that publishing with "-configs ANY" will be a conscious decision and used only for specific purposes, such as the publishing of platform-independent products. "-configs" with any other value preserves its old behavior.
Comment 8 Shenol Yousouf CLA 2011-06-23 10:56:36 EDT
Created attachment 198472 [details]
Tests for "-configs ANY" publishing

The proposed test cases, on one hand, demonstrate the new desired behavior, introduced with the "ANY" configuration spec, and on the other hand, verify that the result items from the CU and product actions does not differ in any other aspect except that the created CUs are without filters.

In fact, the new ANYConfigCUsActionTest (I admit that the test name is not quite inspiring :)) was developed as a variant of the old ConfigCUsActionTest, enhanced with checks for absent filters plus analysis of bundle configuration unit properties.

A new method is added to ProductActionTest to also verify the creation of bundle configuration units according to the expectations during product publishing.
Comment 9 Pascal Rapicault CLA 2011-06-23 22:06:40 EDT
Patch released in HEAD. Thx for shopping at p2, please come again :)
Comment 10 Pascal Rapicault CLA 2011-06-23 22:21:14 EDT
Ah, I forgot one thing. Next time please update copyrights in the files you create.
Comment 11 Pascal Rapicault CLA 2011-06-23 22:21:31 EDT
and / or change.
Comment 12 Sébastien Gandon CLA 2014-12-19 05:33:23 EST
Hello,
Could someone tell me if the no-filter p2 publish method could also be used in Tycho when materializing products ?
Thanks.
Comment 13 Tobias Oberlies CLA 2014-12-23 14:08:49 EST
(In reply to comment #12)
> Could someone tell me if the no-filter p2 publish method could also be used in
> Tycho when materializing products ?
Not that I am aware of. There is some off-topic discussion around this in bug 344095, but we really need a separate bug report for this.