Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 338409 - use b3 as a mirroring application
Summary: use b3 as a mirroring application
Status: CLOSED DUPLICATE of bug 338738
Alias: None
Product: CBI
Classification: Technology
Component: CBI p2 Repository Aggregator (show other bugs)
Version: unspecified   Edit
Hardware: All All
: P3 enhancement (vote)
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-02-28 08:40 EST by Bouchet Stéphane CLA
Modified: 2016-09-16 15:52 EDT (History)
4 users (show)

See Also:


Attachments
testcase using swtbot (1.33 KB, application/octet-stream)
2011-03-01 05:23 EST, Bouchet Stéphane CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Bouchet Stéphane CLA 2011-02-28 08:40:40 EST
Hi,

i am using B3 to aggregate p2 updates sites and it is working very great, so thank you for this great tool. 

I was wondering if this is possible to use b3 like a mirroring application. 
given a number of p2 repository, i would like to have the contents automatically mirrored. 
in fact, i am using p2.mirror Ant task, and would like to have similar using b3.

it it possible now ? 

can it be reasonable to implement this ? 

Cheers,
Comment 1 Henrik Lindberg CLA 2011-02-28 14:03:10 EST
The aggregator can aggregate with or without mirroring. 
Can you be more specific what functionality you are looking for?
Comment 2 Bouchet Stéphane CLA 2011-03-01 05:23:42 EST
Created attachment 190029 [details]
testcase using swtbot

Hi,

taking the attached example, if i try to run the aggregation on the file, the result is only the last version of swtbot, and i would like to get all version specified in the file.

I would like to aggregate these different versions on the same repo.
Comment 3 Matt Biggs CLA 2011-03-08 11:30:22 EST
I would also like to see this functionality. Its currently impossible to get multiple versions of a plug-in. 

For example, if you attempt to aggregate the Orbit repository you will only get 1 version of each plug-in, not all versions. 

Similarly if you try helios, using epp.package.rcp you will only get the latest. This means you can't build a site that contains rcp 3.6.0 3.6.1 and 3.6.2.
Comment 4 Thomas Hallgren CLA 2011-03-08 11:39:40 EST
The objective with the aggregator so far has been to produce a p2 site that is consistent and verified. I.e. everything in the site can be installed together. That scenario rules out multiple versions of most plug-ins since they often are singletons.

What you ask for is raw unvalidated mirroring. We could of course add that too, question is how to mix that with the verified mirroring. It doesn't really make much sense.

Other nice features like contributions with only a few features selected wouldn't be possible either, since in order to figure out the transitive scope from those features, we use the p2 planner.

We could of course provide 'raw' mirroring as a concept. All features would then be turned off and the only thing all contributions could contain would be a repository.

Suggestions on how this would manifest itself in the aggregator UI are very welcome.
Comment 5 Matt Biggs CLA 2011-03-08 12:34:54 EST
Currently if i add a product and then try to add another, it filters the names of the ones i have already created out. Would it be possible in the UI to do the filtering but based on available versions?

So for example i could add epp.package.rcp with a version of 1.0 and then another of the same product with a version of 1.1? b3 would then aggregate both those together? Or is that not possible?

The real amazing thing about the aggregator is the flexibility it provides to pick and choose parts from p2 sites etc. 

Previously we mirrored all of helios and orbit which leads to gb's worth of plug-ins of which we only really wanted the rcp aspect plus a few other bits and pieces such as EMF and SWTBot. 

So adding raw mirroring whilst loosing all of that functionality would be a real shame and probably just leave us with where we were before mirroring all of helios.

I think the reason why this has all become confusing for me is down to orbit. I added the orbit site, and the main orbit feature, and expected to get the versions it contained, but instead got alot less. In particular i expected to get org.apache.commons.lang v2.3 and v2.4 and because i didn't my build failed.
Comment 6 Thomas Hallgren CLA 2011-03-08 13:10:17 EST
(In reply to comment #5)
> So for example i could add epp.package.rcp with a version of 1.0 and then
> another of the same product with a version of 1.1? b3 would then aggregate both
> those together? Or is that not possible?
> 
Anything is possible, but the way it's set up now, an aggregation is very similar to an install. We ask p2 to create a plan for what needs to be installed into a profile. We do that several times with different platforms. The sum of all installable units that p2 includes in those plans is eventually what we include in the final result.

One way of resolving your use case is to create several different aggregations and then group them together using a composite. Being able to declare this in the model is something that we've been discussing as a possible add-on for the aggregator.  In order to make that really useful, I think that all aggregations should share the same artifact repository and thus eliminate all duplicates.
Comment 7 Henrik Lindberg CLA 2011-03-08 20:05:36 EST
(In reply to comment #6)
> (In reply to comment #5)
> One way of resolving your use case is to create several different aggregations
> and then group them together using a composite. Being able to declare this in
> the model is something that we've been discussing as a possible add-on for the
> aggregator.  In order to make that really useful, I think that all aggregations
> should share the same artifact repository and thus eliminate all duplicates.

That sounds very useful.
Comment 8 Matt Biggs CLA 2011-03-09 02:33:29 EST
I agree it does sound very useful. The compositing idea seems like it would solve installing multiple versions of a product etc, and i appologise that i keep banging on about Orbit, but how would it handle the Orbit feature?

Currently the Orbit feature has the following in it, but it seems to ignore the fact there's 3 versions and just picks 1. That kind of implies that it hasn't actually installed the feature in its entirety, but only part of it, and within b3 by selecting the feature you have no control over which versions of the plug-ins it does choose.

   <plugin
         id="org.apache.commons.lang"
         download-size="0"
         install-size="0"
         version="2.1.0.v201005080500"
         unpack="false"/>

   <plugin
         id="org.apache.commons.lang"
         download-size="0"
         install-size="0"
         version="2.3.0.v201005080501"
         unpack="false"/>

   <plugin
         id="org.apache.commons.lang"
         download-size="0"
         install-size="0"
         version="2.4.0.v201005080502"
         unpack="false"/>
Comment 9 Thomas Hallgren CLA 2011-03-09 02:45:41 EST
The orbit feature seems to describe something that, when expressed as a p2 installable unit, is impossible to actually install. We cannot really pass it to the p2 planner and expect it to succeed.

But is there a need for this feature? Isn't it enough that all products that you aggregate also implicitly map all bundles that they need from the Orbit repository?
Comment 10 Matt Biggs CLA 2011-03-09 02:55:47 EST
> Isn't it enough that all products that
> you aggregate also implicitly map all bundles that they need from the Orbit
> repository?

I am currently building a p2 site though that contains RCP etc, and then a few extra bundles from Orbit as its my companies product that then needs these to build/compile. Our product itself isn't in the repository, it consumes it. 

We point buckminster at the aggregated repository to build our app.


Or am i misunderstanding and you are saying i should add each specific bundle that i need and its version into the aggregated repository instead of specifying a feature? At the moment this is what i am trying as we speak. 

I guess it was merely a lazy thing on my behalf to want all of the orbit feature so that if someone adds a new dependency within our app they can without having to re-aggregate the repository.
Comment 11 Thomas Hallgren CLA 2011-03-09 03:15:04 EST
(In reply to comment #10)
> Or am i misunderstanding and you are saying i should add each specific bundle
> that i need and its version into the aggregated repository instead of
> specifying a feature? At the moment this is what i am trying as we speak. 
> 
At present, I suspect you will run into the same limitation as you do when using the feature. At least if the bundle is a singleton. The p2 planner will then refuse to install more than one version of it. If you create composites the way I outlined in comment #6, then adding the bundles as you suggest would be the way to go.
Comment 12 Bouchet Stéphane CLA 2011-03-09 03:31:32 EST
(In reply to comment #7)
> (In reply to comment #6)
> > (In reply to comment #5)
> > One way of resolving your use case is to create several different aggregations
> > and then group them together using a composite. Being able to declare this in
> > the model is something that we've been discussing as a possible add-on for the
> > aggregator.  In order to make that really useful, I think that all aggregations
> > should share the same artifact repository and thus eliminate all duplicates.
> 
> That sounds very useful.

Hi,

yes, sounds good ! 

there is a use case i would like to achieve using b3 : 
a iwant to use the same model to mirror ALL artifacts from helios repository and eclipse updates, and using your idea, be able to "slice" them into a specific places, for example :
repo
   \-  helios-3.6.0-all
    -  helios-3.6.1-all
    -  helios-3.6.2-all

each subdir will contains only the specified versions, repo will contain a composite and i will only maintain 1 file.
Comment 13 Kenn Hussey CLA 2011-03-10 15:06:07 EST
(In reply to comment #7)
> (In reply to comment #6)
> > (In reply to comment #5)
> > One way of resolving your use case is to create several different aggregations
> > and then group them together using a composite. Being able to declare this in
> > the model is something that we've been discussing as a possible add-on for the
> > aggregator.  In order to make that really useful, I think that all aggregations
> > should share the same artifact repository and thus eliminate all duplicates.
> 
> That sounds very useful.

See bug 338738.
Comment 14 Thomas Hallgren CLA 2011-07-08 10:26:05 EDT
Closing this as a duplicate, now that ValidationSets has been added.

*** This bug has been marked as a duplicate of bug 338738 ***
Comment 15 David Williams CLA 2016-09-16 15:52:04 EDT
[Bookkeeping change only. Moving bugs to the new "home" of aggregator, CBI.
Made no changes to assignee's for closed bugs, even though some were old inbox.]