Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 330442 - Aggregating Helios doesn't generate entire repo
Summary: Aggregating Helios doesn't generate entire repo
Status: CLOSED DUPLICATE of bug 303178
Alias: None
Product: CBI
Classification: Technology
Component: CBI p2 Repository Aggregator (show other bugs)
Version: unspecified   Edit
Hardware: All All
: P3 enhancement (vote)
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-11-17 07:36 EST by Paul Webster CLA
Modified: 2016-09-16 15:52 EDT (History)
2 users (show)

See Also:


Attachments
aggregator file (864 bytes, text/plain)
2010-11-17 07:36 EST, Paul Webster CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Paul Webster CLA 2010-11-17 07:36:20 EST
Created attachment 183288 [details]
aggregator file

With the discussions on the other bug, I thought I'd use the b3 aggregator and generate a maven repo from Helios.

When I generated the repo, I only get one version of a plugin, like org.eclipse.ui.workbench-3.6.1.M20100826-1330.jar

The origin helios repo contains both that and org.eclipse.ui.workbench_3.6.0.I20100603-1100.jar

Expected:  If I mirror the entire Helios repo, I should get all of the plugins.

PW
Comment 1 Thomas Hallgren CLA 2010-11-17 07:43:42 EST
(In reply to comment #0)
> Expected:  If I mirror the entire Helios repo, I should get all of the plugins.
> 

That's actually a false expectation. The aggregator uses the p2 planner to create a resolution that is 100% coherent. By doing that, it ensures that everything in the result is always deployable together. As a side effect you will never see two versions of a singleton in the aggregated result.

We are well aware that there are many cases when your expected outcome would be desirable. In order to achieve that, you must create several aggregations that are coherent individually and then combine them using a composite. Or you could simply use the p2 mirroring tool. It will mirror everything and not perform any consistency checks whatsoever.

I'm leaving this bug open because we might want to control this using some settings in the model that in essence would turn the aggregator into a more conventional mirroring tool.
Comment 2 Paul Webster CLA 2010-11-17 07:57:48 EST
So using the aggregation is not like a mirror operation, but an install?

If that's the way it works, that's the way it works.

So that leaves me with 2 issues to overcome:

1) How to I aggregate what's in Helios, given that I have a composite repository?  I can't just point at http://download.eclipse.org/releases/helios (although in my case I'm working from a local copy, so I can in theory walk the directory structure, I couldn't do that in the public case).  Can the editor tell me that the Helios repo actually contains 2 other repos (or 3, whatever)?

2) I'm using this to turn helios into a maven repo.  How do I turn the 2 separate repos into one maven repo, just have them both aggregate to the same build location?

PW
Comment 3 Thomas Hallgren CLA 2010-11-17 08:08:43 EST
(In reply to comment #2)
> So using the aggregation is not like a mirror operation, but an install?
> 
> If that's the way it works, that's the way it works.
> 
> So that leaves me with 2 issues to overcome:
> 
> 1) How to I aggregate what's in Helios, given that I have a composite
> repository?  I can't just point at http://download.eclipse.org/releases/helios
> (although in my case I'm working from a local copy, so I can in theory walk the
> directory structure, I couldn't do that in the public case).  Can the editor
> tell me that the Helios repo actually contains 2 other repos (or 3, whatever)?
> 
When using p2 API, the fact that a repository is a composite is hidden from the consumer of that repository. We just give p2 a URL and then we take it from there. If you have explicit knowledge that you are looking at a composite, then you can probably also figure out the URL's of its children (which in turn may or may not be composites) and point to them instead.

> 2) I'm using this to turn helios into a maven repo.  How do I turn the 2
> separate repos into one maven repo, just have them both aggregate to the same
> build location?
> 
That certainly seems like a reasonable approach but to be honest, I'm not sure if that works. It implies that some files are complemented rather then just added and I don't think that's been tested yet. Filip will have to answer that.
Comment 4 Filip Hrbek CLA 2010-11-18 04:22:24 EST
> > 2) I'm using this to turn helios into a maven repo.  How do I turn the 2
> > separate repos into one maven repo, just have them both aggregate to the same
> > build location?
> > 
> That certainly seems like a reasonable approach but to be honest, I'm not sure
> if that works. It implies that some files are complemented rather then just
> added and I don't think that's been tested yet. Filip will have to answer that.

Unfortunately, there is no support for merging a maven output into an existing one. It would require reading the existing output first and merging its maven-metadata.xml files which is something that current aggregator is not doing.

In my opinion it would be more appropriate to support an "unsafe aggregation" in the aggregator itself which would allow 1:1 mirroring. However, that's a different issue.
Comment 5 Paul Webster CLA 2010-11-18 08:04:43 EST
I've opened bug 330559 to talk about creating a large maven repo from multiple releases (maybe merging them?) so this bug can be about future consideration of more mirror-like behaviour.

PW
Comment 6 Thomas Hallgren CLA 2011-07-10 07:49:57 EDT
Mergin discussions of mirror like behavior into one bug.

*** This bug has been marked as a duplicate of bug 303178 ***
Comment 7 David Williams CLA 2016-09-16 15:52:11 EDT
[Bookkeeping change only. Moving bugs to the new "home" of aggregator, CBI.
Made no changes to assignee's for closed bugs, even though some were old inbox.]