Comment 1 Martin Oberhuber 2009-07-16 15:23:34 EDT

As a scrum master, I need to ask why people would want a Maven Repository? I assume that people want to build local products with Maven but pick up Eclipse technology? Are there any specific projects needed or do people need "everything"?

Up to now, some kind people have imported released Eclipse artifacts into some external Maven repositories. Would it be sufficient to have released artifacts only (e.g. Galileo and the Service Releases), or do people really need I-builds as well?

If I-builds (or even N-builds) are needed, immediately the question pops up when such short-lived artifacts may be removed from the Repositories?

Comment 2 Oisin Hurley 2009-07-16 16:45:18 EDT

Requests to consume Eclipse projects via Maven usually come from projects
outside the Eclipse community. It would be a bit of a boost for those projects
to be able to pick up Eclipse artifacts from central, or from a dedicated
Eclipse.org maven repo.

I'm 'collecting' Eclipse projects that use Maven for their builds, from plugins
to update site to RCP to products. There's the Apache Directory explorer (which
won a prize at Eclipse 2009), and there's a new one about to appear soon in
another community. I'd certainly like to see all of these join forces to fix
and finagle the existing Maven plugins to make building Eclipse projects
'easier' for those who have made that build choice. Do you know of anymore
projects that do that kind of thing?

I have a thought that it might be feasible to take a p2 repo and transform it
into a maven-standard-layout repo, possible mapping Eclipse plug-ins to leaf
components and Eclipse features to parent POMs.

@Martin - people will need to pickup specific projects, I think, rather than 
'everything'. Conventionally, there are two maven repositories - one for 
Release, one for Snapshots. Eclipse R- and S- builds would map to Release
artifacts and Eclipse I- builds would map to Snapshots. N- builds would 
not be pushed to a repo (IMHO).

Comment 3 Chris Aniszczyk 2009-07-16 23:14:04 EDT

(In reply to comment #2)
> I have a thought that it might be feasible to take a p2 repo and transform it
> into a maven-standard-layout repo, possible mapping Eclipse plug-ins to leaf
> components and Eclipse features to parent POMs.

I think that this is probably the best approach. I think we can write a p2 artifact/metadata repository implementation that does the right thing when it lays out files. The other approach is to extend the mirror application to be maven aware... but that isn't optimal imho.

Can you tell me more about the 'maven-standard-layout repo' ?

Comment 4 Oisin Hurley 2009-07-17 03:32:49 EDT

> Can you tell me more about the 'maven-standard-layout repo' ?

Have a butcher's at this link

http://docs.codehaus.org/display/MAVEN/Repository+Layout+-+Final

for the details, and then you could do a little exploration of
the layout in the Central repository at 

http://mirrors.ibiblio.org/pub/mirrors/maven2/

It's a fairly straightforward set of conventions in terms of directory
structure, and we should have enough dependency information in 
the p2 metadata to automatically generate a POM for each artifact.

Comment 5 Stefan Seelmann 2009-08-13 15:19:02 EDT

That's a great idea, It would be great if we could use it for the Apache Directory Studio build. At the moment we maintain our own maven repository.

Comment 6 Scott Lewis 2009-08-13 15:20:38 EDT

From the project perspective I don't see a huge need for a Maven repo for these reasons:

1) We already have p2 repos for bundle distribution
2) Maven repos can be converted to p2 repos
3) We have Orbit project/build for access to third-party-created bundles that
creates p2 repos

Of course, I know that other orgs (outside of Eclipse Foundation members) use
Maven repos, but in these days of scarce resources it seems like a waste (IMHO)
to use such resources for converting, building, maintaining, and managing Maven repos rather than other things most projects desperately need (e.g. easier builds,
simultaneous release coordination, new contributions and committers, etc).

$0.02

Comment 7 David Green 2009-08-13 16:18:44 EDT

There has been at least one request for Mylyn WikiText to be put in a Maven repository: http://greensopinion.blogspot.com/2008/11/mylyn-wikitext-stand-alone-package.html#c861930722670101955

This question has come up since Mylyn WikiText was moved to eclipse.org, and before the move a couple of times when Mylyn WikiText was known as Textile-J.

Comment 8 Mickael Istria 2009-08-13 17:05:25 EDT

This is for sure an interesting thing to have a place where projects could store some Maven artifacts.
Indeed, not all pieces of a project are intended to be delivered as OSGi bundles provisionned with p2. There are some pieces that must be usable when developping any kind of application, and Maven is a very common and comfortable tool for everything that is not OSGi.
For example, JWT or STP/SCA publish some artifacts on other repositories, such as OW2. It would make more sense if they could be hosted on Eclipse.org.
Another example of project that could benefit of a Maven repository is SWT (see bug 199302 and http://dev.eclipse.org/newslists/news.eclipse.platform.swt/msg45288.html).

In case such a repository appears, JWT would use it.

Comment 9 Scott Lewis 2009-08-13 17:17:35 EDT

(In reply to comment #8)
> This is for sure an interesting thing to have a place where projects could
> store some Maven artifacts.
> Indeed, not all pieces of a project are intended to be delivered as OSGi
> bundles provisionned with p2. 

I would question this.  Why shouldn't an Eclipse Foundation project be expected to deliver itself as bundles?  That's the runtime form (i.e. OSGI in general and Equinox specifically) that is shared with all Eclipse Foundation projects.

Sure...I understand it's possible that some projects want to/do also deliver part of their value as non-bundle jars (my project has had some requests of this nature)...but should this be the concern of the EF? (in terms of expending resources...like admin, build/releng, etc) *over* other needed things ...because when we're talking about shared resources...like EF admin, people, etc. we are necessarily prioritizing some things over others.  

Comment 10 David Green 2009-08-13 17:25:06 EDT

(In reply to comment #9)
> I would question this.  Why shouldn't an Eclipse Foundation project be expected
> to deliver itself as bundles?  That's the runtime form (i.e. OSGI in general
> and Equinox specifically) that is shared with all Eclipse Foundation projects.
> 
> Sure...I understand it's possible that some projects want to/do also deliver
> part of their value as non-bundle jars 

Mylyn WikiText core (non-ui) jars are both OSGi bundles and 'regular' jars that can be used on a classpath without OSGi.  I believe that the same can be said of many other Mylyn bundles, such as core connector jars.

Comment 11 Scott Lewis 2009-08-13 17:46:34 EDT

(In reply to comment #10)
<stuff deleted>
> > Sure...I understand it's possible that some projects want to/do also deliver
> > part of their value as non-bundle jars 
> 
> Mylyn WikiText core (non-ui) jars are both OSGi bundles and 'regular' jars that
> can be used on a classpath without OSGi.  I believe that the same can be said
> of many other Mylyn bundles, such as core connector jars.

I accept this is true, but how does it relate to my question

>Why shouldn't an Eclipse Foundation project be expected
> > to deliver itself as bundles?

Comment 12 Antoine Toulmé 2009-08-13 18:47:06 EDT

As a reaction to Wayne's post:
1. I would definitely use this.
2. I am working right now on a script to build the BPMN modeler with buildr, which uses a Maven repository to store artifacts. I will comment more on this bug when I have something concrete to share.

Comment 13 David Green 2009-08-13 19:33:17 EDT

(In reply to comment #11)
> I accept this is true, but how does it relate to my question
> 
> >Why shouldn't an Eclipse Foundation project be expected
> > > to deliver itself as bundles?
> 

It relates to the following:

> this nature)...but should this be the concern of the EF? (in terms of expending
> resources...like admin, build/releng, etc) *over* other needed things
> ...because when we're talking about shared resources...like EF admin, people,
> etc. we are necessarily prioritizing some things over others.

Correct me if I'm wrong: as I understand it there's nothing in the EF's mandate that should cause it to prioritize OSGi bundles over anything else.

Comment 14 Chris Aniszczyk 2009-08-13 19:40:11 EDT

This is how I view things happening. If there is demand in the community for this functionality to happen, hopefully someone steps up. Here's two potential ideas:

1) add functionality to the p2 mirroring utility to mirror repos as maven repos

2) extend the p2 Simple{Metadata,Artifact}Repository implementation to optionally write out Maven metadata so p2 repos double as maven repos. I think the p2 team can advise the best way to do this with the p2 APIs.

I would love to see the M2E/IAM projects potentially getting involved in this effort.

Comment 15 Scott Lewis 2009-08-13 20:05:55 EDT

(In reply to comment #13)
<stuff deleted>
> 
> Correct me if I'm wrong: as I understand it there's nothing in the EF's mandate
> that should cause it to prioritize OSGi bundles over anything else.

I just want to say before I respond to this...it's not my intention or desire
to argue about what defines EF's mandate (as there are admittedly lots of
different views about what EF's mandate is), but it does say in the bylaws [1]
that 

"...The purpose of Eclipse Foundation Inc.,
(the “Eclipse Foundation”), is to advance the creation, evolution, promotion,
and support of the
Eclipse Platform and to cultivate both an open source community and an
ecosystem of
complementary products, capabilities, and services."

Which I would interpret to mean that EF's mandate (and by extension the EF
projects) is to advance and promote the Eclipse Platform.

And if one assumes that 'Eclipse Platform' refers in part to the runtime
technologies (i.e. OSGi/Equinox) which are part of/underly the Eclipse
Platform, then that does imply (to me) some preference for EF projects focusing on running on OSGi/Equinox.

Incidently and FWIW, I agree Chris' statement and suggestions in comment 14.

[1]
http://www.eclipse.org/org/documents/Eclipse%20BYLAWS%202008_07_24%20Final.pdf

Comment 16 Brian Fox 2009-08-13 21:03:25 EDT

I was actually talking to Jesse from Webtide yesterday about this very subject so I could better understand the internal process and how it affects their build. We'll be going through this soon as part of the M2e CQ process.

In the Nexus repository manager, we currently have support to convert repository formats and support Maven and P2 repositories. I think the only part missing is an adapter for any special Orbit layout and then that data could directly be exposed as P2 and Maven2 compliant repositories while maintaining the current Orbit structure underneath and not interfering with the current process.

Since many if not most of the artifacts in Orbit don't have poms, part of this adapter would be to construct a pom on the fly, most likely using the OSGI metadata inside a jar to determine dependencies.

We would also need to decide how to organize the artifacts here that originate outside of Eclipse. I understand that part of the CQ process essentially "Eclipsifies" the jars produced by third parties. These jars should be hosted under an org.eclipse groupId in the Maven layout. However, we would want to be cognizant of introducing potential classpath conflicts if someone happens to also transitively depend on the original un-Eclipsified artifact. This may actually require some enhancements to Maven3 to fully support this...think of some kind of alias information in the pom to be used for conflict resolution.

Anyway, we at Sonatype are definitely interested in helping to further this effort to more easily allow bidirectional consumption of artifacts inside and outside eclipse.org.

Comment 17 Chris Aniszczyk 2009-08-13 21:11:22 EDT

(In reply to comment #16)
> In the Nexus repository manager, we currently have support to convert
> repository formats and support Maven and P2 repositories. I think the only part
> missing is an adapter for any special Orbit layout and then that data could
> directly be exposed as P2 and Maven2 compliant repositories while maintaining
> the current Orbit structure underneath and not interfering with the current
> process.
> 
> Since many if not most of the artifacts in Orbit don't have poms, part of this
> adapter would be to construct a pom on the fly, most likely using the OSGI
> metadata inside a jar to determine dependencies.

Back to my comment #14, this seems the best way to do it. Do you have your own repository implementation in p2? I don't think we necessarily need to convert repositories, I think we can have a p2 repository that doubles as a Maven repo (with my limited understanding of Maven).

> Anyway, we at Sonatype are definitely interested in helping to further this
> effort to more easily allow bidirectional consumption of artifacts inside and
> outside eclipse.org.

That's great to hear :)!

Comment 18 Brian Fox 2009-08-13 21:20:20 EDT

"Back to my comment #14, this seems the best way to do it. Do you have your own
repository implementation in p2?"

No, we use the eclipse implementation to generate the p2 metadata.

"I don't think we necessarily need to convert
repositories, I think we can have a p2 repository that doubles as a Maven repo
(with my limited understanding of Maven)"

Oh sorry, I didn't mean a full scale static conversion, I mean essentially on the fly mapping paths. The pom.xml generation would likely be done on the fly the first request (or scheduled) and then stored for future requests since it should never change. We currently do a very similar process for M1 to M2 conversion and have been looking at P2 to M2 for a while. M2 to P2 is more complicated since the bundle information would be hard to generate without more information about the code, but it seems like P2 to M2 is more readily possible.

Comment 19 Chris Aniszczyk 2009-08-13 21:30:38 EDT

(In reply to comment #18)
> "Back to my comment #14, this seems the best way to do it. Do you have your own
> repository implementation in p2?"
> 
> No, we use the eclipse implementation to generate the p2 metadata.
> 
> "I don't think we necessarily need to convert
> repositories, I think we can have a p2 repository that doubles as a Maven repo
> (with my limited understanding of Maven)"
> 
> Oh sorry, I didn't mean a full scale static conversion, I mean essentially on
> the fly mapping paths. The pom.xml generation would likely be done on the fly
> the first request (or scheduled) and then stored for future requests since it
> should never change. We currently do a very similar process for M1 to M2
> conversion and have been looking at P2 to M2 for a while. M2 to P2 is more
> complicated since the bundle information would be hard to generate without more
> information about the code, but it seems like P2 to M2 is more readily
> possible.

Yes, my initial observation was that M2 to P2 is much more complicated. When I looked at this problem brief and how I would attack it in the p2 code, we could actually modify the simple repository implementation (or extend it) to lay down artifacts in a Maven-esque fashion along with the proper POMs. However, like I mentioned earlier, the p2 team would give us the best advice on this.

Comment 20 Patrick Roumanoff 2009-08-13 23:20:34 EDT

I manage the build process for our internal RCP applications and I would definitely use a maven repository for eclipse artifacts (since I am already using the maven repository provided by spring source to source my third party bundles @ http://www.springsource.com/repository/).

But I think that we are talking about here is how a maven build can consume those artifacts (since that where the demand is).

Since maven is itself plug-in based, wouldn't it be possible to define P2 as a layout for repositories in maven which would generate the POM on the fly (following comment #18). If it is possible, this option would allow maven based build to consume P2 artifact with no work from the P2 repository provider.

That is probably a question for a maven guy...

Comment 21 David Samuelsson 2009-08-14 01:17:42 EDT

I would use it, especially if you also used the mechanism to publish the source with it which you can with Maven.

As we are currently replacing our build system with Maven and i pull source directly from cvs sometimes as well this would be a good enhancement.

Comment 22 Karsten Thoms 2009-08-14 05:13:48 EDT

I would really appreciate having a central Eclipse Maven Repository. At the moment I maintain Eclipse artifacts needed during builds in a handcrafted repository, which is error prone, high effort and sometimes late for users. I know some projects using EMF, EMFT, M2T, TMF and all that it relies on in Maven based builds.

I read that Maven 3.0 will have some support for OSGi and P2 repositories, but at the moment there's not much news out there about it. Also Maven Tycho can use an Eclipse Target Platform for building, but I prefer plain Maven2 based builds.

Comment 23 Mickael Istria 2009-08-14 07:33:26 EDT

I agree that the best thing would be to have only one repository that could be consumed by Maven and p2. This would be a perfect solution...
However, I don't think this is a very easy thing to do (but I don't know enough of Maven or p2 from the inside to be sure). Having a Maven repository would be an intermediary working and easy solution until the new p2/Maven repository gets ready.

Comment 24 Aaron Digulla 2009-08-14 11:08:01 EDT

FYI: IIRC, Maven 3.0 (due next year) will be able to use OSGi bundles, so they are already working on a way to migrate Eclipse bundles and Maven artifacts back and forth between the two structures.

Comment 25 Micah Whitacre 2009-08-18 10:55:35 EDT

Has anyone talked to the Maven/Sonatype guys about doing this?  Specifically they sell a product[1] that from my understanding will make a P2 repository available for Maven builds.  Just wondering if this is an opportunity to not reinvent the wheel.

[1] - http://www.sonatype.com/people/2009/04/nexus-pro-support-for-eclipse-p2-repositories/

Comment 26 Brian Fox 2009-08-18 11:08:15 EDT

Hi Micah, see comment #16: https://bugs.eclipse.org/bugs/show_bug.cgi?id=283745#c16

;-)

Comment 27 Sebastian Jancke 2009-09-03 04:20:22 EDT

This is definitely needed. Currently others (outside eclipse) host lot’s of eclipse artifacts under the groupId org.eclipse. Worse: these artifacts are broken, inconsistent and all different - sometimes without any stated dependencies. This is something that seems to violate the assumptions and rules of maven central repository.

Eclipse should move into this area and it should be at least part of the release train. One repository for all eclipse projects seems a good option. However you will need volunteers inside eclipse foundation (and maybe Eclipse IAM project?!) to maintain this repository. Building eclipse itself with maven is another topic, but publishing eclipse bundles is possible with “mvn eclipse:to-maven” and some extra manual checks.

Promoting nightly / integration builds - this is called “SNAPSHOT” in maven - makes only sense when eclipse is build with maven. 

Please remember: a public repository has to guarantee a high degree of consistency and correctness. Currently this seems only be maintained for maven central and a few others. 

This “rule” - the foundation of maven - might be softended for a (separate) snapshots-repository. However the amount of artifacts (per day!) leaves you to totally trust “mvn eclipse:to-maven” for this. Perhaps invastigating it’s correctness should be done by eclipse foundation.

In closing: having maintained, correct eclipse artifacts in an official eclipse maven repository is needed and was far too long not discussed, plannend, an option.

Comment 28 Sebastian Jancke 2009-09-03 04:28:17 EDT

Part of the eclipse platform is the eclipse modelling tools project. Huge parts of it can be used with maven (fornax platform!) and have to be used outside eclipse - otherwise it is not possible to integrated eclipse modelling (former OpenArchitectureWare, MWE, EMF, ...) into a standard build process that is used in continuous integration. Eclipse headless is not an option here - it is a pain. 

There is without doubt demand an reason for people working with eclipse (as an ide) to have parts of it "outside" OSGi. These parts are cabable of it. The only thing missing is a real eclipse maven repository.

Comment 29 Martin Oberhuber 2009-09-04 09:45:57 EDT

(In reply to comment #15)

The Eclipse Architecture Council briefly discussed the idea of Maven repos at Eclipse in its Aug 13 meeting: 
  http://wiki.eclipse.org/Architecture_Council/Meetings/August_13_2009
and was mostly supportive of the idea.

Fact is, that many projects outside the inner "Eclipse Universe" use Maven for their builds, and cannot easily consume p2 repositories. Thus serving Eclipse artifacts from a Maven Repository does seem like a way to ease adoption of Eclipse technology and thus "advance...the evolution...of the Eclipse Platform" in alignment with the Bylaws.

Scott, your main concerns seem to be about using Eclipse Foundation Resources for preparing / converting / serving a Maven repository, and not so much about the technology ... is that correct?

In our Aug 13 meeting we thought that the next steps for advancing towards a Maven Repository at Eclipse would be to gather community and expertise around the idea, such that not all the burden of work would rest on Eclipse Foundation employees... does that mitigate your concerns? We also thought that as a first step, we would only serve the Orbit repository in a Maven format to gather experience.

Up to now, we have 
 * lots of "We want this" comments on this bug, 
 * some "We could do...with p2" ideas from Chris (including mirroring), and
 * some "We could do...with Nexus" from Brian (generate pom.xml on the fly).
 * anything I am missing?

Many thanks for all comments so far! With 32 people on CC and 6 votes, we do seem to have a good deal of Community behind this, although I would still like to see Scott's concerns addressed.

What could be the next steps?
Who could step up actually implementing something?
What do people perceive as blocking progress on this?

Comment 30 Antoine Toulmé 2009-09-04 10:08:08 EDT

Martin, I just filed 288644 (Decide of group ids for Eclipse Maven artifacts). 

I think resolving that bug would be a good step forward to fix this problem.

I think the Eclipse Foundation should decide to host artifacts by explicitely setting a list of criteria.

Comment 31 Scott Lewis 2009-09-04 10:32:30 EDT

(In reply to comment #29)
<stuff deleted>
> 
> Scott, your main concerns seem to be about using Eclipse Foundation Resources
> for preparing / converting / serving a Maven repository, and not so much about
> the technology ... is that correct?


Eclipse Foundation resources...primarily...yes, although if the creation of a Maven repository was added as a requirement to the release train the resource problem created would be much worse...i.e. because it would also burden all the simultaneous release projects.


> 
> In our Aug 13 meeting we thought that the next steps for advancing towards a
> Maven Repository at Eclipse would be to gather community and expertise around
> the idea, such that not all the burden of work would rest on Eclipse Foundation
> employees... does that mitigate your concerns? 


Not particularly.  Reason:  ultimately some amount of maintenance and management of a Maven repository will have to be done by the foundation IT staff.  And the creation of the Maven repositories (i.e. for simultaneous release projects) would likely fall upon the projects themselves (since it's unlikely that the foundation staff can/would take this on for all projects).  This extra burden of creating (and managing) release artifacts is my main concern...both for Foundation staff/resources and for the projects.


> We also thought that as a first
> step, we would only serve the Orbit repository in a Maven format to gather
> experience.


I think this is a good idea, but recognize that Orbit is a little different from other projects (perhaps unrepresentative) in that the content of Orbit bundles is not actually changing very rapidly.  With a non-Orbit project, things will be changing more rapidly (e.g. builds being broken, new things added), and this will mean additional work on the build to be able to create both p2 and Maven repositories under those conditions.

> 
> Up to now, we have 
>  * lots of "We want this" comments on this bug, 
>  * some "We could do...with p2" ideas from Chris (including mirroring), and
>  * some "We could do...with Nexus" from Brian (generate pom.xml on the fly).
>  * anything I am missing?
> 
> Many thanks for all comments so far! With 32 people on CC and 6 votes, we do
> seem to have a good deal of Community behind this, although I would still like
> to see Scott's concerns addressed.

Me too.  For reference a number of other concerns were expressed by Nick Boldt in his blog posting http://divby0.blogspot.com/2009/08/we-dont-need-another-repo.html.  Apologies if this is duplicated somewhere else, but I didn't see these issues raised on this bug.

Comment 32 Antoine Toulmé 2009-09-16 09:46:37 EDT

Guys,

I'd like to publish Eclipse artifacts to a Maven repository. It can either be an Eclipse repository or the Maven central.

I will do it at some point, and I will make sure all the dependencies are correctly figured out.

I will do it for GMF, GEF, EMF, Draw2d, EMF validation, EMF transaction, and the Eclipse platform.

If you are interested, please let me know. If you'd like to have that on the Eclipse Foundation disks, please let me know. Otherwise I'll proceed with other repositories.

Thanks!

Comment 33 Oisin Hurley 2009-09-16 10:19:53 EDT

Hi Antoine, 
Thanks for taking the step forward - it's of interest to my company (although perhaps not in the short term) as it will help unify the discovery process for product pieces. If you have the kudos to push to
central, that would probably be the most effective approach.

@Scott, your concern for the IT staff at the foundation is both touching and praiseworthy. How about we get some of the IT staff to chime in on this thread? Anyone know who would be the guy to add to the cc: list?

@Sebastien, I agree that having versions of Eclipse-produced bundles with screwed-up metadata floating around the mavenverse is bad for the reputation of the projects. However, it's absolutely up to the discretion of the project leadership to wash their hands of the issue and reject the needs of potential consumers, should they wish to do so. 

On the subject of the release train, I agree that creating a maven repo should not be a project deliverable, so as not to add extra burden on the project teams. It should not even be mentioned as part of the release plan, until such time as some individual has become motivated enough to automate it.

I think that the major impact of hosting a maven repo at the Eclipse foundation is a bandwidth one - should the repository become popular, then there will be a considerable amount of pull on the content. The maven repo can of course be mirrored, but (if I understand it correctly) the onus is on the consumer to apply the mirror details in their settings.xml -- there is no 'server' which will automatically redirect. Perhaps third-party innovations like Nexus could intervene in a positive way to help spread the load. The downside of having a server like Nexus in place is that there is a requirement to have individuals educated and empowered to manage the server, maintain it, etc.

I think that the way to make headway here is to have some trailbreaking individual(s) go ahead and look into what can be done without involving Eclipse IT folk and without running servers on the Eclipse infrastructure - i.e. exactly what Antoine is intending to do.

Comment 34 Scott Lewis 2009-09-16 10:47:39 EDT

(In reply to comment #33)
<stuff deleted>
> 
> @Scott, your concern for the IT staff at the foundation is both touching and
> praiseworthy. How about we get some of the IT staff to chime in on this thread?
> Anyone know who would be the guy to add to the cc: list?

As with all shared resources (i.e. Foundation staff), time spent on things such as this is clearly not spent on other things of import to the (entire) community.

One way to gauge the load btw is to search for open bugs in the Eclipse Foundation project.

<stuff deleted>
> 
> On the subject of the release train, I agree that creating a maven repo should
> not be a project deliverable, so as not to add extra burden on the project
> teams. It should not even be mentioned as part of the release plan, until such
> time as some individual has become motivated enough to automate it.

I agree...as the burden on projects for creating new release artifacts would be greater than that for the foundation staff.

Comment 35 Antoine Toulmé 2009-09-16 11:45:51 EDT

Adding Denis to CC.

Comment 36 Denis Roy 2009-09-16 16:00:16 EDT

> Adding Denis to CC.

I'm not sure I understand why...

Comment 37 Antoine Toulmé 2009-09-16 16:19:45 EDT

(In reply to comment #36)
> > Adding Denis to CC.
> 
> I'm not sure I understand why...

Denis, see comment 33, second paragraph.

Comment 38 Denis Roy 2009-09-16 16:39:18 EDT

I'm not sure to what extent my team would be involved in "converting, building, maintaining, and managing Maven repos"...  but if the committers and the community believe it's the best usage of our time and servers, then so be it.

Scott nailed it in the quote below.  If someone sais 'get Maven stuff done now' then that would obviously push back other stuff, like git for instance.

> As with all shared resources (i.e. Foundation staff), time spent on things such
> as this is clearly not spent on other things of import to the (entire)
> community.

Comment 39 Jeff McAffer 2009-09-20 17:33:57 EDT

(In reply to comment #32)

> I will do it for GMF, GEF, EMF, Draw2d, EMF validation, EMF transaction, and
> the Eclipse platform.
> 
> If you are interested, please let me know. If you'd like to have that on the
> Eclipse Foundation disks, please let me know. Otherwise I'll proceed with other
> repositories.

As with Oisin, I applaud the initiative. I assume you will wait for a resolution of bug 288644 (and the as yet unopened bug relating to version number handling) prior to publishing?

BTW, if you have write access to download.eclipse.org for your projects then you are able to put this on foundation servers all on your own.

Comment 40 Antoine Toulmé 2009-09-20 18:00:48 EDT

I think groupIds are mostly figured out ; I will open a bug for version range differences between Maven and Eclipse, with the current workaround being not to translate version ranges when uploading artifacts but rather depend on solid version numbers. That's my personal preference for having reproducible builds.

Comment 41 Jason van Zyl 2009-09-27 12:49:21 EDT

Antoine,

We (Sonatype) has tooling for dealing with conversions of OSGi-based repositories (P2 & OBR) to Maven repositories so I would urge you not rewrite a bunch of tooling. We can probably put some prototype repositories based on agreed upon sources and then folks here can vet the output. We have lots of experience here so we're happy to help.

Comment 42 Antoine Toulmé 2009-09-27 14:48:02 EDT

Allright Jason, anything towards achieving this goal is very welcome.

Comment 43 Sebastian Jancke 2009-10-06 06:52:01 EDT

(In reply to comment #40)
> I think groupIds are mostly figured out ; I will open a bug for version range
> differences between Maven and Eclipse, with the current workaround being not to
> translate version ranges when uploading artifacts but rather depend on solid
> version numbers. That's my personal preference for having reproducible builds.

As far as I know, there is already some conversion and handling of these differences in the Eclipse Buckminster Project.

Comment 44 Nick Boldt 2009-10-09 17:01:29 EDT

See also bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=291940, which would add support for converting a p2 repo zip to a maven repo zip for subsequent publication @ eclipse, apache, or wherever.

Comment 45 Andrew Overholt 2009-10-09 17:35:40 EDT

Before committing ourselves to providing a Maven repository of Eclipse bits, I think we should consider reasons people want this.  One possible question related to this is:  are people asking for this so that they can make use of Eclipse bits in non-OSGi runtimes?  If this is the case, do we care?

I have also seen situations where people tend to peg themselves against a particular version because it is so easy to just keep using it from a Maven repository.  We should make it clear up front how many previous versions we will "support".

Comment 46 Alex Blewitt 2009-10-10 04:52:30 EDT

(In reply to comment #45)
> Before committing ourselves to providing a Maven repository of Eclipse bits, I
> think we should consider reasons people want this.

Why? The requirement is there; why should it matter the reasons?

> One possible question related to this is:  are people asking for this so that they can make use of
> Eclipse bits in non-OSGi runtimes?  If this is the case, do we care?

There are (very) valid reasons for some of this. For example, ECJ is used in Ant tasks to compile Java code outside of an OSGi environment. Libraries like ECF and JGit are designed to be used both inside and outside of an OSGi runtime. Whether it's an OSGi runtime or not should not even be a consideration in this case.

> I have also seen situations where people tend to peg themselves against a
> particular version because it is so easy to just keep using it from a Maven
> repository.  We should make it clear up front how many previous versions we
> will "support".

That's their problem, not Eclipse's. For any open-source project, there is a supported release (and quite possibly a developer/HEAD release). Whether the bits are available or not in any mechanism is almost entirely irrelevant - I can go and download the bits from http://archive.eclipse.org for Eclipse 1.0 - that doesn't mean it's supported.

Comment 47 Paedagogus 2009-10-10 10:54:37 EDT

I can appreciate the desire expressed by some commenters to avoid adding unnecessary complexity to the Eclipse project and development process as a whole. This desire has been voiced repeatedly by many of the people who have commented on this "bug":

> I need to ask why people would want a Maven Repository?
> I don't see a huge need for a Maven repo
> should this be the concern of the EF?
> do we care?

Looking at such comments, I have come to two conclusions:
     - Some people do not understand Maven
       and/or
     - they feel a need to defend their "religion"

Like OSGi, reducing complexity is PRECISELY the reason that Maven exists. If you don't understand the benefits of having Eclipse dependencies available from a Maven repository, you cannot possibly understand the benefits of Maven. In that case, I suggest that you take the time to become very familiar with it. If you still do not understand the benefits, then it is likely that you do not work on projects that have the size and complexity that Maven was created for.

There is also a bit of a religious objection in the tone of these comments and questions, but the reality is, this is not about OSGi, repackaging any jars, Eclipse vs Apache, etc. This is about providing Eclipse binaries to a broader, more sophisticated audience, and thereby increasing the adoption of Eclipse projects and components as a whole. If you feel your OSGi, Eclipse or other religion being threatened, I simply ask that you curb your dogma, and continue to seek further enlightenment.

Kumbaya.

Comment 48 Lom Hillah 2009-10-12 13:05:13 EDT

Hi all,

Thanks for having this issue at the Eclipse level, because we are right now in middle of it in our lab. Lots have been said about whether or not it is worth setting up, so I am just going to report our experience. 

We use a Sonatype Nexus Maven repository manager (Open source version). Two of our projects heavily depends on GEF, Draw 2D and EMF. When building these projects using Maven, we have noticed that very often the public repositories out there are outdated, or the dependency trees are broken, concerning Eclipse plugins. So we have uploaded a complete Eclipse modeling distribution onto our repository manager. But that is not a sustainable way of doing things. We also happen to face broken dependency trees because of the updates and backwards compatibility issue (e.g., dependency tree has changed).

Although it won't be a trivial issue to resolve for Eclipse and all its projects,
it would be very interesting to have such repository we could mirror. You may think of it as a cluster of repositories. Guys at Sonatype certainly have a lot of experience to share. 

Cheers,
Lom

Comment 49 Wayne Beaton 2009-10-13 13:26:05 EDT

I'd like to try and steer this conversation toward requirements.

Here's a start:

1) Repositories are hosted at eclipse.org
2) Projects must opt-in to be included in the repository
3) Burden on those projects choosing to opt-in must be minimal

Unfortunately, I don't know all that much about Maven (it's on my list, honest), but it seems to me that there may be more than one way to distribute code via Maven. For some projects, bundles is the only way to deliver. For others, there may be non-bundle alternatives. I'd like there to be an easy way for projects to get OSGi bundles into an "Eclipse Uber Repository", but it may also be valuable to allow projects that want to do some extra work to include content in other forms.

Comment 50 Alex Blewitt 2009-10-13 13:55:37 EDT

(In reply to comment #49)
> Unfortunately, I don't know all that much about Maven (it's on my list,
> honest), but it seems to me that there may be more than one way to distribute
> code via Maven. 

There isn't really more than one way - a maven repository is just a collection of things. They could be JARs, and most are, but there's also WARs etc. The components should really be the OSGi bundles to allow those using Maven to build both Java-based and OSGi-based systems (e.g. using PAX Runner/PAX Construct). Myself, all my OSGi builds run on Maven instead of PDE for practical reasons, so we definitely don't need to strip out any OSGi stuff.

Furthermore, a Maven repository is much easier than a P2 repo, because there's no master 'list'. Instead, dependencies and location are done on a per-name basis and it's just a location on disk.

http://maven.apache.org/guides/introduction/introduction-to-repositories.html

For example, here's an (old) JDT at Maven:

http://www.ibiblio.org/maven/org.eclipse.jdt/
http://www.ibiblio.org/maven/org.eclipse.jdt/jars/core-3.2.0.653.jar  
http://www.ibiblio.org/maven/org.eclipse.jdt/poms/core-3.2.0.653.pom

That's it. Deploy that on a webserver, tell them it's the 'org.eclipse.jdt' groupId and 'core' artifact, and that they'll need 3.2.0.653, and anyone can import it. There's a separate process for putting it up on ibiblio; but if you want to host these at (say) maven.eclipse.org, then that shouldn't be a concern. 

The only trick is the creation of the .pom (the .jar will fall out of the bottom of PDE's build). The POM is an XML file which contains the group/version and dependencies. Others with m2eclipse experience or tycho may have some thoughts on the matter.

Maven can have version range dependencies, but does it by name (ala Require-Bundle). Mapping those components which have dependencies exposed by Import-Package will require some kind of (arbitrary) dependency list, but isn't insurmountable.

You should look at pax-construct, and create a simple bundle using that; it'll show some of the poms 

http://www.osgilook.com/2009/04/27/pax-construct-from-zero-to-osgi/

I can help out if there is a candidate project which is interested in a Maven representation, or reach out off-line if you have questions which will be best served either via IM or Twitter.

Comment 51 David Carver 2009-10-13 14:59:41 EDT

(In reply to comment #50)
> 
> I can help out if there is a candidate project which is interested in a Maven
> representation, or reach out off-line if you have questions which will be best
> served either via IM or Twitter.

You may want to touch base with the jetty runtime project.  I know they currently use maven pretty heavily, but are interested in both osgi bundles and maven repos as well.

I believe they are monitoring this bug as well.

Comment 52 David Carver 2009-10-13 15:09:29 EDT

(In reply to comment #9)
> (In reply to comment #8)
> Sure...I understand it's possible that some projects want to/do also deliver
> part of their value as non-bundle jars (my project has had some requests of
> this nature)...but should this be the concern of the EF? (in terms of expending
> resources...like admin, build/releng, etc) *over* other needed things
> ...because when we're talking about shared resources...like EF admin, people,
> etc. we are necessarily prioritizing some things over others.

Scott sorry for following this thread late, but let me say from my point of view, the responsibility of the foundation is to provide ways for the user community to consume the bundles that are produced.   OSGI dependency management and P2 is only one way, and in many cases is not the ideal way.  There is difficulty in a project consuming a P2 ZIP update, if they aren't using or installing P2.

An example like David Green's Mylyn WikiText is the PsychoPath XPath 2.0 processor that is currently avaiable as a P2 Repo, does my adopters like Xerces-J no good, if they can't get to the necessary JAR.  The JAR itself is packaged as an osgi bundle, but it has no dependencies on any OSGI runtimes or eclipse related components.  It is designed to be used outside of eclipse.   Having this available in a Maven Repo with the necessary dependencies specified, can only be a good thing for this project, as it allows a wider user community beyond just the P2/OSGI/Eclipse community to consume it.

To me the foundations role is about providing the necessary choice for as wide adoption as possible for the artificats produced by the project.   P2 repos are just too narrow of a view.

Yes these take resources to create, but I think it's worth the extra effort by the community to undertake.

Anyways sorry for the late two cents.

Comment 53 Scott Lewis 2009-10-13 16:00:02 EDT

Hi Dave,

(In reply to comment #52)
> (In reply to comment #9)
> > (In reply to comment #8)
> > Sure...I understand it's possible that some projects want to/do also deliver
> > part of their value as non-bundle jars (my project has had some requests of
> > this nature)...but should this be the concern of the EF? (in terms of expending
> > resources...like admin, build/releng, etc) *over* other needed things
> > ...because when we're talking about shared resources...like EF admin, people,
> > etc. we are necessarily prioritizing some things over others.
> 
> Scott sorry for following this thread late, but let me say from my point of
> view, the responsibility of the foundation is to provide ways for the user
> community to consume the bundles that are produced.   

Although I agree that it's the general responsibility of the foundation to provide ways for the user community to consume bundles produced by projects, I disagree that it's their (I mean our :) responsibility to do this in all possible ways...or even all ways that some in the community might desire.  

This is simply too much to ask.  For example, consider all that's gone into the work on the simultaneous releases over the past few years...e.g. all the work of the project leads and teams, the planning council, and the foundation employees and ip team to make the simultaneous releases occur...on time and with high quality. 

As a participant and project lead, it's been a HUGE commitment, and asking to do additional work to support another distribution/build/repo technology is just too high a burden IMHO (not to mention the burden on the IT folks...i.e. see comment 38).  

In response to my burden argument, the usual response has been that the burden on projects of producing Maven artifacts should/will be 'minimal'.  But this does not at all jive with reality IMHO.  The reality of build/deploy/releng has been and is that *no matter what the build/repo format or tech* the work in creating and then maintaining/supporting the distribution of a significant set of jars/bundles (and their associated meta-data) is significant.  And if that work *is* significant, who is going to do it?  The IT staff at EF? (see comment 38).  The project leads/project team members (through extra/new requirements), or someone else?  If someone else, who?

>OSGI dependency
> management and P2 is only one way, and in many cases is not the ideal way. 
> There is difficulty in a project consuming a P2 ZIP update, if they aren't
> using or installing P2.

Ok, so if they aren't using or installing p2, then perhaps they would like to take on the responsibility (and cost/resources) of building, deploying, and then maintaining a Maven repo with the contents of various projects.

> 
> An example like David Green's Mylyn WikiText is the PsychoPath XPath 2.0
> processor that is currently avaiable as a P2 Repo, does my adopters like
> Xerces-J no good, if they can't get to the necessary JAR.  The JAR itself is
> packaged as an osgi bundle, but it has no dependencies on any OSGI runtimes or
> eclipse related components.  It is designed to be used outside of eclipse.  
> Having this available in a Maven Repo with the necessary dependencies
> specified, can only be a good thing for this project, as it allows a wider user
> community beyond just the P2/OSGI/Eclipse community to consume it.

Then if they desire, they can create/build/deploy with other technologies (i.e. their own, Maven, or whatever).  No one wants to prevent that...and it can be done right now.  

> 
> To me the foundations role is about providing the necessary choice for as wide
> adoption as possible for the artificats produced by the project.   P2 repos are
> just too narrow of a view.

I don't agree.  You could also argue that OSGi (bundles) are too narrow of a view as well, but my counter argument is the same:  it's not *too* narrow, it's rather rightly focused on the build and distribution technologies that this community's artifacts are based on (Equinox/OSGi/RT and p2).

> 
> Yes these take resources to create, but I think it's worth the extra effort by
> the community to undertake.

I see that you think it's worth the extra effort, but you are not the (only) one being asked to commit the extra effort.  And I don't think you would get agreement with this view if a poll were taken among project leads and releng engineers within projects (i.e. realistically the people that will continue to do the bulk of the work of build/deployment).  That would be a poll I would personally like to see:  given a choice of producing Maven repo artifacts or not would you agree that it's worth it for your project?  (among committer project leads and project team members...i.e. the people that do the releng work for the projects...not pmcs, not council members, not prominent bloggers, not EF people, not BOD members, etc).

Comment 54 David Carver 2009-10-13 16:40:49 EDT

(In reply to comment #53)
> 
> I see that you think it's worth the extra effort, but you are not the (only)
> one being asked to commit the extra effort.  And I don't think you would get
> agreement with this view if a poll were taken among project leads and releng
> engineers within projects (i.e. realistically the people that will continue to
> do the bulk of the work of build/deployment).  That would be a poll I would
> personally like to see:  given a choice of producing Maven repo artifacts or
> not would you agree that it's worth it for your project?  (among committer
> project leads and project team members...i.e. the people that do the releng
> work for the projects...not pmcs, not council members, not prominent bloggers,
> not EF people, not BOD members, etc).

I think that people make the build/deployment issue into too large of an issue.  Builds are complicated because we tend to make them complicated.  They don't have to be.  Our design choices and historical decisions tend to complicate the whole issues as well.  Anyways, that is a whole different issue.    Chris has a good idea, in that if P2 has the information and has a Model already, then it should be a matter of adding the necessary plugin to P2 to have it generate POMs instead of P2 Repos.  It's a mapping and translation exercise at that point.  Maybe there needs to be an XSLT or something that converts from the Artifact.xml to POM format, or whatever is the necessary format.

I do think that this would be a good thing for the Eclispe Maven projects to take up and help provide, but only focusing on the P2 way, locks out usage of eclipse components from other ways of building applications.   The eclipse way isn't the only and may not necessarily be the best way.   Providing choice is something we as a community should strive for regardless of our projects.  Sometimes in the eclipse community we think that everything is going to be used or consume by only osgi, however there are many examples, from WikiText, PsychoPath, and even the EMF Runtime that can be used outside of eclipse.  Having items that are consumable from multiple places is a good thing, it gives our adopters choice on how they want to deploy or consume our stuff, not necessarily how we think it should be consumed.

I think we can come up with a way that isn't overly burdsome to the projects, we just have to be open to exploring those ideas and new ways of doing things.

Comment 55 Scott Lewis 2009-10-13 17:08:08 EDT

(In reply to comment #54)
<stuff deleted>
> 
> I think that people make the build/deployment issue into too large of an issue.
>  Builds are complicated because we tend to make them complicated.  They don't
> have to be.  

As much as I agree that they don't have to be complicated, the fact is that they are (at least when you are talking about non-trivial codebases and dependency structures...like all EF projects :).

<stuff deleted>
> I think we can come up with a way that isn't overly burdsome to the projects,
> we just have to be open to exploring those ideas and new ways of doing things.

Well, I won't argue against exploring ideas and new ways of doing things :), but I'm not so certain that it's possible to avoid making it overly burdensome to the projects...along with meeting other constraints.

Comment 56 Wayne Beaton 2009-10-13 17:19:12 EDT

Ultimately, this is about adding value to the community and projects.

If we can't make this dead-easy for a project, my hope is that we make it opt-in. Projects that feel there's value can participate.

If we determine that this is overly burdensome and doesn't add enough value, then we abandon the effort.

I'm curious about what folks think about the inclusion requirements. Specifically, are only release-train projects allowed to participate, or is this separate from the release train?

Comment 57 Thomas Hallgren 2009-10-13 17:21:20 EDT

We have active work going on in this area that we hope will be a good solution for Eclipse. See bug 291986.

In essence, we will provide a way for our Aggregator tool (an elaborated version of the current Galileo Builder) to produce a combined p2 and Maven repository that shares the physical artifacts.

Anyone can use the aggregator to create a combined repository and when we update the Galileo Builder to the latest version, a Helios Maven repository will come at no cost.

Comment 58 David Carver 2009-10-13 20:41:48 EDT

(In reply to comment #56)
> Ultimately, this is about adding value to the community and projects.
> 
> I'm curious about what folks think about the inclusion requirements.
> Specifically, are only release-train projects allowed to participate, or is
> this separate from the release train?

My personal opinion is to make it opt-in.   The reason being that it can be slowly adopted and if it proves useful then it can be migrated as a requirement later.  Gives time for the tooling to be worked on and the kinks ironed out.

I attended the COJUG (Centrak Ohio Java Users Group) meeting tongiht, and the talk was on Java and OSGI.  The tool being used to build OSGI bundles, Maven.

Comment 59 Eric Gwin 2009-10-14 10:44:08 EDT

As Build Engineer for EclipseLink, the notion of allowing the Eclipse P2 repository to double as a Maven repos is very attractive as it would actually make our build process easier. We currently already make available a Maven repository for our builds, and have recently joined the "simultaneous release" process at Eclipse. However, as a runtime piece we build outside of Athena/Buckminster/PDE. So we now generate and make available p2 as well as Maven repositories via Ant.

Some background into our processes:
- We deliver our "bits" for 'traditional' development as install, javadoc, src, and OSGi Bundle zips (accessed via download pages). 
- We publish our bundles to Maven for use by external projects.
- We create 'features' and publish them along with our bundles in P2 for Eclipse integration and Eclipse users. 
- Nightly builds are published on our download page, pushed to our Maven repos, and put in a nightly P2 repos.
- Milestone and release builds are 'promoted' from our nightly builds to the web, Maven and P2 as well.
- We provide our builds to Galileo/Helios via a separate "staging" P2 repository for each.

In essence, we publish to 11 repositories (3 formats * 3 promote states + 2 for Eclipse Integration).

All of this publishing and most of the maintenance is currently automated. However, the ability to publish just once for P2/Maven would be helpful to me, our project's users (ease of delivery), and would actually reduce the physical storage and maintenance time currently dedicated to separate repositories. 

I would happily contribute any way I can to help make this a reality. I am currently in the process of adapting our current Galileo methodologies to our 2.0.0 release for Helios and upgrading our P2 generation to use the new tools. I'll have more concrete feedback on requirements once I've made more progress.

Comment 60 Jesse McConnell 2009-10-14 10:52:23 EDT

the jetty project (lately a part of the eclipse runtime) builds with maven and all of our 'releases' have been published to maven central, including our recent release of jetty 7.0.0.v20091005, into the org.eclipse.jetty space in maven central.

http://repo2.maven.org/maven2/org/eclipse/jetty/

All of our components that we release under the org.eclipse namespace are also osgi bundles, suitable for use in equinox or felix, or 'hopefully' any other osgi containers people are want to use.  

Our current concern is that nothing we produce and publish into maven central can be used from an 'eclipse' perspective.  I do toss together an update site through a manual process involving the renaming of the artifacts we produce and publish into maven central but that just smells like a very dirty solution and I would very much like an automated process that 'just works'.  I understand there are some ant scripts that will do conditioning processes and sign with foundation keys, etc etc but there is a touch of black magic involved in these processes and we are not keen on adding any sort of manual process to our build and release cycles.  Right now our 'release' process involves running two commands on the command line (mvn release:prepare && mvn release:process) followed by drinking a small cup of coffee and writing announcement mails.  It would take a lot of convincing to move away from that simple of a system.

What I would love to see is a mechanism by which I could deploy our artifacts into an eclipse maven repository which was backed by whatever workflow eclipse requires to sign/seal/deliver bundles.  Any eclipse osgi type users that wants to use jetty7 right now can pull the artifacts from maven central or download the distribution from eclipse that we publish to dev.eclipse.org but as of right now there is no simple way for eclipse people to depend on jetty in an eclipse way...

But generally speaking I think the onus is on the project to do what it feels needs to be done to deliver their software to the communities they want to use it.  We want osgi containers to want to use jetty and we want maven (or ant+ivy, buildr, whoever) users to use jetty so we do everything we reasonably can do to make jetty easy for these people to consume.  That being said, I perceive the step for getting people to consume jetty in an 'eclipse' build way is a bit like shaving hair off a frog in terms of difficulty.

Anyway, that is my view as a developer on the jetty project.

Comment 61 Bob Fields 2009-10-14 17:35:36 EDT

Please add source artifacts to the repository, and standardize the groupId, artifactId, and version to make them compatible with maven conventions. See comment at https://bugs.eclipse.org/bugs/show_bug.cgi?id=288644. The UML2 artifact dependencies are broken in the maven central ibiblio repository currently. There are at least four different 'mirrored' versions of the Eclipse UML2 artifacts, all having different group/artifact/version conventions and/or outdated versions.

Comment 62 Naci Dai 2009-10-15 11:53:19 EDT

First of all  a big +1 to this initiative.  I served as the build master for WTP for a few years, and what I learned from that experience was:

a) You cannot make something that is essentially complex simple with the tools you use.  Builds for complex software is complex.

b) But, good tools makes life easy fewer accidents.  Maven repository is a good and simple one.


Here are my 0.02 cents worth for the requirements.


1) Standardize maven group/artifact ids to match eclipse naming and osgi bundle ids.  Most eclipse projects adhere to a naming convention already.  This should be the basis for group id.

          org.eclipse/
                wtp/
                    wst/
                    common/
etc.


Artifact ids should follow the bundle IDs.  Similarly, versioning should follow bundle versining all th way to the qualifier bit.


2) Static OSGi dependencies are burried into Manifest, where mavenn externalizes these in POMs.  A good maven repository should contain artifacts with coherent (with OSGi) dependency meta-data.   Maybe tools such as the Aggregator that Thomas mentioned can help here.


3) Orbit keeps IP clean third-party libraries.  They should be in the maven repository too.

4) Typical eclipse build shares the sources with an "SDK" package.  The way to manage this is via src artifacts in the repository.  We should add these artifacts to the repository for the same SDK feel.

5) I believe the success of this initiative will depend on the projects to "opt-in". If the amount of work needed to opt-in is zero, then it will be a huge success ;-).  Kudos will go to Thomas and buckminster guys if they can do this with the aggregator and still scale to the bandwidth such as repository will require.

Comment 63 Wayne Beaton 2009-10-19 11:04:11 EDT

(In reply to comment #57)
> We have active work going on in this area that we hope will be a good solution
> for Eclipse. See bug 291986.
> 
> In essence, we will provide a way for our Aggregator tool (an elaborated
> version of the current Galileo Builder) to produce a combined p2 and Maven
> repository that shares the physical artifacts.
> 
> Anyone can use the aggregator to create a combined repository and when we
> update the Galileo Builder to the latest version, a Helios Maven repository
> will come at no cost.

Thomas, can you provide pointers to any open bugs related to this work?

Can you comment on the technology you're using? Are you using Maven itself or Tyco to write the Maven repo?

Comment 64 Thomas Hallgren 2009-10-19 12:19:30 EDT

(In reply to comment #63)
> Thomas, can you provide pointers to any open bugs related to this work?
> 
There are two relevant bugs:

Bug 291986 - "Add ability to store result in Maven repository format"
This is an enhancement request on the Buckminster Aggregator where we discuss the ongoing work go make the resulting aggregation a combined p2/maven repository. Comments welcome.

Bug 292587 - "[Version] Version format should express semantics to translate MAX/MIN value"
A p2 bug that adresses some more esoteric details on how to represent min/max qualifiers in the OmniVersion. The maven versioning scheme differs from OSGi (lack of qualifier in OSGi means less then any existing qualifier, lack of qualifier in maven means greater then any existing qualifier) and the Omni Version currently lacks a piece to make that puzzle complete.

> Can you comment on the technology you're using? Are you using Maven itself or
> Tyco to write the Maven repo?

Tycho and other Maven technology seems to have problems passing the IP barrier at Eclipse.org. That, and the fact that our Aggregator already uses EMF ecore to describe both p2 and the aggregation itself made us decide to do create a model using EMF, using the Maven XSD as input, and continue from there. We get a lot for free that way (serialization, boiler plate code, etc.).

Comment 65 Jacek Furmankiewicz 2009-11-02 15:58:12 EST

+1 from me. There is a lot of opportunity to do SWT/JFace applications that do not rely on RCP (i.e. just stand-alone applications).

For those apps, the whole OSGi/RCP overhead is way too much (and sometimes the RCP UI paradigm just does not apply).

In that case, trying to create a stand-alone SWT/JFace app using Maven is quite an ordeal.

First, the existing SWT/JFace jars in the central Maven repo are horribly messed up and show be plain removed.

The only alternative is to pick JARs one by one from your local Eclipse install and then manually add them to the local Maven repo.

I have a local shell script that does not and getting it to work took a lot of time. It would be amazing if instead of all of this we could just add a dependency on jface in our pom.xml and have everything just work out of the box in a Maven project.

Comment 66 Filip Hrbek 2009-11-02 16:11:02 EST

Maven support specified in Bug 291986 is now implemented. To convert a p2 repository into a Maven 2 repository, install Buckminster Aggregator from http://download.eclipse.org/tools/buckminster/tools-3.5, map a p2 repository, switch "maven result" attribute to "true" and run "Build Repository". This operation can be performed in the headless fashion as well. Any feedback on usability is welcome.

(In reply to comment #65)
> +1 from me. There is a lot of opportunity to do SWT/JFace applications that do
> not rely on RCP (i.e. just stand-alone applications).
> 
> For those apps, the whole OSGi/RCP overhead is way too much (and sometimes the
> RCP UI paradigm just does not apply).
> 
> In that case, trying to create a stand-alone SWT/JFace app using Maven is quite
> an ordeal.
> 
> First, the existing SWT/JFace jars in the central Maven repo are horribly
> messed up and show be plain removed.
> 
> The only alternative is to pick JARs one by one from your local Eclipse install
> and then manually add them to the local Maven repo.
> 
> I have a local shell script that does not and getting it to work took a lot of
> time. It would be amazing if instead of all of this we could just add a
> dependency on jface in our pom.xml and have everything just work out of the box
> in a Maven project.

Comment 67 Illarion Kovalchuk 2009-11-25 04:09:50 EST

Here's my user story for maven repository for all eclipse stuff:

As an external developer I would like to be able to use separate org.eclipse artifacts so that my particular needs are satisfied using modern software libraries developed by a huge eclipse community.

Comment 68 Thomas Hallgren 2009-11-25 04:52:21 EST

(In reply to comment #67)
> As an external developer I would like to be able to use separate org.eclipse
> artifacts so that my particular needs are satisfied using modern software
> libraries developed by a huge eclipse community.

Until we have everything in place automate the production of maven repositories of the release train, I suggest you try the Buckminster Aggregator. Just feed it with the Galileo repository and instruct it to generate a mirror that includes maven output. That will give you a combined p2/maven repository.

Documentation here: http://wiki.eclipse.org/Buckminster_Aggregator_User_Guide

The link noted in comment #66 is deprecated. The aggregator can now be found at http://download.eclipse.org/tools/buckminster/updates-3.5

Comment 69 Karsten Thoms 2009-11-25 07:57:17 EST

Maven 3 is coming, and it has with the Tycho plugins from Sonatype out-of-the-box support for P2 repositories.

First tests look promising when building Eclipse plugins and features. I have also to test mixing "normal" and OSGi dependencies, but do not expect troubles here. Further, Maven 3 is promising to be full backward compatible to Maven 2.

So it might be that this whole discussion can be obsolete when Maven 3 / Tycho is complete production ready. No need to create also a Maven2 style repository, just add P2 repositories and done.

For now, an alpha-4 version is ready for download at Apache. With alpha-4 Tycho needs no separate Maven distro anymore. Tycho version 0.5.0 has been released end of last week.

See
- http://www.sonatype.com/people/2008/11/building-eclipse-plugins-with-maven-tycho/
- http://www.sonatype.com/people/2009/11/maven-30-alpha-3-released/

Comment 70 Nick Boldt 2010-02-11 12:05:14 EST

(In reply to comment #66)
> Maven support specified in Bug 291986 is now implemented. To convert a p2
> repository into a Maven 2 repository, install Buckminster Aggregator from
> http://download.eclipse.org/tools/buckminster/tools-3.5, map a p2 repository,
> switch "maven result" attribute to "true" and run "Build Repository". This
> operation can be performed in the headless fashion as well. Any feedback on
> usability is welcome.

Some feedback here re: size of resulting maven/p2 repo - see bug 302604.

Comment 71 Ben Arnold 2010-03-13 08:11:36 EST

First I want to say thanks for all the good work you all put in.

But now my whinge: I have just read through the comments and I'm quite taken aback at how difficult you all make this sound.

As a developer, all I want is to add SWT dependencies to my Java projects so I can write decent GUI applications.

Every other open source Java project I've ever encountered releases Maven artefacts and few of them have anywhere near the resources of the Eclipse Foundation.

I have to wonder, are you overcomplicating the problem?

The old world of manually copying jars around is long gone. Join the revolution!

Comment 72 Thomas Hallgren 2010-03-13 08:28:45 EST

(In reply to comment #71)
> The old world of manually copying jars around is long gone. Join the
> revolution!

If you think that we are over complicating it then I suggest you propose a simpler way to get things done that will satisfy all parties. Keep in mind though, that this isn't about finding a solution to replace manually copying of jars. Our task here is to try and converge two types of advanced provisioning systems (p2 and Maven) so that one can consume what the other provides.

Comment 73 Martin Oberhuber 2010-03-15 07:21:19 EDT

(In reply to comment #72)

Ben has a good point though, there is big value in going for the low-hanging fruit quickly rather than trying to solve-it-all at once.

> As a developer, all I want is to add SWT dependencies to my Java projects so I
> can write decent GUI applications.

This is a great user story. What can we do to make SWT available via Maven quickly, even if not all corner cases are handled in a first iteration?

Comment 74 Thomas Hallgren 2010-03-15 08:17:47 EDT

Martin, I'm not sure I understand your point. We have a p2 repository and we want to make it available as a Maven repository. So, we do that. Where is this immensely complicated and where is it in any way slowed down by higher hanging fruit?

Comment 75 Martin Oberhuber 2010-03-15 09:04:05 EDT

Well, I just observe that this bug has been open since July 2009 (that's 8 months by now). And there have been lots of ideas along different lines.

I just thought that having a clear goal / acceptance criterion (making SWT consumable via Maven) might help getting some end user value completed sooner, even if not all corner cases are considered in a first iteration.

I haven't followed all the discussions recently, and as per comment #68 it looks like it's possible today to set up a Maven Mirror for p2 repositories. So if you think we're on the right track with exposing SWT as Maven (to quote the concrete scenario), that's good. Are there any known major roadblocks for getting this done with Helios - including questions of disk space, ownership, performance and who'll maintain it?

Comment 76 Thomas Hallgren 2010-03-15 09:48:00 EDT

(In reply to comment #75)
> Are there any known major roadblocks for getting this done with Helios

I don't think so. Current status is that we have a slot-in replacement ready for the Helios builder in test right now. Once M6 is out, I think it's a good time to start running that in parallel with the regular builder. This slot-in replacement is capable of generating a hybrid repository (i.e. accessible from both regular Maven 2 and from p2). A Maven index is also generated.

> - including questions of disk space, ownership, performance and who'll maintain it?

The only disadvantage with the hybrid is the increased disk consumption. I don't think a regular Maven is capable of reading pack200 packed files which means that Helios must retain unpacked jars. I'm not sure if that's an issue [1].

Other then that, I see no maintenance overhead. It's still just one repository so ownership etc. is the same as for the p2 artifacts.

[1] Off topic perhaps, but if size becomes an issue, then I think we should start looking for .jar.pack.gz files that do not contain classes. There's a large amount of them and many are fairly large too (documentation and help). Packing them means doubling the size in this scenario.

Comment 77 Alex Blewitt 2010-11-09 11:14:56 EST

As there are more Maven-consumable artefacts being generated at Eclipse (Jetty, JGit) and with existing artefacts being published under http://repo1.maven.org/maven2/org/eclipse/
(though with some age discrepancies), it would be good if we could resolve this, even if it's via an external group like http://oss.sonatype.org/ (Perhaps Jason could comment?)

I suspect we wouldn't want to publish every snapshot, but releases might make sense to be published. Although Tycho helps out with the maven-building-eclipse-plugins, such bundles aren't consumable by other Maven clients at the moment.

Having a means to publish and resolve Eclipse generated code through existing published builds would be great; what needs to make it move forwards?

Btw Chris - this isn't really a PC/Mac OSX bug - can you replace it with All?

Comment 78 Brian Fox 2010-11-09 14:09:14 EST

(In reply to comment #77)
> As there are more Maven-consumable artefacts being generated at Eclipse (Jetty,
> JGit) and with existing artefacts being published under
> http://repo1.maven.org/maven2/org/eclipse/
> (though with some age discrepancies), it would be good if we could resolve
> this, even if it's via an external group like http://oss.sonatype.org/ (Perhaps
> Jason could comment?)
> 
> I suspect we wouldn't want to publish every snapshot, but releases might make
> sense to be published. Although Tycho helps out with the
> maven-building-eclipse-plugins, such bundles aren't consumable by other Maven
> clients at the moment.
> 
> Having a means to publish and resolve Eclipse generated code through existing
> published builds would be great; what needs to make it move forwards?
> 
> Btw Chris - this isn't really a PC/Mac OSX bug - can you replace it with All?

We would love to host those artifacts via oss.s.o as a mechanism to getting them vetted and into central.

Comment 79 Alex Blewitt 2010-11-09 14:45:34 EST

Brian,

Is there a document somewhere people can read up on as to how to achieve this? Would the plan be to create one top-level groupid per Eclipse project? That way (say) ECF would look after their bundles and ?Git would look after theirs and so on.

Comment 80 Bob Fields 2010-11-09 15:03:04 EST

I got tired of waiting on eclipse.org, and used Sonatype to add the artifacts to Central that I needed myself: The 3.6 Helios release EMF and UML2 artifacts plus all the dependencies needed to build the artifacts and create the javadocs. See https://issues.sonatype.org/browse/OSSRH-739.

When artifacts are finally ready to be added, they still need to meet the minimum requirements such as source + javadoc artifacts available as jar artifacts, metadata including developer names and SCM and bug tracking system URLs, and all dependencies needed to build the artifacts also available in the Central repository. See https://docs.sonatype.org/display/Repository/Central+Sync+Requirements. The OSGI version ranges still cause the maven build and dependency selection to fail.

I added a maven client test case and additional details on this process on a related bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=312656.

Comment 81 Thiago Leão oreira 2010-11-09 15:09:19 EST

Created attachment 182768 [details]
shell script to deploy the JFace artifact

Comment 82 Thiago Leão oreira 2010-11-09 15:13:24 EST

Hey all,

Attached there is a shell script that I'm using to deploy Eclipse's artifacts to oss.sonatype.org. I think we can use it as a start point to deploy others artifacts. I have scripts ready for at least 8 artifacts.

Comment 83 Jeff McAffer 2010-11-09 15:20:21 EST

as a point of interest, what is the general convention around putting other people's stuff into Maven central using their namespace?  In the past there have been problems with random people putting things in Maven repos.  Either the group ids were wrong or the versions were bogus or...  I would have thought that one of the criteria for getting in the central repo would be that the contributor is a viable/reliable source of the artifacts who has permission to use the namespace.

Comment 84 Bob Fields 2010-11-09 16:03:45 EST

I used the exact same group/artifact/version from the helios repository
http://build.eclipse.org/helios/hybrid/final. I had to change the dependency
versions from range to exact version number because maven 2.x assumes a three
part version number and selects no versions when the four part version is used.
I also changed some of the unnecessary dependencies to platform/runtime scope,
since they are included with the Eclipse platform itself and should not be
bundled with other artifacts (I run independently from the Eclipse platform in
my project though). The (very manual) process used to create the 27 required
artifacts was something like:
- Download the source and runtime artifacts from Eclipse p2 repository
- Unzip the source artifacts to a new maven project with the right
group/artifact/version, using the helios hybrid as a pom starting point but
replacing all version ranges with exact version numbers, and changing the
platform jars scope to 'optional' and 'runtime'. org.eclipse.core.*, equinox.*,
osgi.*, etc.
- Add a reference to the (manually created) Eclipse parent pom containing the
required metadata such as developers, SCM, bug tracking, lists, etc.
- Build the extracted source using the maven project dependencies. Adjust
dependencies for each project as necessary. This will also create the javadoc
artifact jar needed for Central synch.
- Replace the built jar runtime artifact with the original signed jar from
Eclipse (otherwise you will get a jar signing error when attempting to use).
- Create an artifact upload bundle for each artifact, signed with a public key,
and upload to sonatype OSS staging repository.
- Test the uploaded artifacts against an external dependent maven project.
- Open a Sonatype JIRA to synch the staged artifacts to maven Central.

I would of course prefer this entire process to be automated and not upload the
artifacts myself, for all releases plus the current snapshot build, but in the
end it needs to provide usable artifacts. I haven't tested with maven3 or with
Sonatype Nexus/Tycho or Buckminster turning a p2 repo into a maven repo.

As you can see from the Sonatype JIRA reference, it took about a month to get
the artifacts right. The convention around putting others stuff into maven
central is: Ask them first, if they don't add the artifacts within a reasonable
period of time then an external user can add the artifacts as long as all of
the Central synch requirements are met. Before October, it has been over four
years since EMF/UML2 artifacts have been available as usable maven
dependencies.

Comment 85 Jeff McAffer 2010-11-09 20:36:16 EST

Thanks for the detail Bob.  Unfortunately you lost me at step 2.  Luckily, the net important points are :

- Are original, unmolested, Eclipse.org signed, bundles being put in the Maven repo?
- Is the group id etc being used acceptable to the Eclipse community?

and I think you are saying "Yes" to both. Please confirm. 

There is a perhaps lesser concern about which versions of the bundles are being published.  For example, did the producing project intend for that specific version to see widespread use? Perhaps we can provide some general guidance in that area.

Comment 86 Bob Fields 2010-11-09 22:43:48 EST

(In reply to comment #85)
> Thanks for the detail Bob.  Unfortunately you lost me at step 2.  
The only reason I had to build from sources was to create a javadoc artifact for upload to maven Central, otherwise I would have simply bundled the source and runtime jars directly without creating a maven project. If the javadoc jar had been available through OSGI I would have used that instead.

> - Are original, unmolested, Eclipse.org signed, bundles being put in the Maven
> repo?
Yes

> - Is the group id etc being used acceptable to the Eclipse community?
There was considerable discussion about the groupId naming convention in another bugzilla thread (sorry I couldn't find it). I thought they had ended up with org.eclipse for everything, which would have resulted in thousands of artifacts under the single group, but the helios maven repository had everything as org.eclipse.<projectname> which is also the convention followed by all other projects such as jetty which also publish artifacts to Maven Central. The artifactId follows the full org.eclipse.projectname.artifact, while a number of other projects simply use artifact. The full artifact name is preferable, otherwise you will see a number of artifacts named 'common.jar'. The version is the exact same full four part major.minor.patch.vDate[-qualifier] convention used in OSGI.

> and I think you are saying "Yes" to both. Please confirm. 
> 
> There is a perhaps lesser concern about which versions of the bundles are being
> published.  For example, did the producing project intend for that specific
> version to see widespread use? Perhaps we can provide some general guidance in
> that area.
Other than the EMF and UML2 artifacts required to load models into memory, the only other uploaded artifacts were the platform jars required to build the sources. Only the 3.6.0 major release bundles were uploaded.

Comment 87 Jeff McAffer 2010-11-09 23:04:32 EST

Thanks again Bob. I understand that you have done only a particular set of bundles.  In the context of this bug we should be looking for the general approach for how Eclipse output can be feed into Maven repos in a more automated, repeatable and ultimately, reliable, fashion.  The Javadoc thing is distressing.  Presumably we can do better in our builds though I've lost track of how that part of it works.  

Is that Javadoc requirement something related to Maven Central or Maven itself?  If its the former then I'm not sure if somewhere in the 80+ comments here (or elsewhere) there has been a determination that we should or should not be putting things in Maven Central. If we are not then perhaps it doesn't matter.

One thing that did stand out in your last comment was the bit about the group ids.  We should nail that down. If there was considerable discussion (presumably involving folks with reasonable understanding of the issues) and a particular direction was decided, it seems that folks ought to follow that guidance. Otherwise we end up with inconsistencies and the consumer community suffers.

Comment 88 Bob Fields 2010-11-09 23:32:58 EST

(In reply to comment #87)
> Is that Javadoc requirement something related to Maven Central or Maven itself?
It's a requirement from Sonatype OSS, in order to use their Central Synch service, the only way I know to get artifacts into maven Central.

Here's the bug with the maven groupId discussion: https://bugs.eclipse.org/bugs/show_bug.cgi?id=288644.

Comment 89 Aaron Digulla 2010-11-10 06:31:39 EST

(In reply to comment #82)
> Hey all,
> 
> Attached there is a shell script that I'm using to deploy Eclipse's artifacts
> to oss.sonatype.org. I think we can use it as a start point to deploy others
> artifacts. I have scripts ready for at least 8 artifacts.

Your script uses the wrong artifactId. Always include the *whole* groupId in the artifactId. Otherwise you'll get many artifacts that are called "ui.jar" or "source.jar".

Not a problem with Maven but breaks your neck when you try to use that to build web sites.

Comment 90 Brian Fox 2010-11-10 10:01:44 EST

> Is that Javadoc requirement something related to Maven Central or Maven itself?
>  If its the former then I'm not sure if somewhere in the 80+ comments here (or
> elsewhere) there has been a determination that we should or should not be
> putting things in Maven Central. If we are not then perhaps it doesn't matter.


Javadocs and sources along with pgp signatures for everything is a requirement of Central: https://docs.sonatype.org/display/Repository/Home

> 
> One thing that did stand out in your last comment was the bit about the group
> ids.  We should nail that down. If there was considerable discussion
> (presumably involving folks with reasonable understanding of the issues) and a
> particular direction was decided, it seems that folks ought to follow that
> guidance. Otherwise we end up with inconsistencies and the consumer community
> suffers.

The group id should be org.eclipse.[project/groupname] so it's separated by project but everything lives under the org.eclipse domain. You can take a look at org/apache for an example

Comment 91 Bob Fields 2010-11-10 11:06:31 EST

Eclipse currently also uses the org.eclipse.<projectname> for groupId in maven Central. See all of the subdirectories under http://repo1.maven.org/maven2/org/eclipse. I count 50 projects (groupId values). Each groupId may have dozens of subproject artifacts underneath. The maven directory layout is org/eclipse/<projectname>/<artifactId>/<version>/<artifactId>-<version>.jar .pom -sources.jar amd -javadoc.jar. Most artifacts are from Nov 2007, except for the ones I updated in Sept/Oct plus jetty, jdt, and xsd.

If the current groupId values are changed, maven cannot connect the old with the new artifact, and there is the possibility of loading two incompatible versions in the same classloader when some dependencies have transitive dependencies on different versions. They are seen as two completely different artifacts instead of two different artifact versions where it must resolve which one to use. That's usually OK if the package internal structure has changed (i.e. UML2 1.x -> 2.x) but otherwise it can cause runtime problems depending on the artifact load order. By convention, groupId follows the top level package within the child artifacts, that makes it easy to find dependent jars given a missing package.classname. Collisions from 2007 are unlikely, but are very likely if Eclipse starts updating all releases and patches to Central or provides an alternate repository location, and changes the current groupId somewhere in the middle. The groups that were updated in Sept under org.eclipse were: ant, core, eclipse, emf, equinox, osgi, and uml2.

The maven Central sources/javadocs requirement must be something new, because there are plenty of artifacts in Central without sources or javadocs. Is there an Eclipse OSGI artifact jar/zip containing only javadocs available for each runtime artifact, or is it only online? It seems like that would be useful since Eclipse project dependencies can point to both source and javadoc jars locally.

Comment 92 Jeff McAffer 2010-11-10 12:09:31 EST

(In reply to comment #91)
> By convention, groupId follows the top
> level package within the child artifacts, that makes it easy to find dependent
> jars given a missing package.classname.

Would be great if there was (perhaps there already is) a wiki page that captured all this group id stuff.  There are some places that say org.eclipse.<projectname> and others that say org.eclipse.<potentially multisegment dominant package name>.  These are different things.  

> The groups that were updated in Sept under org.eclipse
> were: ant, core, eclipse, emf, equinox, osgi, and uml2.

These are good examples.  There are no packages org.eclipse.eclipse...  org.eclipse.osgi (bundle and package) are part of the Equinox project.

> Is there
> an Eclipse OSGI artifact jar/zip containing only javadocs available for each
> runtime artifact, or is it only online? It seems like that would be useful
> since Eclipse project dependencies can point to both source and javadoc jars
> locally.

Source is available on a per-bundle basis but in general we have been shipping "doc" bundles that have the doc for a number of different bundles.  As I recall there were (many years ago) issues around getting well integrated Javadoc references if the doc was built/shipped separately.  Perhaps that has changed or we are smarter now.

Comment 93 Jeff McAffer 2010-11-10 12:32:14 EST

For more confusion fun around group and artifact id, see the following mail thread
   http://dev.eclipse.org/mhonarc/lists/cross-project-issues-dev/msg04947.html

Comment 94 Aaron Digulla 2010-11-11 06:38:43 EST

Just for the record: I use this command line to import my current Eclipse installation into my local maven repository:

mvn eclipse:make-artifacts -DstripQualifier=true -DeclipseDir=.../eclipse

This works with Maven 2 and 3 and the resulting names are useful for working with stable Eclipse versions (i.e. when you don't need to be able to distinguish between todays and yesterdays nightly build).

It's also the best naming scheme if you always want the latest build :-)

The documentation of the maven-eclipse-plugin (i.e. the one which Maven uses to talk to Eclipse) suggest that this is obsolete and shouldn't be used but the recommended way causes lots of problems.

Comment 95 Aaron Digulla 2010-11-11 08:34:57 EST

(In reply to comment #93)
> For more confusion fun around group and artifact id, see the following mail
> thread
>    http://dev.eclipse.org/mhonarc/lists/cross-project-issues-dev/msg04947.html

Should I comment there or here?

Comment 96 David Carver 2010-11-11 10:45:50 EST

(In reply to comment #94)
> Just for the record: I use this command line to import my current Eclipse
> installation into my local maven repository:
> 
> mvn eclipse:make-artifacts -DstripQualifier=true -DeclipseDir=.../eclipse
> 
> This works with Maven 2 and 3 and the resulting names are useful for working
> with stable Eclipse versions (i.e. when you don't need to be able to
> distinguish between todays and yesterdays nightly build).
> 
> It's also the best naming scheme if you always want the latest build :-)
> 
> The documentation of the maven-eclipse-plugin (i.e. the one which Maven uses to
> talk to Eclipse) suggest that this is obsolete and shouldn't be used but the
> recommended way causes lots of problems.

Becareful of depending on the eclipse:* plugin for maven, it is going away and being deprecated.   

The best way to work with maven with in eclipse is to use m2eclipse which is coming over to eclipse.org.

Comment 97 Thomas Hallgren 2010-11-11 11:26:35 EST

I'm curios. Doesn't Maven 3 / Tycho consume p2 repositories today? If so, why is this still an issue?

Comment 98 David Carver 2010-11-11 11:35:33 EST

(In reply to comment #97)
> I'm curios. Doesn't Maven 3 / Tycho consume p2 repositories today? If so, why
> is this still an issue?

Correct, Maven 3 / Tycho consumes p2, but there are many people still using Maven 2 and plugins like Pax.

Honestly, I'm not sure how much of a concern this should be for Maven 3 devs, as they can consume from P2 repos using Tycho.

Comment 99 Thomas Hallgren 2010-11-11 11:46:59 EST

OK, so the people who benefits from this are using a tool that doesn't (yet)
provide p2 access. Wouldn't it be fair to assume that Pax will either provide
such support soon or that people will move over to Tycho?

I'm just concerned that we spend a lot of efforts trying to cater for a perhaps
rapidly diminishing group of users. I might be completely wrong of course but
shouldn't we move in a direction that helps people use Eclipse Technology? In
this case Tycho and P2.

From a p2 perspective, perhaps it would be more interesting to help Pax with p2
support.

Comment 100 Wayne Beaton 2010-11-11 12:28:17 EST

I presented something that I've been calling Slingshot at EclipseCon last year. Slingshot is intended to be a monster repository that Eclipse projects can opt into joining. Using the same B3 aggregation technology as Helios, it pulls together many different repositories and presents them as a single unified.

http://wiki.eclipse.org/Slingshot

Unfortunately, the effort has been stalled while I've been trying to sort out some issues from the Foundation side of things. Primarily, I am concerned that the webmaster team doesn't have extra cycles to babysit this.

FWIW, the aggregator used by the release train can spit out Maven repository metadata alongside the p2 metadata; I'm pretty sure that we're turning on this feature for Indigo.

It occurs to me that an important question was never asked: is this what we want? Do we really want to have a Maven repository hosted at eclipse.org or does it make more sense to have the artifacts maintained at Maven Central? Or both?

Also... can somebody summarize the current state of this bug? I don't understand Maven well enough to make sense of the ~100 comments.

Comment 101 Jesse McConnell 2010-11-11 12:33:10 EST

"It occurs to me that an important question was never asked: is this what we
want? Do we really want to have a Maven repository hosted at eclipse.org or
does it make more sense to have the artifacts maintained at Maven Central? Or
both?"

If we as eclipse projects are allowed to consume from maven central then sure, it makes sense to just have everything at maven central.  However if we as eclipse projecta couldn't use it then it makes more sense to have a maven repository @ eclipse that we _could_ use and then let the sonatype folks manage syncing those contents to maven central for wider usage by non-eclipse projects if the broader community desires.

imo at least

Comment 102 Wayne Beaton 2010-11-11 12:42:13 EST

(In reply to comment #101)
> If we as eclipse projects are allowed to consume from maven central then sure,
> it makes sense to just have everything at maven central.  However if we as
> eclipse projecta couldn't use it then it makes more sense to have a maven
> repository @ eclipse that we _could_ use and then let the sonatype folks manage
> syncing those contents to maven central for wider usage by non-eclipse projects
> if the broader community desires.

Okay, so maybe the question was asked. But after 100 comments, I start to lose focus.

Anyway... is it enough that the release train repository be exposed as a Maven repository? Or is there other projects that you need to consume?

Does an upgrade to Maven 3 solve your problem as Thomas suggests? i.e. can you consume the p2 repository?

Is it a reasonable workaround to say "just use Maven 3"? Will the answer be different in six months?

Comment 103 Jesse McConnell 2010-11-11 14:23:56 EST

Personally I would like to see orbit exposed as a durable maven repository.

Durable in that once something goes into the maven repository, it stays there.  

This would enable to me have our normal jetty build (maven 2) distribution creation address something that is reliable for the handful of signed orbit dependencies we need to pull.  We have tried using orbit p2 repositories before but their lifecycle is too non-deterministic for our release tags to build reliably against (our fault for using I** p2 repos but at the time they were the only place to get the given dependency).

However I doubt our specific case is the norm for most eclipse projects, we have a strong history in our process and changing it up too much would strand a lot of our users.  It could very well be for the majority of people within eclipse maven 3 and tycho will handle all of their problems, either now or over the course of the next year.

Comment 104 Wayne Beaton 2010-11-11 14:39:30 EST

(In reply to comment #103)
> Personally I would like to see orbit exposed as a durable maven repository.

Doesn't this, then, become a requirement for the Orbit project? Is there already a bug open on Orbit for this?

Comment 105 Jesse McConnell 2010-11-11 15:10:30 EST

I'll raise an issue with orbit if eclipse as an organization decides to support maven style repositories...which is in essence this issue.

Reading back over this issue I suspect the answer is no and the expectation will be that people that want to consume things like SWT into their normal maven builds will have to use maven 3 and tycho pointed at eclipse p2 repositories.  It seems that multiple groups of people have come up with workable solutions to the problem of making a maven repository of eclipse artifacts but none have been picked up and accepted by the organization, codified and supported.

For jetty, we have our work arounds and we publish into both maven central and into eclipse p2 repositories to support as wide an array of people as possible.

Comment 106 Mathieu Baudier 2010-11-11 15:25:40 EST

To add my 2 (euro) cents to this already long discussion:

We have been struggling for years to integrate Maven and OSGi, and even hacked our own Maven plugins for this purpose.

We followed Tycho development but were unsure about its development pace, about the community around it and about its sustainability.

Tycho has a very interesting approach and it now seems to have become a real option with the official release of Maven 3 (this was always a show stopper for us that it did not rely on a stable release of Maven: we are talking of release management tools after all...)

IMHO, as other have suggested, Eclipse should focus on p2, and those of us interested in Maven and OSGi should focus on Tycho and help them.

I think that people will often want to build their own bundle ecosystem by mixing various sources and best would be to focus on providing them with the right tools to do so.

We, for example, use massively the Spring Source Enterprise Bundle repository [1] (compatible with Maven) along with our own repackaged third parties and with Eclipse releases imported via mvn eclipse:make-artifacts [2].

Being able to integrate better with p2 would be great, but for the reason above (creating our own mix), we are actually not so interested in an Eclipse Maven repository as such.

[1] https://ebr.springsource.com/repository/app/
[2] http://maven.apache.org/plugins/maven-eclipse-plugin/make-artifacts-mojo.html

Comment 107 Wayne Beaton 2010-11-11 15:26:24 EST

(In reply to comment #105)
> I'll raise an issue with orbit if eclipse as an organization decides to support
> maven style repositories...which is in essence this issue.

How do you define "support"? AFAIK, we don't "support" Maven builds at all, and yet projects build with Maven. Heck, up until a couple of years ago, we didn't support the notion of a p2 repository either.

The Orbit project may opt to expose a Maven repository. The Eclipse Foundation does not have the resources to do it for them.

The more that I think about the more that I think the Slingshot idea isn't the right one. If the release train and Orbit repositories are exposed with Maven metadata, is that enough? Is anything else missing?

What amount of effort are we looking at to provide the stability and metadata required to Orbit?

Are there projects outside of the release train and Orbit that need to be exposed via Maven? Is it too harsh to suggest that these hypothetical projects should just join the release train to get that benefit?

Comment 108 Jeff McAffer 2010-11-11 16:22:46 EST

Actually I am intrigued by the Tycho potential.  If it really is a solution that Maven folks can simply consume p2 repos then, with apologies to Maven 2 folks, that seems like the path forward.  We would then be done on this topic.

Comment 109 Alex Blewitt 2010-11-11 18:36:19 EST

(In reply to comment #108)
> Actually I am intrigued by the Tycho potential.  If it really is a solution
> that Maven folks can simply consume p2 repos then, with apologies to Maven 2
> folks, that seems like the path forward.  We would then be done on this topic.

There seems to be a misconception here. 

Maven 3 is fully backwardly compatible with Maven 2. Both use the same Maven repository layout. Both Maven 2 and Maven 3 can build JARs as well as OSGi bundles - Felix is built this way. 

Tycho can consume P2 repositories and Maven repositories. It is basically a replacement for PDE build. However, other builds cannot consume anything that Tycho produces. 

In essence, Tycho is only an Eclipse build solution. Whether or not Eclipse projects moved to tycho or not, the only way bundles would be usable is if they were then published into a Maven repository format. The chance of everyone to Tycho is zero. 

On the other hand, several bundles that Eclipse produces can be used in other OSGi runtimes as well as non-OSGi runtimes. P2 is not suitable for either of these cases. 

I believe the way forward would be to find a way of converting release repositories to maven metadata, and then either do redirects to the locations of JARs in a Helios repository, or let Sonatype mirror the bits. 

Incidentally adopting a Maven repository format would solve the problem of repeated JARs which plagues Eclipse distributions. I can't count how many copies of org.eclipse.osgi_* there are in the Eclipse mirror ...

Comment 110 Ian Bull 2010-11-12 00:14:23 EST

(In reply to comment #75)
> Well, I just observe that this bug has been open since July 2009 (that's 8
> months by now). And there have been lots of ideas along different lines.

We are now at 16 months ;-)

This bug report has a lot of interesting ideas, real world use cases and concrete suggestions, but like most bugs with > 100 comments, we are unlikely to resolve this in a way that suits everyone.  Looking through the comments, here are the general themes I see:

1. Produce maven artifacts for downstream Eclipse use:
- This is less interesting now that Tycho can consume p2 repositories. Most Eclipse projects should be building p2 repositories, so if you use Maven and require another Eclipse project, Tycho should be your technology of choice.

2. Create Maven repos for Orbit:
- This has come up here, and while it's and interesting idea, this request should be directed to the fine folks over at orbit.  Like point #1 though, Orbit should have a p2 repository, and consumers of Orbit should use that.  If there is a specific need for Orbit artifacts in a Maven repository, let's take this over to Orbit for discussion.

3. Decide on naming / version conventions for our content:
- This has been covered (and resolved I think) in other bugs. See bug 288644.

4. Create a maven repository for Eclipse content (for use outside Eclipse):
- This appears to be the original request (comment#0).  Since this bug was first opened, Buckminster now has support for creating a Maven repository from a p2 one.  So here's the question: is that enough?  Is this easy to setup? How much *extra* work will this be for the release train?  Can we simply run this tool on the Indigo repository, and will everyone be happy with that?  IMHO I don't think we need all Eclipse artifacts in an Maven repository (how many people use pde.ui in a non Eclipse IDE/RCP way?).  It might be better to have an opt-in approach (for Indigo), and if those projects are pleased with the results, then we can consider publishing all of Indigo++ as a Maven repository.  Are there any Eclipse projects that would like to be automatically published to a Maven repository? (Note: I'm asking the project (teams), not the consumers).

5. Put Eclipse content on maven central:
- This point has also come up (with some scripts to actually make this happen).  I assume if some projects opt-in to a p2 -> maven conversion, we can simply take that output and put it in Maven central (with help from the fine folks at Sonatype).  

I hope that helps summarize the discussion here, and if I missed anything please comment.

Comment 111 Thomas Hallgren 2010-11-12 03:19:38 EST

(In reply to comment #109)
> Incidentally adopting a Maven repository format would solve the problem of
> repeated JARs which plagues Eclipse distributions. I can't count how many
> copies of org.eclipse.osgi_* there are in the Eclipse mirror ...

This can be cured with p2 as well. Just use a big shared artifact repository for everything that Eclipse produces. p2 allows for good separation between meta-data and artifacts. There could well be hundreds of different meta-data repositories sharing one huge artifact repository and they could be aggregated and tailored for specific use-cases. So far nobody has made any effort to actually exploit that route.

Adopting the Maven 2 repository format, where the separation isn't as good and the meta-data is less descriptive, would not be a good solution for this IMO.

Comment 112 Alex Blewitt 2010-11-12 03:44:28 EST

(In reply to comment #110)

> 4. Create a maven repository for Eclipse content (for use outside Eclipse):
> - This appears to be the original request (comment#0).  Since this bug was
> first opened, Buckminster now has support for creating a Maven repository from
> a p2 one.  So here's the question: is that enough?

Probably. For an official released version of Eclipse content, if Buckminster can generate a Maven repository then that's probably sufficient.

> IMHO I don't think we need all Eclipse artifacts in an Maven repository (how many
> people use pde.ui in a non Eclipse IDE/RCP way?).  

There are potential downstream use-cases like bndtools (https://github.com/njbartlett/bndtools/tree/master/bndtools.core/) which might want to integrate ones that you wouldn't otherwise think of. Doing all is probably easier than doing some.

There's also three types of use cases:

* For use as standalone JARs in a Java application
* For use as OSGi bundles in a different framework (Felix, Knopflerfish)
* For use inside an Eclipse runtime

Tycho was designed for the last one only. But using 'can run outside OSGi' is probably not a sufficient filter. And even some of the *.core.* products might be useful outside - for example, if CDT's parser were available, or the PDE API analysis, then if these were available on a Maven repository then there's a possibility that others could use them to write Maven reports for the Codan static analysis coverage or PDE API coverage.

> It might be better to have an opt-in approach (for Indigo) ...

I'd suggest that it might not be useful to do that, since you may then miss out on functionality that might not be seen originally. For example, what if EMF doesn't opt-in? 

> 5. Put Eclipse content on maven central:
> - This point has also come up (with some scripts to actually make this happen).
>  I assume if some projects opt-in to a p2 -> maven conversion, we can simply
> take that output and put it in Maven central (with help from the fine folks at
> Sonatype).  

An automatically generated Maven repo is necessary but not sufficient for this case. Sonatype also require the full list of committers, and a separate source and JavaDoc JAR for each item. I don't know if this is practical or not.

BTW the Maven repository has a standard format but has its metadata and JARs individually accessible, so one could use Eclipse's existing mirrors e.g.

org/eclipse/jdt/3.0.0/org.eclipse.jdt.pom // metadata file
org/eclipse/jdt/3.0.0/org.eclipse.jdt-3.0.0.jar ->redirect to http://mirror.eclipse.org/helios/plugins/org.eclipse.jdt_3.0.0.v20101010.jar

This would probably need to be running as a servlet or equivalent to set up the redirections automatically, and may fail if the remote end is only available in packed form.

The oss.sonatype would be a much better way of handling it if Buckminster can generate the necessary parts.

Comment 113 Aaron Digulla 2010-11-12 08:02:03 EST

(In reply to comment #97)
> I'm curios. Doesn't Maven 3 / Tycho consume p2 repositories today? If so, why
> is this still an issue?

Because I can't find any useful information how to *use* Tycho on the project's homepage. It just says how great Tycho is and that it's the future and everything but there is no easily consumable tutorial, no link "Documentation" or "Download" or "Getting Started".

Comment 114 Aaron Digulla 2010-11-12 08:07:49 EST

(In reply to comment #98)

> Honestly, I'm not sure how much of a concern this should be for Maven 3 devs,
> as they can consume from P2 repos using Tycho.

They can? From what I understand, Tycho is just a replacement for PDE. I need a way to be able to add Eclipse artifacts as dependencies in my POM. Can Tycho do that? Is there documentation for that anywhere? How do I add Tycho to my POM?

Comment 115 Barrie Treloar 2010-11-12 08:19:24 EST

(In reply to comment #110)
> 4. Create a maven repository for Eclipse content (for use outside Eclipse):
> - This appears to be the original request (comment#0).  Since this bug was
> first opened, Buckminster now has support for creating a Maven repository from
> a p2 one.  So here's the question: is that enough?  Is this easy to setup? How
> much *extra* work will this be for the release train?  Can we simply run this
> tool on the Indigo repository, and will everyone be happy with that?  IMHO I
> don't think we need all Eclipse artifacts in an Maven repository (how many
> people use pde.ui in a non Eclipse IDE/RCP way?).

I use Maven to seed my Eclipse RCP Target Platform. But this requires me (as a once off task) to manually seed our internal Maven Repository from our installed Eclipse versions.

As Alex points out in comment #109, "I can't count how many copies of org.eclipse.osgi_* there are in the Eclipse mirror".

My RCP version is still stuck back in 3.2.2 and I haven't revisited whether the Ant build support can pull these from P2 repos, but ultimately I want one master copy on my hard drive (which is currently my ~/.m2 repo) and an automated way to seed that in my projects so I can avoid putting jars into version control systems and avoid having to manually building Target Platforms.

I'd much rather avoid doing the manual seed step and pull the artifacts from a sanctioned site.

Comment 116 luke w patterson 2010-11-12 08:39:31 EST

(In reply to comment #113)
> (In reply to comment #97)
> > I'm curios. Doesn't Maven 3 / Tycho consume p2 repositories today? If so, why
> > is this still an issue?
> 
> Because I can't find any useful information how to *use* Tycho on the project's
> homepage. It just says how great Tycho is and that it's the future and
> everything but there is no easily consumable tutorial, no link "Documentation"
> or "Download" or "Getting Started".

Another important aspect of the current state of Tycho: it _heavily_ favors manifest-first development over POM-first development.  i.e. create a MANIFEST.MF first, and let Maven dynamically figure which <artifact>s you need.

A lot of teams will only want to use POM-first development.

Comment 117 David Samuelsson 2010-11-12 09:03:21 EST

(In reply to comment #114)
> They can? From what I understand, Tycho is just a replacement for PDE. I need a
> way to be able to add Eclipse artifacts as dependencies in my POM. Can Tycho do
> that? Is there documentation for that anywhere? How do I add Tycho to my POM?

see https://docs.sonatype.org/display/TYCHO/Tycho+reference+card

also the discussion board that is used by tycho is used here:

http://software.2206966.n2.nabble.com/Tycho-Users-f3053503.html

Comment 118 David Carver 2010-11-12 09:07:57 EST

(In reply to comment #117)
> (In reply to comment #114)
> > They can? From what I understand, Tycho is just a replacement for PDE. I need a
> > way to be able to add Eclipse artifacts as dependencies in my POM. Can Tycho do
> > that? Is there documentation for that anywhere? How do I add Tycho to my POM?
> 
> see https://docs.sonatype.org/display/TYCHO/Tycho+reference+card
> 
> also the discussion board that is used by tycho is used here:
> 
> http://software.2206966.n2.nabble.com/Tycho-Users-f3053503.html

And for the top level docs:

https://docs.sonatype.org/display/TYCHO/Index

Comment 119 Ian Bull 2010-11-12 12:18:01 EST

I need to apologize, as I should have never assumed 'how' people intend to use the artifacts from Eclipse.org.  I assumed that anybody consuming Eclipse bits as OSGi bundles would be using an OSGi build technology like PDE/Build or Tycho, but this is clearly not the case.

If software engineering has taught me anything, it's that you should never make assumptions about how your software will be used.

Having said that, I still think an opt-in approach (at first) is a better way to go.  We don't know all the implications of publishing a Maven repository along side our release, and it's not clear to me who would be responsible for the contents of this repository. For example, let's say Bundle XYZ worked properly when consumed from a p2 repository, but didn't work correctly when built with Maven.  Who would shoulder the responsibility for this?  -- Maybe someone will guarantee that this can never happen, and in that case, I think I would have my answer ;).

Please don't get me wrong, I have nothing against Maven or Maven repositories. I just think we need an incubation period where we put some of our artifacts in a Maven format before we decide the the entire release train will be delivered this way and we sign this up as a supported format.

In the end, I think my question in comment#110 is still valid, which projects would like their builds published as Maven repositories?  There are clearly users who will benefit from this, but the reality is, if the projects (and thus the committers) are not interested in making this happen, then it won't happen at Eclipse.org.

Comment 120 Scott Lewis 2010-11-12 13:14:46 EST

(In reply to comment #119)
<stuff deleted>
> 
> In the end, I think my question in comment#110 is still valid, which projects
> would like their builds published as Maven repositories?  

ECF is one project that would benefit from having our builds published as Maven repositories.  We've heard repeatedly that ECF consumers would like to use

a) parts of ECF (e.g. remote services) on non-Equinox frameworks (like Felix)
b) parts of ECF within non-OSGi applications (e.g. ECF generic provider client and/or servers)

Some (but not all) of these consumers would clearly like to use Maven for build and/or install.
 
>There are clearly
> users who will benefit from this, but the reality is, if the projects (and thus
> the committers) are not interested in making this happen, then it won't happen
> at Eclipse.org.

For us, our desire/interest in making this happen for our community has little to do with it.  We simply do not have the resources to build and then support the usage of multiple ECF distributions.  In other words, in our case this is not a matter of will or listening to a community...rather it is a matter of means.

An implication of this is that an 'opt-in' strategy by EF/release train means that we don't really have a choice...we cannot 'opt-in' without some resources to do so...otherwise we are setting ourselves up on an unsustainable path (unsustainable for both committers and consumers).

My guess is that when it comes to releng resources, there are at least several high-profile mature projects that also couldn't/wouldn't 'opt-in' for similar resource reasons.  So pushing the responsibility for something like this on the projects directly is in effect saying that only those projects that have resources to spare on things like this will be able to participate...which is completely separate from whether their community even cares about consuming project artifacts in multiple ways...as ECF's community clearly does.  

Sadly, IMHO this is in keeping with a long tradition at EF of having the projects fend mostly for themselves rather than actually sharing resources (e.g. testing, build/releng, proj mgmt, IP, inter-project coordination, marketing, etc), but I think most can see that at some point that approach breaks down.

Comment 121 Alex Blewitt 2010-11-12 15:00:15 EST

An opt-in strategy will be a sure way to kill off a Maven repository. You need the transitive closure of your dependencies available (not necessarily in the 'same' repo, but in 'a' repo) and as soon as a project like that doesn't opt-in, it blocks everyone who depends on it. Take ECF and EMF out of the box, and you've neutered about half of the Eclipse projects in a single go ...

Additionally, I think this is bigger than a single project. It's really an overlay activity, like EPP, that takes the output from P2 and makes it available in a builder for the general good. Maybe we need a (volunteer) MPP project to make it happen.

By the way, Google App Engine is now available in Maven, and they would be well positioned to take advantage of newer Eclipse bits funnelling through that way.

http://code.google.com/p/googleappengine/issues/detail?id=1296
http://googlewebtoolkit.blogspot.com/2010/08/how-to-use-google-plugin-for-eclipse.html

Comment 122 Bob Fields 2010-11-12 15:28:00 EST

(In reply to comment #109)
> Maven 3 is fully backwardly compatible with Maven 2. Both use the same Maven
> repository layout. Both Maven 2 and Maven 3 can build JARs as well as OSGi
> bundles - Felix is built this way. 

Ummm not quite. Some of the maven site plugins and antrun do not currently work
with maven3, see
https://cwiki.apache.org/MAVEN/maven-3x-plugin-compatibility-matrix.html. There
are also some internal plugin API changes which must be fixed, especially
around documentation output. It will probably be a while before our project can
migrate from maven2.2 to maven3. We are OK for now until the next major release
of Eclipse, but if there still isn't something publicly available from Eclipse
then I will probably have to repeat the painful process of creating the eclipse
artifacts manually and synching to Central again, with whatever group and
artifact names are acceptable to the community.

As an independent non-Eclipse maven consumer of Eclipse artifact dependencies,
I don't want to have to maintain my own public Tycho or Buckminster or Nexus or
whatever repository, I simply want to point to somewhere stable with release
artifacts that I (and any other user of my public open source project) can
consume. Central is preferred, because there is less project and end-user
configuration to use it, plus if I ever want my project to be in Central than
all of my project dependencies must also be in Central. I don't need to build
Eclipse plugins or OSGI bundles. Uploading artifacts once or twice per year is
preferable to maintaining a public repository.

Comment 123 Wayne Beaton 2010-11-12 16:20:19 EST

(In reply to comment #121)
> An opt-in strategy will be a sure way to kill off a Maven repository. You need
> the transitive closure of your dependencies available (not necessarily in the
> 'same' repo, but in 'a' repo) and as soon as a project like that doesn't
> opt-in, it blocks everyone who depends on it. Take ECF and EMF out of the box,
> and you've neutered about half of the Eclipse projects in a single go ...

Is it enough, then, to expose release train as a Maven repository? I think this would be enough to serve the outside community? Does that sound right? To serve the "inside" community, would it be enough to expose Orbit in addition to the release train?

I agree that opt-in probably isn't going to work. Historically, we get a lot of push-back any time we require even a little bit of extra administrative overhead.

I'd like to hear how much extra work David and the B3 folks think exposing the release train as Maven in addition to p2 will take. I believe it's a relatively small amount of work, but please correct me if I'm wrong.

I agree that Mavenification of Orbit is an Orbit problem, but we may be able to leverage some energy from some of the folks here to make that happen. Does anybody have any sense for how much work (initial and ongoing) will be required to get Orbit to spit out a Maven repository? I understand that stability of artifacts is an issue, so that would have to be factored into the estimate. Is it time to open a bug against Orbit?

Comment 124 Jesse McConnell 2010-11-12 16:27:41 EST

IMO it would also be foolish to not ask the sonatype folks how they might help this effort, they have volunteered to help in the past

I suspect a couple of them are likely watching the issue

Comment 125 Aaron Digulla 2010-11-15 05:10:27 EST

(In reply to comment #123)

> I agree that opt-in probably isn't going to work. Historically, we get a lot of
> push-back any time we require even a little bit of extra administrative
> overhead.

IMO, this has to be part of the standard build. I've just tried (without success) to build BIRT from sources. A while ago, I tried to build SWT and Equinox. Let me summarize my experience like so: I won't try that again without someone paying me a lot of money.

Building (parts of) Eclipse is too brittle today. As a developer, I don't expect a lengthy explanation (even though building SWT seems to have become more simple: http://www.eclipse.org/swt/faq.php#howbuildplugin), it should be a two-liner:

1. Download source from ...
2. Run <tool>

where <tool> should be smart enough to figure out the rest. Example: If I run <tool> on Linux, it should prepare my build for the current system because most people will want to start there.

That is what Maven is all about: It's "./configure ; make" for Java.

No need to read build properties spread over 10 files (build.xml, build.properties, plugin.xml, MANIFEST.MF, customBuildCallbacks.xml, feature.properties, feature.xml, *.product, site.xml -- did I miss one?)

So when people complain about "little bit of extra overhead", there is nothing "little" in there: Any tiny change to the build system can cause hours and days of grief and pain. *This* has to be fixed because that really hurts.

Whether the build system also produces artifacts for Maven is of little interest to the projects. They need a simple, stable tool to do their work.

From my point of view, PDE and p2 are a complete failure in that regard. I didn't try Buckminster yet but it seems to be based on PDE, so I'm not expecting much.

Or to put it even more bluntly:

- Build unknown maven project: A couple of minutes followed by a possibly long coffee break.

- Build Eclipse project: A few weeks of struggling with odd error messages, unfriendly Ant scripts, slow SAT solvers, outdated documentation, bugs and frustration.

PS: My "solution" is to download the sources and the binary, compile the modified source files against the binary, unJAR the plugin, replace the class files and JAR it up again and pray that Eclipse will pick the changes up when I replace the original JAR with my copy.

Comment 126 Aaron Digulla 2010-11-15 05:11:50 EST

(In reply to comment #117)

> see https://docs.sonatype.org/display/TYCHO/Tycho+reference+card
> 
> also the discussion board that is used by tycho is used here:
> 
> http://software.2206966.n2.nabble.com/Tycho-Users-f3053503.html

Thanks for the links. I opened a bug; hopefully they will soon show up on the home page.

Comment 127 Thomas Hallgren 2010-11-15 06:49:28 EST

(In reply to comment #125)
> - Build unknown maven project: A couple of minutes followed by a possibly long
> coffee break.
> 
I'm sure everyone agrees with how splendid and trouble free Maven is ;-)

I don't think that's whats being debated here though. Eclipse.org provides p2 repositories of everything. Those repositories are not just serving various builds, they are also the very source from where all Eclipse instances install and upgrade.

The question isn't if we're going to replace this complete failure of a provisioning system. It's what needs to be done in order for Maven users to use the repository. I'm still questioning if a complete Maven 2 style copy is the best way to achieve that objective.

It would be interesting to hear from the Sonatype guys on this matter. How hard was it to make Maven grok p2?

Comment 128 Aaron Digulla 2010-11-15 09:38:58 EST

(In reply to comment #127)

> It would be interesting to hear from the Sonatype guys on this matter. How hard
> was it to make Maven grok p2?

IMO this is not the question we should ask.

Right now, Tycho 0.10.0 has a lot of bugs which make it useless to build Eclipse artifacts (TYCHO-404, 533, 524 and 527). 

So that leaves me with two approaches:

1. I can use eclipse:make-artifacts which just works and is rock solid. But it's deprecated. And people can't agree which name to give the artifacts.

2. I can use Tycho which produces nice error messages but doesn't support enough of the p2/PDE corner cases to allow me to build something like BIRT (I just tried with org.eclipse.birt.core which just depends in ICU and eclipse.core.runtime).

Which is the core of my rant here: Stop making the lives of the 20-30 everyday committers more easy, start making it easier for *everyone*. And one of the first steps is to admit that Ant is not a good tool to build something as complex as Eclipse. The tool is just too simple for the task.

The next step is to have a good look at p2 and figure out why Maven can resolve dependencies in a few milliseconds when p2 needs several minutes for the same task. If no one can get the SAT solver right, well, then the SAT solver is not the solution. Yes, Pascal is under a lot of stress and I feel sorry for him. But I don't think that justifies sticking with something that no one can get working.

That's why I ask to consider: Make Maven the default build tool for Eclipse plug-ins and drop support for anything else (Features, Ant scripts, weird property files, p2 repositories, update sites). It should be much more simple to create a p2 repo from a Maven repo than the other way around if someone really can't migrate to Maven.

PS: He who complains cares.

Comment 129 Thomas Hallgren 2010-11-15 09:46:47 EST

(In reply to comment #128)
> ... Stop making the lives of the 20-30 everyday
> committers more easy, start making it easier for *everyone*.

So how do we get there? Could a fast and efficient way be to fix the bugs in Tycho?

Comment 130 Mickael Istria 2010-11-15 10:01:12 EST

Maybe someone already pointed it in one of the 129 previous messages and I missed it, but I'd like to highlight the fact that there are 2 use cases about Maven and Eclipse repositories:

* The first one is to have a standard Maven repository where people can get some Eclipse artifacts that are not Eclipse plugins. We can think of EMF, Jetty and others that can be used standalone outside of Eclipse.
For this use case, there is need for a Maven repository at Eclipse.org, where consumer could get the jars for these projects in the Maven way. Otherwise, these project are responsible of pushing their Maven artifacts to another Maven repository.

* The second use case is about building Eclipse Plug-ins with Tycho. In this case, there is not at all need for a Maven repository containing all Eclipse bundles since Tycho is mainly about consuming p2 repositories to resolve dependencies specified in MANIFEST.MF.

Basically, this bug is only refering to the first use case. Tycho does not change the need to solve (or not) the first use case.
The only thing that Tycho brings to the debate is about naming conventions, but that should be discussed on bug #288644

Comment 131 Ed Merks 2010-11-15 10:08:31 EST

Aaron,

You approach this issue like a bull in a china shop.  Surely you can see that suggesting "drop support for anything else (Features, Ant scripts, weird
property files, p2 repositories, update sites)." as patently ridiculous.  Take note that there are quite a few more than 20-30 committers at Eclipse.  We're all very busy people.  We produce stuff consumed by an uncounted number consumers so of course making life easier for those consumers is fundamentally important, but making life easy for those who are the source of all things ought not to be wiped aside by a wave of a dismissive hand.  

Note too that the purpose of p2 isn't just to help drive builds, i.e., it's not just a development time technology.  It's more fundamentally a runtime provisioning technology that can't just be flushed down the toilet.

Comment 132 Aaron Digulla 2010-11-15 10:30:39 EST

(In reply to comment #131)

>  Surely you can see that
> suggesting "drop support for anything else (Features, Ant scripts, weird
> property files, p2 repositories, update sites)." as patently ridiculous.

I'd prefer the term "consequent". How many people have volunteered to work on Eclipse and gave up because of weird NullPointerExceptions in PDE? Ant just can't produce meaningful error messages by default. How should it know what's wrong when I forget to specify a build property?

A specific build tool like Maven, OTOH, understands the semantics of the various options, it can check them and it can produce a useful error message like "You try to build against linux/gtk/x85. I only know linux/gtk/x86 and linux/gtk/x86_64."

As for p2, that really doesn't belong here anymore. From my point of view, Maven also downloads plugins and builds a runtime classpath just like p2, and it does that fast, efficient and reliably. 'nuff said.

Comment 133 David Williams 2010-11-15 10:43:35 EST

> 
> I'd like to hear how much extra work David and the B3 folks think exposing the
> release train as Maven in addition to p2 will take. I believe it's a relatively
> small amount of work, but please correct me if I'm wrong.
> 

Mechanically, using b3, it is not much work. See bug 312656. But, that still leaves a lot of work to fine tune it, to test it (both the maven artifacts, and the "new" p2 repo structure), and to gain consensus, I left bug 312656 as "who is willing to own this" and so far no one has said "I will". It should be someone who has a vested interest in seeing it (and p2, and b3 aggregator) succeed and can make a long term commitment to supporting the effort.

Comment 134 Thomas Hallgren 2010-11-15 13:15:07 EST

(In reply to comment #130)
> For this use case, there is need for a Maven repository at Eclipse.org, where
> consumer could get the jars for these projects in the Maven way.

Couldn't that be rephrased to:
For this use case, there is a need for a Maven plugin that makes it possible for Maven builds to consume p2 repositories.
?

Comment 135 Mickael Istria 2010-11-15 13:39:29 EST

I'm no(In reply to comment #134)
> (In reply to comment #130)
> > For this use case, there is need for a Maven repository at Eclipse.org, where
> > consumer could get the jars for these projects in the Maven way.
> 
> Couldn't that be rephrased to:
> For this use case, there is a need for a Maven plugin that makes it possible
> for Maven builds to consume p2 repositories.
> ?

I am not sure that people who want to use standalone delivery of EMF, Jetty and so on actually want to consume Eclipse plugins,
A p2 repository fits to OSGi bundles and Eclipse plugins, whereas a Maven repository fits to "traditional" jars.

I don't think making a Maven repository from the Helios repository would be very useful. What would be useful would be the ability for Eclipse projects to ship jars in the Maven way if they want to.

Just imagine that you want to create a GWT webapp with Maven that uses EMF but that you don't want to ship all Equinox. A Maven repo with a standalone EMF would be welcome in that case.

The question is now: which projects want to be shipped in both way (p2 and Maven repo) ?

Comment 136 Wayne Beaton 2010-11-15 13:52:34 EST

(In reply to comment #135)
> I don't think making a Maven repository from the Helios repository would be
> very useful. What would be useful would be the ability for Eclipse projects to
> ship jars in the Maven way if they want to.

This is the sort of feedback I'm looking for. So that's one vote against the "Mavenification" of Helios. Are there other votes?

Eclipse projects can currently ship code in any format they choose. Some projects today ship Maven plug-ins (I think EclipseLink does this). But is that enough? Intuitively (and possibly naively), I expect that some kind of aggregation is required to make it really useful. Is this implied in your statement?
 
> Just imagine that you want to create a GWT webapp with Maven that uses EMF but
> that you don't want to ship all Equinox. A Maven repo with a standalone EMF
> would be welcome in that case.

> The question is now: which projects want to be shipped in both way (p2 and
> Maven repo) ?

It seems to me that I haven't understood this problem well enough. It sounds like it's not enough to just provide Maven metadata for the existing p2 repositories. At least some of the projects will need to ship with fundamentally different configurations (i.e. not OSGi bundles).

Can you paint a very specific picture of what the ideal looks like? How many different repositories are we talking about? What is in each of them? In your perfect scenario (from the consumer's point of view), what does this look like?

We (I) need to have a better grasp of the requirements so that we can sort out what effort is required here.

I've moved past the argument that Maven users can just consume p2 repositories as I believe that this is the sentiment that has been uniformly expressed by the Maven representatives.

Comment 137 Brian Fox 2010-11-15 15:46:49 EST

(In reply to comment #136)
> (In reply to comment #135)
> > I don't think making a Maven repository from the Helios repository would be
> > very useful. What would be useful would be the ability for Eclipse projects to
> > ship jars in the Maven way if they want to.
> 
> This is the sort of feedback I'm looking for. So that's one vote against the
> "Mavenification" of Helios. Are there other votes?
> 
> Eclipse projects can currently ship code in any format they choose. Some
> projects today ship Maven plug-ins (I think EclipseLink does this). But is that
> enough? Intuitively (and possibly naively), I expect that some kind of
> aggregation is required to make it really useful. Is this implied in your
> statement?
>

I don't think mavenizing the entire repository is the right approach.
 
> 
> It seems to me that I haven't understood this problem well enough. It sounds
> like it's not enough to just provide Maven metadata for the existing p2
> repositories. At least some of the projects will need to ship with
> fundamentally different configurations (i.e. not OSGi bundles).
>

I don't think that's a requirement of what is being asked for here. Some tools do consume osgi bundles from Maven repositories, so being in Maven does not imply they aren't osgi bundles.

 
> Can you paint a very specific picture of what the ideal looks like? How many
> different repositories are we talking about? What is in each of them? In your
> perfect scenario (from the consumer's point of view), what does this look like?

You would be talking about one repository that contains everything from Eclipse and is updated as new releases are produced. Maven scopes the dependency resolution based on the data in the pom, not by constraining the contents of the repository, so it's the expected norm that you have a static url to a repo that contains ever increasing sets of artifacts.

> 
> We (I) need to have a better grasp of the requirements so that we can sort out
> what effort is required here.
> 
> I've moved past the argument that Maven users can just consume p2 repositories
> as I believe that this is the sentiment that has been uniformly expressed by
> the Maven representatives.

Maven cannot consume P2 repositories today, and generally this isn't what people are asking for. Tycho is for building Eclipse plugins so that's a solution to a specific problem that is out of scope for this bug.

The bigger problem here is that in order for Maven repos to be truely useful, the pom must contain proper dependency information. I don't think it's very straightforward to convert from the p2 dependency info into the Maven standard for many nuanced reasons.

Comment 138 Aaron Digulla 2010-11-15 16:05:45 EST

(In reply to comment #135)

> I don't think making a Maven repository from the Helios repository would be
> very useful. What would be useful would be the ability for Eclipse projects to
> ship jars in the Maven way if they want to.

"If they want to" sounds ominous to me. This bug is about that people keep asking for a complete, working Maven repository.

> Just imagine that you want to create a GWT webapp with Maven that uses EMF but
> that you don't want to ship all Equinox. A Maven repo with a standalone EMF
> would be welcome in that case.

When I include EMF JARs in my maven projects (see comment #94 how I do that), I don't get anything from Equinox. The JAR might be an OSGi bundle but that doesn't mean I have to use it in such a way. I just ignore the file "plugin.xml" in the archive and it still works.

My woes are missing/wrong dependencies. Most Eclipse plugins say "give me EMF[3.0,4.0)". So when I include CDO (which needs EMF), I can run into problems. Example: On Machine A, I have an versions of EMF+CDO (from Eclipse 3.5). So I say "I want CDO 3.5" and it works.

On Machine B, I have upgraded from 3.5 to 3.6. This means my Maven repository now offers me EMF 3.5 and 3.6 and CDO 3.5 and 3.6. When I build my project on B, something unexpected happens: I will get CDO 3.5 with EMF 3.6 since CDO just says "anything better than 3.0 is OK".

Problems like that are one reason why Maven urges to specify exact dependency versions instead of version ranges. To make this manageable, you can define all the versions as properties (variables) in a central place, so you need to update them only once.

Now we have a problem. I understand that the CDO guys won't be happy when we Maven developers come over and demand that they use specific versions - it just doesn't fit well into the way OSGi/PDE/p2 looks at the world. So that's extra effort for them.

So if you say "projects can opt in", that means the result will be a repository no one uses because everyone who needs a stable build will use eclipe:make-artifacts: It's the only way to get what you need at a time when you need it and where you can fix bugs (like bad version ranges) without waiting a few months for the next Eclipse release.

If you say "Maven is a must for every project", that's a lot of effort because you'll be opening a completely new can of worms. As I just showed, it's not simply "we'll add a maven target to PDE".

Which is why I'm saying: If we really want that, PDE must be replaced by something that creates Eclipse plugins and Maven artifacts by default and you must invest effort to *turn it off*. That way, more people will be able to join Eclipse projects. Project maintainers will have more time because builds will break less often. The overall situation will improve.

But ... someone with a lot of energy and time will have to push the Eclipse community until it starts moving. Because everyone is afraid to change anything in the builds, resistance to change will be high. Unless the new build system is so much better that everyone will *want* to use it.

Which just translates to: someone with a lot of energy and time will have to get Tycho to a point where it can build all of Eclipse.

Comment 139 Aaron Digulla 2010-11-15 16:18:51 EST

(In reply to comment #136)

> We (I) need to have a better grasp of the requirements so that we can sort out
> what effort is required here.

What I usually do:

- Import all Eclipse plugins from an official release into my local repository (at home) or into the Maven proxy at work (for a team). That includes tweaking a couple of dependency versions or replacing dependencies with official releases (for example for stuff that Eclipse takes from Apache).

- I build SWT-based apps without OSGi/Equinox. Maven resolves all my dependencies for me, I have no use for a plug-in system that just slows me down. If a plugin really needs its OSGi startup code called, I get the source, make all the relevant code public and just call the methods in my startup code.

I would prefer if every Eclipse project would just release pure code JARs which are included in thin OSGi bundles but since that's not going to happen, I'm not asking for it.

- When I need to fix bugs or add features, I take existing plugins, unpack them, recompile the classes I need and JAR them up again (without signatures). I use Maven or my own Ant scripts for that.

As you can see, almost nothing that Eclipse offers besides the bare code has much value to me.

Comment 140 Martin Oberhuber 2010-11-15 16:43:39 EST

The version story is interesting, since it sounds like through explicit version dependencies, Maven implicitly builds your stack. 

Assume that 
    A-1.0 depends on foo-1.0 , and
    B-1.5 depends on foo-1.1,
could you be using A-1.0 and B-1.5 together in a system built by Maven?

At Eclipse, A-1.0 and B-1.5 can be used together when their dependency version ranges on foo overlap. The concrete version of foo is determined by the person who builds the product ... or, it can be determined by restricting selection of bundles to only those that are in the same release train.

So it looks like the "on the same release train relationship" adds the explicit version info that's needed for migrating from OSGi to Maven. And as a corollary, it seems like Mavenizing a release train repo may in fact make sense.

Comment 141 Bob Fields 2010-11-15 19:15:04 EST

(In reply to comment #140)
> The version story is interesting, since it sounds like through explicit version
> dependencies, Maven implicitly builds your stack. 

This also fixes the issue with the current helios maven artifacts depending on a version range rather than a specific version, which breaks when using maven 2.x because NO dependencies are selected. Maven2 assumes a three part version, Eclipse uses a four part version. In general, specific dependency versions are HIGHLY encouraged because that makes a build reproducible. Maven has taken this a step further with the maven-enforcer-plugin, which also enforces specific OS/version, JDK/version, and build plugin versions and other environment parameters. On the same release train (even on the same patch level) is much preferred for version dependencies.

Comment 142 Aaron Digulla 2010-11-16 06:05:36 EST

(In reply to comment #140)

> Assume that 
>     A-1.0 depends on foo-1.0 , and
>     B-1.5 depends on foo-1.1,
> could you be using A-1.0 and B-1.5 together in a system built by Maven?

Yes. I'd have to try it to tell you which version gets picked up. If you don't like the default, you can overwrite it at the project C (which depends on A and B).

> At Eclipse, A-1.0 and B-1.5 can be used together when their dependency version
> ranges on foo overlap. The concrete version of foo is determined by the person
> who builds the product ... or, it can be determined by restricting selection of
> bundles to only those that are in the same release train.

Same with Maven, just the implicit rules are different. Which means you *might* get different result when you build the project in Eclipse and with Maven. Danger Will Robinson.

Comment 143 Paul Webster 2010-11-16 07:14:12 EST

(In reply to comment #141)
> This also fixes the issue with the current helios maven artifacts depending on
> a version range rather than a specific version, which breaks when using maven
> 2.x because NO dependencies are selected. Maven2 assumes a three part version,
> Eclipse uses a four part version. In general, specific dependency versions are
> HIGHLY encouraged because that makes a build reproducible. Maven has taken this
> a step further with the maven-enforcer-plugin, which also enforces specific
> OS/version, JDK/version, and build plugin versions and other environment
> parameters. On the same release train (even on the same patch level) is much
> preferred for version dependencies.

Where are the explicit dependencies going to coming from?  When we create a maven repo out of a release train, like Helios?  Is this what Buckminster Aggregator does when it generates a maven repo from a p2 repo?

Or does maven 3 support version ranges and this then becomes a moot point?

PW

Comment 144 Aaron Digulla 2010-11-16 07:42:37 EST

(In reply to comment #143)
> (In reply to comment #141)
> > This also fixes the issue with the current helios maven artifacts depending on
> > a version range rather than a specific version, which breaks when using maven
> > 2.x because NO dependencies are selected. Maven2 assumes a three part version,
> > Eclipse uses a four part version. In general, specific dependency versions are
> > HIGHLY encouraged because that makes a build reproducible. Maven has taken this
> > a step further with the maven-enforcer-plugin, which also enforces specific
> > OS/version, JDK/version, and build plugin versions and other environment
> > parameters. On the same release train (even on the same patch level) is much
> > preferred for version dependencies.
> 
> Where are the explicit dependencies going to coming from?

To clarify: We are talking about explicit dependencies *versions* here. The versions will come from the MANIFEST.MF files that are part of the release train.

So if we include EMF 3.2.1, all plugins that depend on that will get their version range replaced with "3.2.1".

> Or does maven 3 support version ranges and this then becomes a moot point?

All versions of Maven support version ranges. It's just not smart to use them because for any two people, two different software products could be built, depending on their work history. Which is why Maven *encourages* explicit versions over version ranges. If you like/need it, you can still use ranges.

Comment 145 Paul Webster 2010-11-16 08:00:00 EST

(In reply to comment #144)
> 
> To clarify: We are talking about explicit dependencies *versions* here. The
> versions will come from the MANIFEST.MF files that are part of the release
> train.
> 
> So if we include EMF 3.2.1, all plugins that depend on that will get their
> version range replaced with "3.2.1".

But not in the manifest, right?  Just in the generated poms, for the purposes of consuming a maven repo of Helios?

In general, dependencies are expressed as the lowest version number to meet API requirements.  i.e. org.eclipse.ui depends on org.eclipse.core.runtime [3.2.0,4.0.0) as 3.2 was the last time we used new API.  But Indigo will be shipped with 3.6.100

Even in Indigo, things like PDE and JDT will be built against Platform UI 3.7.x and 3.100.x.  The output must be able to run against both, since 3.100.x supports the 3.7.x API, and so users of the 3.7.x API can run against 3.100.x.

PW

Comment 146 Aaron Digulla 2010-11-16 10:38:50 EST

(In reply to comment #145)

> But not in the manifest, right?  Just in the generated poms, for the purposes
> of consuming a maven repo of Helios?



> 
> In general, dependencies are expressed as the lowest version number to meet API
> requirements.  i.e. org.eclipse.ui depends on org.eclipse.core.runtime
> [3.2.0,4.0.0) as 3.2 was the last time we used new API.  But Indigo will be
> shipped with 3.6.100
> 
> Even in Indigo, things like PDE and JDT will be built against Platform UI 3.7.x
> and 3.100.x.  The output must be able to run against both, since 3.100.x
> supports the 3.7.x API, and so users of the 3.7.x API can run against 3.100.x.
> 
> PW

(In reply to comment #145)
> (In reply to comment #144)
> > 
> > To clarify: We are talking about explicit dependencies *versions* here. The
> > versions will come from the MANIFEST.MF files that are part of the release
> > train.
> > 
> > So if we include EMF 3.2.1, all plugins that depend on that will get their
> > version range replaced with "3.2.1".
> 
> But not in the manifest, right?  Just in the generated poms, for the purposes
> of consuming a maven repo of Helios?
> 
> In general, dependencies are expressed as the lowest version number to meet API
> requirements.  i.e. org.eclipse.ui depends on org.eclipse.core.runtime
> [3.2.0,4.0.0) as 3.2 was the last time we used new API.  But Indigo will be
> shipped with 3.6.100
> 
> Even in Indigo, things like PDE and JDT will be built against Platform UI 3.7.x
> and 3.100.x.  The output must be able to run against both, since 3.100.x
> supports the 3.7.x API, and so users of the 3.7.x API can run against 3.100.x.
> 
> PW

(In reply to comment #145)
> (In reply to comment #144)

> > To clarify: We are talking about explicit dependencies *versions* here. The
> > versions will come from the MANIFEST.MF files that are part of the release
> > train.
> > 
> > So if we include EMF 3.2.1, all plugins that depend on that will get their
> > version range replaced with "3.2.1".
> 
> But not in the manifest, right?  Just in the generated poms, for the purposes
> of consuming a maven repo of Helios?
> 
> In general, dependencies are expressed as the lowest version number to meet API
> requirements.  i.e. org.eclipse.ui depends on org.eclipse.core.runtime
> [3.2.0,4.0.0) as 3.2 was the last time we used new API.  But Indigo will be
> shipped with 3.6.100
> 
> Even in Indigo, things like PDE and JDT will be built against Platform UI 3.7.x
> and 3.100.x.  The output must be able to run against both, since 3.100.x
> supports the 3.7.x API, and so users of the 3.7.x API can run against 3.100.x.

When I write a project, I don't care whether the API has changed or not. My "version filter" is based on bug fixes. So I set the versions of my dependencies to those values where I know all bugs I care about have been fixed.

So even though my code would probably work against 1.3.1, I'm still requesting 1.3.7.

Comment 147 Paul Webster 2010-11-16 10:49:48 EST

(In reply to comment #146)
> (In reply to comment #145)
> 
> > But not in the manifest, right?  Just in the generated poms, for the purposes
> > of consuming a maven repo of Helios?
> 
> 
> 
> > 
> > In general, dependencies are expressed as the lowest version number to meet API

It looked like there was a statement missing here.  please re-post (if that's true, if not, don't worry :-)

PW

Comment 148 Pascal Rapicault 2010-11-16 15:55:14 EST

In reply to comment #128
>The next step is to have a good look at p2 and figure out why Maven can resolve dependencies in a few milliseconds when p2 needs several minutes for the same task. If no one can get the SAT solver right, well, then the SAT solver is not the solution. Yes, Pascal is under a lot of stress and I feel sorry for him. But I don't think that justifies sticking with something that no one can get working.

   I'm not sure where you are getting those facts from, but the SAT solver just works and it performs fine (actually very very fine). In fact, should there have ever been any performance problem, the approach would have not made it in Eclipse at all. The perception you are getting by looking at the UI is nowhere near the reality of the resolution time.
   p2 and Maven are based on two very different models both in terms of what the metadata allows to describe and in terms of how repositories are structured. This explain why the mapping is not trivial to go from p2 (or OSGi for that matter) to Maven, and also the perception of p2 slowness. FWIW, my attempts at mapping maven metadata to p2 one and then resolving using SAT (using an approximation of the Maven algorithm) just blow up maven resolver, and on top of that provides a clear semantics.
  One last point on performance, when you run a tycho build, for every module in the build, tycho uses the p2 resolver to figure out what bundles need to be added.

Comment 149 Paul Webster 2010-11-17 07:51:28 EST

(In reply to comment #146)
> 
> When I write a project, I don't care whether the API has changed or not. My
> "version filter" is based on bug fixes. So I set the versions of my
> dependencies to those values where I know all bugs I care about have been
> fixed.
> 
> So even though my code would probably work against 1.3.1, I'm still requesting
> 1.3.7.

Right, but my question was about fixing the version in the MANIFEST.MF (which we can't do) or in the generated poms (which we could do, but I don't understand the implications).

If we're publising a maven repo, it seems based on this bug report there would be 2 kinds of consumers.  Ones that wanted to use the software in a non-OSGi environment (SWT, JFace, core commands, parts of EMF, the extension registry, they can all be used without OSGi).  The other consumers want the OSGi bundles, as they build OSGi apps with maven.

Fixing the versions in the pom seems like it would satisfy both usecases, no?

PW

Comment 150 Alex Blewitt 2010-11-17 08:15:29 EST

(In reply to comment #149)
> 
> Right, but my question was about fixing the version in the MANIFEST.MF (which
> we can't do) or in the generated poms (which we could do, but I don't
> understand the implications).

Yes, fix in the pom but leave as is for the manifest.

> If we're publising a maven repo, it seems based on this bug report there would
> be 2 kinds of consumers.  Ones that wanted to use the software in a non-OSGi
> environment (SWT, JFace, core commands, parts of EMF, the extension registry,
> they can all be used without OSGi).  The other consumers want the OSGi bundles,
> as they build OSGi apps with maven.
> 
> Fixing the versions in the pom seems like it would satisfy both usecases, no?

Yes, it would. In fact, for consistency's sake, I would suggest that the dependency from A->B if both are in the same (Helios/Indigo) P2 repository, that you freeze the version of B at that time into A's dependency list. It means, when you request A, you'll get the same B version that's in Helios/Indigo together.

e.g. for ECF in Helios:

<groupId>org.eclipse.ecf</groupId>
<artifactId>org.eclipse.ecf.sync</artifactId>
<version>2.0.0</version>
...
<dependencies>
<dependency>
<groupId>org.eclipse.core</groupId>
<artifactId>org.eclipse.core.runtime</artifactId>
<version>3.6.0</version>
</dependency>
<!-- if we also had an optional dependency on JFace Text -->
<dependency>
<groupId>org.eclipse.jface</groupId>
<artifactId>org.eclipse.jface.text</artifactId>
<version>3.6.1</version>
<optional>true</optional>
</dependency>
...
</dependencies>

Even though the import is unversioned in the ECF Sync package (Bundle-ImportPackage: org.eclipse.core.runtime) it's likely only been tested against the Helios version of the bundle. (If multiple bundles export the org.eclipse.core.runtime package, we might need to add all of them here)

For optional imports, if a bundle (or bundle's packages) are solely optional := true, then we can annotate it as above.

Comment 151 Aaron Digulla 2010-11-17 15:50:16 EST

(In reply to comment #148)

>   One last point on performance, when you run a tycho build, for every module
> in the build, tycho uses the p2 resolver to figure out what bundles need to be
> added.

And every time I see "Adding repository http://", the build hangs for 10s to 10 minutes.

As I understand, downloading stuff from slow network connections without proper caching is a fundamental design of p2. But I still don't like to wait for a couple of minutes when the install dialog hangs or half an hour if p2 gets lost somewhere and can't find a solution.

So maybe the code is fast but the overall performance experience as a user is very bad.

I know that I can get that better by disabling all update sites in Eclipse but I prefer the Maven approach: Do the right thing even if there are a dozen online repositories registered in the POM.

Comment 152 Aaron Digulla 2010-11-17 15:59:48 EST

(In reply to comment #150)

> > Right, but my question was about fixing the version in the MANIFEST.MF (which
> > we can't do) or in the generated poms (which we could do, but I don't
> > understand the implications).
> 
> Yes, fix in the pom but leave as is for the manifest.

Note: Maven allows to override the version in your project. So even if B asks for A:1.3.7, C can say "I want B:1.1 and A:1.3.1".

Note2: I've ironed out most of my problems with Tycho (took me about 15min to find the bug which caused the build to fail and then 30min to write a small patch to support jars.extra.classpath). Which means I can now build some parts of BIRT.

Anyone with me to create POMs for all of Eclipse? My final goal is a single script or Java program which allows you to setup a workspace where you can open any part of Eclipse and build patches in a consistent way.

If you want to get started with Tycho, start with this page: https://docs.sonatype.org/display/TYCHO/Tycho+reference+card

The important parts are under "Examplary parent POM" and "Examplary bundle/fragment POM". Note that you don't need a build.xml, just the plugin.xml and the build.properties and META-INF/MANIFEST.MF.

Comment 153 Paul Webster 2010-11-18 10:43:28 EST

(In reply to comment #150)
> Yes, fix in the pom but leave as is for the manifest.
> 

I ran a quick test using the b3 aggregator to generate a maven repo for Helios.  If we were going to publish helios, could we just host the converted repo on download.eclipse.org for a milestone or two and get feedback on what works and what doesn't work?

- The default behavior for b3 is a group id of org.eclipse.<segment> ... that seems reasonable.  

- It generates ranges in the dependencies, it's true, but that wouldn't prevent using the repo for now (since it's only one slice of eclipse releases).  Maybe an enhancement to b3 could write out the specific versions for the dependencies to what's in that slice during pom generation.

Some things I saw that might cause problems:

- I didn't notice anything platform-filter like generated for the SWT fragments.  Not sure what information would need to be there for someone to build a windows SWT app vs a linux SWT app in a more general way than directly including the artifact id for the fragment itself (which worked for a non-OSGi JFace test app)

- the dependencies don't seem to include anything (like the derived bundle) from an Import-Package

- most of helios was generated with the new "Eclipse-SourceReferences: scm:cvs:..." header ... is that something that can be surfaced in poms?

PW

Comment 154 Aaron Digulla 2010-11-18 11:30:22 EST

(In reply to comment #153)
> (In reply to comment #150)
> > Yes, fix in the pom but leave as is for the manifest.
> > 
> 
> I ran a quick test using the b3 aggregator to generate a maven repo for Helios.
>  If we were going to publish helios, could we just host the converted repo on
> download.eclipse.org for a milestone or two and get feedback on what works and
> what doesn't work?

Sounds like a plan :-)

> - The default behavior for b3 is a group id of org.eclipse.<segment> ... that
> seems reasonable.  

Good.

What does the artifactId look like?

> - It generates ranges in the dependencies, it's true, but that wouldn't prevent
> using the repo for now (since it's only one slice of eclipse releases).  Maybe
> an enhancement to b3 could write out the specific versions for the dependencies
> to what's in that slice during pom generation.

I see no problem here as well. If it becomes one for a project, they can always override the range in their own POM or override the range and file a bug report asking to adjust the version range.

For example for cases like "...org.eclipse.birt.core;bundle-version=1.0.0" when there never was such a version.

> Some things I saw that might cause problems:
> 
> - I didn't notice anything platform-filter like generated for the SWT
> fragments.  Not sure what information would need to be there for someone to
> build a windows SWT app vs a linux SWT app in a more general way than directly
> including the artifact id for the fragment itself (which worked for a non-OSGi
> JFace test app)

If anyone is interested, I have a POM fragment which automatically gets the right version of SWT for your target hardware. It also allows to include all versions of SWT so you can do some classloader magic to "see" only the right one. Bloats the result by 20MB but makes the same download usable for everyone.

> - the dependencies don't seem to include anything (like the derived bundle)
> from an Import-Package

Can't comment on that.

> - most of helios was generated with the new "Eclipse-SourceReferences:
> scm:cvs:..." header ... is that something that can be surfaced in poms?

Yes, see http://maven.apache.org/pom.html#SCM

Comment 155 Martin Oberhuber 2010-11-18 11:38:03 EST

(In reply to comment #153)
> I ran a quick test using the b3 aggregator to generate a maven repo for
> Helios.

Thanks so much, Paul, for taking action and moving forward!

It would be great if you could share this on some public "staging" site for the Maven community to inspect (and likely come up with more things to be done). Also, since there seems to be some consensus that we're mostly interested in the Release Train for now, we might continue this specific discussion on 
bug 312656 (which only has 35 comments at the moment).

Comment 156 Bob Fields 2010-11-18 14:23:11 EST

(In reply to comment #155)
> It would be great if you could share this on some public "staging" site for the
> Maven community to inspect (and likely come up with more things to be done).
> Also, since there seems to be some consensus that we're mostly interested in
> the Release Train for now, we might continue this specific discussion on 
> bug 312656 (which only has 35 comments at the moment).

And also test against the very simple maven project test case I uploaded to https://bugs.eclipse.org/bugs/show_bug.cgi?id=312656 at https://bugs.eclipse.org/bugs/attachment.cgi?id=174984, which demonstrates the dependency version range failure when referencing the published artifacts with four part version in the other helios maven repository. You'll have to change the repository location in the pom.xml file. You can also test using maven 3.0 to see if it works with the new repository.

Comment 157 Scott Lewis 2010-11-18 14:55:55 EST

FYI: ECF has started to create maven repos as part of our own build:

http://dev.eclipse.org/mhonarc/lists/ecf-dev/msg04544.html

Obviously this is just for our community, and we're not completely sure whether we can sustain and/or support it going forward...so in my view the results of this bug are still of high relevance and importance to our community.

Comment 158 Brian Fox 2010-11-18 15:24:08 EST

(In reply to comment #157)
> FYI: ECF has started to create maven repos as part of our own build:
> 
> http://dev.eclipse.org/mhonarc/lists/ecf-dev/msg04544.html
> 
> Obviously this is just for our community, and we're not completely sure whether
> we can sustain and/or support it going forward...so in my view the results of
> this bug are still of high relevance and importance to our community.

Scott, it looks like this repo contains all the dependencies used in your build, but many of them are duplicates of artifacts on Central with altered coordinates. This might work ok for some limited use, but it would be quite a mess for a maven user to consume a single ECF jar and then have to manage hundreds of exclusions to get the correct tree in the context of dependencies from Central. This is equivalent to the mess we had to sort out in the Jboss repos because people essentially refused to use it.

The ideal repo would publish only those things not already in Central, and refer to external dependencies using the original coordinates to ensure conflict detection works properly for the users. I'm not saying this is easy, but it's an example of why automated conversion from p2 to Maven is tricky.

Comment 159 Martin Oberhuber 2010-11-18 15:40:06 EST

Bob, this sounds like a conceptual problem. All Eclipse.org builds consume 3rd party libs from Eclipse/Orbit, which performs OSGi bundling among other things. Some of the OSGi bundling work is nontrivial.

While I see that plain Java / library consumers prefer consuming from Central, it looks like replacing the Orbit dependencies with Maven Central dependencies could break a lot in the OSGi world. I've seen enough subtle discussions about re-exporting, multiple implementations... (eg in the areas of slf4j, commons logging, javax.mail/glassfish/gernoimo...) to know it's not easy.

Can we even have a single Maven Repo that caters to both OSGi and plain Java consumers?

Comment 160 Barrie Treloar 2010-11-18 20:02:51 EST

(In reply to comment #159)
> All Eclipse.org builds consume 3rd party libs from Eclipse/Orbit, 
> which performs OSGi bundling among other things.
> Some of the OSGi bundling work is nontrivial.
[del]
I've seen enough subtle discussions about
> re-exporting, multiple implementations... (eg in the areas of slf4j, commons
> logging, javax.mail/glassfish/gernoimo...) to know it's not easy.
> 
> Can we even have a single Maven Repo that caters to both OSGi and plain Java
> consumers?

I think this re-exporting is what Brian refers to with the JBoss repo, I know Spring and others project have done similar work and re-exported artifacts.

Ideally the original project should be educated (and convinced) to include the OSGi support as part of their release process.  Having non-related third parties enhance the jars with additional information is what is causing the pain (i.e duplicate classes with duplicate dependencies but different artifact coordindates)

Comment 161 Aaron Digulla 2010-11-22 14:37:53 EST

(In reply to comment #159)
> Bob, this sounds like a conceptual problem. All Eclipse.org builds consume 3rd
> party libs from Eclipse/Orbit, which performs OSGi bundling among other things.
> Some of the OSGi bundling work is nontrivial.

There is a pretty simple solution but it needs some good-will from the maintainers.

Maven supports "profiles", that is you can put part of your POM (project object model) into an "#ifdef".

Project maintainers will have to add the official Maven central name to MANIFEST.MF. So if you provide an OSGi bundle for slf4j, you add this line to MANIFEST.MF:

Maven-Dependencies: org.slf4j:slf4j-api:1.6.1

This tells the POM builder what the name of the dependency is on Maven Central.

Let's say you have a bundle org.eclipse.something which needs the slf4j bundle.

When the Maven repo is generated for org.eclipse.something, you will generate a POM that contains two profiles: "use-osgi-bundles" and "use-maven-central". In the former, you create a dependency to the OSGi bundle, in the latter, you create Maven dependencies from the "Maven-Dependencies" line in your manifest file.

Consumers of your bundle will only need to select the right profile and their build will work as expected. And if I want to do my own thing, I can ignore the profiles and add my own dependencies.

Comment 162 Martin Oberhuber 2010-11-22 15:35:42 EST

(In reply to comment #161)
Interesting. Is the proposed MANIFEST.MF markup somehow "standard" ? Does this work with Nexus/Tycho/b3 today, or is this a suggestion for how a MANIFEST.MF -> POM translation could work in the future?

When I get you right, your comment applies mostly to Orbit which bundles 3rd party Jars. That is, as the Orbit maintainer of Apache Commons Net 2.0
I'd add the 

   Maven-Dependencies: commons-net:commons-net:2.0

into my MANIFEST.MF ? That sounds more like the Maven Central Alias of my Orbit lib, than a dependency though...

Comment 163 Aaron Digulla 2010-11-23 03:24:10 EST

(In reply to comment #162)

> (In reply to comment #161)
> Interesting. Is the proposed MANIFEST.MF markup somehow "standard" ? Does this
> work with Nexus/Tycho/b3 today, or is this a suggestion for how a MANIFEST.MF
> -> POM translation could work in the future?

No. It's just an idea today.

> When I get you right, your comment applies mostly to Orbit which bundles 3rd
> party Jars. That is, as the Orbit maintainer of Apache Commons Net 2.0
> I'd add the 
> 
>    Maven-Dependencies: commons-net:commons-net:2.0
> 
> into my MANIFEST.MF ? That sounds more like the Maven Central Alias of my Orbit
> lib, than a dependency though...

You're right. This would be more to the point:

   Maven-Central-Alias: commons-net:commons-net:2.0

Is it possible to bundle more than a single JAR in a plug-in? If so, would it be possible to make it a strict rule that each bundle must map to one plug-in?

Otherwise, we'd need "Maven-Central-Alias*es*". For some reason, I feel uneasy about this.

Comment 164 Martin Oberhuber 2010-11-23 04:58:04 EST

Bundles do support nested Jars (take the org.apache.ant bundle in Eclipse SDK for example, it has 24 nested Jars - each of which has its own maven entry: http://repo2.maven.org/maven2/ant ).

The Bundle-ClassPath enumerates nested jars.

Comment 165 Aaron Digulla 2010-11-23 07:44:00 EST

(In reply to comment #164)
> Bundles do support nested Jars (take the org.apache.ant bundle in Eclipse SDK
> for example, it has 24 nested Jars - each of which has its own maven entry:
> http://repo2.maven.org/maven2/ant ).
> 
> The Bundle-ClassPath enumerates nested jars.

Then Maven-Central-Aliases it should be unless we can convince all such bundle maintainers to split their work up -- anyone feeling that might happen?

As I see it, chances are much better when I send a patch with "please add this to your MANIFEST.MF" to get this through than when I ask "can you please split your bundle in 24 bundles?"

Comment 166 Martin Oberhuber 2010-11-23 09:23:41 EST

Yes, likely so... at any rate, if you want to pursue the Maven-Central-Aliases idea, please open a new bug for it since it's not directly related to this discussion.

Comment 167 Chris Aniszczyk 2011-02-10 14:22:44 EST

At the last Architecture Council meeting (Feb 10th, 2011), we discussed this issue again. While we didn't reach a magical solution to this problem, we concluded that it would be beneficial for projects who are currently shipping maven artifacts discuss how they do it (along with what drawbacks there currently are). This should help other projects looking to do the same.

As far as I know EclipseLink (Shaun Smith) and Jetty (Jesse McConnell) are the two projects at eclipse.org currently shipping maven artifacts as part of their release. Would you guys care to share any nuggets of wisdom here?

On a side note, I think this is less of an issue now as with the advent of Maven Tycho... it's pretty easy to consume Eclipse artifacts as part of your Maven build. When this bug was first open, Tycho was in its infancy... it's matured quite a bit and is now used many community members. There are still those people out there that would require a maven only repository, but I still think that is a small cross-section of the community.

Comment 168 Alex Blewitt 2011-02-10 15:00:33 EST

(In reply to comment #167)
> On a side note, I think this is less of an issue now as with the advent of
> Maven Tycho... it's pretty easy to consume Eclipse artifacts as part of your
> Maven build.

This misses the point by a country mile. Tycho is a replacement for PDE build, not a replacement for maven. 

At the moment, the Eclipse artifacts all hid behind a P2 wall which is only of use to the handful of people building Eclipse RCP apps who need it. Normal maven clients cannot access this, so in effect everything built at Eclipse is invisible to those who just use maven. 

We are migrating away from EclipseLink to OpenJPA not because of technical superiority but because the EclipseLink maven repo (which I applaud them for trying) is basically useless when thrown behind the Eclipse throw-it-over-the-wall mirror redirector. Half of the sites don't mirror the full content, and a number return some HTML garbled page which in no way represents a valid JAR. if there was a stable, reliable maven repo at Eclipse then that would not be an issue. 

Then there are other small projects - like ECF - who arguable have much superior implementations but which are invisible thanks to hiding behind P2. Those using Maven - and a nod to Ivy - can't consume these at all. 

All of this means that even though there are valid non-Eclipse (and hey, non OSGi too) consumers they end up walking away from the Eclipse implementations or upload bodged versions to Maven central. 

Tycho only helps those already using P2/OSGi.

Comment 169 Markus Kuppe 2011-02-10 15:04:30 EST

(In reply to comment #168)
 
> Then there are other small projects - like ECF - who arguable have much
> superior implementations but which are invisible thanks to hiding behind P2.

ECF 3.4 is available at http://download.ecf-project.org/maven

Comment 170 luke w patterson 2011-02-10 15:05:57 EST

(In reply to comment #168)
> All of this means that even though there are valid non-Eclipse (and hey, non
> OSGi too) consumers they end up walking away from the Eclipse implementations
> or upload bodged versions to Maven central. 

Yes!  Someone gets it!

Comment 171 Jesse McConnell 2011-02-10 15:08:30 EST

(In reply to comment #167)

> Would you guys care to share any nuggets of wisdom here?

I'll just explain what we do.  

We develop jetty trunk using maven2 or maven3 interchangeably and have no usage
of tycho.  It is as standard a maven project as you can get, everything
declared pom first as god intended.  We use the bundle-maven-plugin to wire
these artifacts up to be osgi bundles as well, suitable for use within felix or
eclipse.  We use the maven release plugin to release trunk periodically,
staging the artifacts to sonatype's oss nexus setup where we review the
artifacts and distribution (which is what one slice of typical jetty user
consumes) and when we like it we release it.  Upon release of that staging
repository it syncs to maven central and it available for the world to use. 
Many of our users use maven as well and address an individual artifact they
want to use in their build like jetty-server and life continues on.  Other
users of felix grab the artifacts they want and they start as bundles.  

Simultaneously to the eclipse release we build a distribution and release
modules from The Codehaus for jetty hightide which contains integrations with
things that we wouldn't want to attempt to CQ on the eclipse side.  Terracotta
integrations, jboss, whatever the case maybe.

The primary pain point here for me is that the eclipse downloadable
distribution we release _has_ to use artifacts from orbit to be compliant with
eclipse ip processes and all but S- orbit repositories are transitory, today
they are here, tomorrow they are gone.  As a consequence we have had to take
the artifacts we need and put them on our download site so our build has access
to them.  Its a hack but I can go back in time and rebuild on a release tag and
things work again.  On the hightide downloadable distribution we consume those
same dependencies from maven central so don't suffer that issue.

As a separate process to the normal release process we leverage tycho in a
build that produces a p2 repository.  This process takes a few minutes and is
run on hudson.  I wrote a maven plugin that processes that p2 repository and
handles the pack, repack and sign business to produce a valid eclipse p2 repo. 
This repo is published to the jetty download location as a composite repository
and is used by other builds that Hugues generally handles for things like the
RT Runtime projects and the various things like helios and indigo.  I manually
edit the composite repo files to bring in each new release.

All of this lets us basically support download and run jetty users, embedded
jetty users and osgi jetty users of both the felix and equinox variety.

Comment 172 Shaun Smith 2011-02-10 15:08:49 EST

(In reply to comment #167)
> As far as I know EclipseLink (Shaun Smith) and Jetty (Jesse McConnell) are the
> two projects at eclipse.org currently shipping maven artifacts as part of their
> release. Would you guys care to share any nuggets of wisdom here?

We do provide a Maven repo for EclipseLink which was setup to support the
GlassFish team who build with Maven.  I don't know much about it but I'll ask
our build engineer Eric Gwin to comment on his experiences.

Comment 173 Shaun Smith 2011-02-10 15:14:31 EST

(In reply to comment #168)
> We are migrating away from EclipseLink to OpenJPA not because of technical
> superiority but because the EclipseLink maven repo (which I applaud them for
> trying) is basically useless when thrown behind the Eclipse
> throw-it-over-the-wall mirror redirector. Half of the sites don't mirror the
> full content, and a number return some HTML garbled page which in no way
> represents a valid JAR. if there was a stable, reliable maven repo at Eclipse
> then that would not be an issue. 

This is the first I've heard about this.  But it must be a really big issue if you're willing to sacrifice EclipseLink's advanced features and superior performance on your application due to a build issue! ;-)

Did you raise this issue with the EclipseLink team?  A quick search didn't turn anything up.

Comment 174 Denis Roy 2011-02-10 15:18:59 EST

(In reply to comment #168)
> useless when thrown behind the Eclipse
> throw-it-over-the-wall mirror redirector.

I'm open to ideas (via another bug) as long as it's not "let download.eclipse.org serve everything" since we don't have unlimited bandwidth to serve the world.

Comment 175 Jesse McConnell 2011-02-10 15:24:06 EST

the fundamental issue I see in this thread is the reconciling of the transitive dependencies between the orbit modified forms and the original artifacts in maven central itself...

a maven addressed dependency from a transformed p2 repo would point to a different copy of say..commons-foo then the typical one in maven central and that would be a pita to handle with excludes and all that jazz.

various solutions have been presented in this issue, nothing embraced (from what I have seen)

and yes, tycho gaining steam doesn't solve the fundamental issues in reconciling these two different repository systems...imo at least, your need to be either dependency 'closed' within eclipse or within maven central...bridging the two like we do in jetty is a cause for heartburn

Comment 176 Rafael Chaves 2011-02-10 15:42:12 EST

One thing to keep in mind is that the majority of components developed by Eclipse.org projects cannot be consumed by non-Eclipse (even if OSGi-based) applications, they just are not developed with that audience in mind, either because the component is heavily integrated into Eclipse, or just because care was no taking to avoid tying the code to Eclipse-specific API.

So one thing that needs to be addressed is whether Eclipse projects want to be usable in other (non-Eclipse) contexts. Changing the code to be functional outside of Eclipse can be a hard task, and may be even impossible if components consumed by the project do not run outside of Eclipse (and their respective committers are not interested in the move). IMO, unless we see an Eclipse.org-wide initiative of considering non-Eclipse/non-OSGi applications as first class consumers (where applicable), what is being requested in this issue may be a futile effort.

Comment 177 David Carver 2011-02-10 16:02:35 EST

The PsychoPath XPath 2.0 processor in wtp source editing actually has no OSGI dependencies at all.  It is built with tycho, but could easily be deployed to a maven repository.

Some of this issue could be eliminated for those that want to release maven artificats if Eclipse hosted a Nexus repository.   The Apache Foundation does this and it satisfies the needs of those projects that need to make a maven accessible version of their artifacts.

As we go forward, P2 is going to work for some situations, but a project needs to address it's communities needs and more and more runtime related items may not necessarily need P2.

Anyways, count PsychoPath in as wanting or needing a way to make SNAPSHOTS and releases available as Maven artifacts.

Comment 178 Jesse McConnell 2011-02-10 16:16:51 EST

backing up Dave on this...I would also like to see all in Orbit put into nexus @ eclipse...

and heck, I would take the forced requirement to have people do excludes and whatnot in some form just to have a maven repo and get this ball moving, that detail can be hammered out down the road.

Comment 179 Pascal Rapicault 2011-02-10 16:28:53 EST

This discussion has suffered from a lack of scope. 
It looks like there is a rally around making available in a Maven repo only the artifacts that can be run without OSGi (e.g. SWT, JFace, Jetty, etc) since for the rest ppl can use Tycho.
Can we agree on this scope so the task looks less daunting?

Comment 180 Jesse McConnell 2011-02-10 16:31:48 EST

(In reply to comment #179)

+1 yes please

Comment 181 Alex Blewitt 2011-02-10 17:16:19 EST

(In reply to comment #179)
> This discussion has suffered from a lack of scope. 
> It looks like there is a rally around making available in a Maven repo only the
> artifacts that can be run without OSGi (e.g. SWT, JFace, Jetty, etc) since for
> the rest ppl can use Tycho.

I have no idea where you get that idea from, nor should it be a goal to do so. Why should I not be able to get Equinox's DS implementation to consume by a non-P2 client? Or Equinox itself? Just because Eclipse is built on OSGi and Eclipse uses P2, it does not follow that all OSGi clients use P2. Paremus service fabric uses OSGi heavily and only uses OBR; why should you prevent it using the JDT baseline APIs?

There are some components that cannot be used outside of Eclipse, like EMF. However tarring all bundles with the same brush is neither useful nor helpful.

Comment 182 Jeff McAffer 2011-02-10 18:28:15 EST

Just 'cause I hate to miss a party...

I agree completely with Pascal that this discussion is all over the map.  As a relatively outsider (not consuming or producing Maven stuff) here is what I see being discussed.

- Orbit repos are not durable

- Some mirrors are flakey/incomplete/wrong/...

- What parts of Eclipse make sense to consume with maven

- Assumptions of using tycho == using maven == using pde build == using p2 == ...

- How/whether to make available a Maven repo at Eclipse (vs Maven Central or whatever)

I'm merely pointing out that there are at least a handful of discussions going on in parallel in this one bug.  Progress will only be made if some focus can be had.

FWIW I do agree that this is an interesting problem to solve.  As there is more and more Runtime related stuff coming out of Eclipse there are more and more "non-traditionally Eclipse" people consuming (for use with or without OSGi/Equinox).  Many of them would like to see things via Maven.

Comment 183 Bob Fields 2011-02-10 18:43:06 EST

(In reply to comment #181)
> (In reply to comment #179)
> > This discussion has suffered from a lack of scope. 
> > It looks like there is a rally around making available in a Maven repo only the
> > artifacts that can be run without OSGi (e.g. SWT, JFace, Jetty, etc) since for
> > the rest ppl can use Tycho.
> 
> I have no idea where you get that idea from, nor should it be a goal to do so.
> Why should I not be able to get Equinox's DS implementation to consume by a
> non-P2 client? Or Equinox itself? Just because Eclipse is built on OSGi and
> Eclipse uses P2, it does not follow that all OSGi clients use P2. Paremus
> service fabric uses OSGi heavily and only uses OBR; why should you prevent it
> using the JDT baseline APIs?
> 
> There are some components that cannot be used outside of Eclipse, like EMF.
> However tarring all bundles with the same brush is neither useful nor helpful.

I consume EMF artifacts every day in a maven-only, non OSGI environment using a non-Eclipse open source project. www.andromda.org. I was the one that manually rebuilt and updated the 3.1.0 EMF/UML2 and related dependencies in maven Central, because the last time they were provided as a maven dependency in any public repository was four+ years ago.

Is there a public Tycho repository URL that I can point to use as an external maven repository for my builds? I'm using maven 2.2.1, will probably be for a while due to maven plugin upgrade issues. It sounds like the answer is still no.

Comment 184 Pascal Rapicault 2011-02-10 19:05:29 EST

> Is there a public Tycho repository URL that I can point to use as an external
> maven repository for my builds? I'm using maven 2.2.1, will probably be for a
> while due to maven plugin upgrade issues. It sounds like the answer is still
> no.
   Bob, the answer will still be no if we can't focus our efforts on a subset of the problem (see my call in comment #179). On top of that, it may actually make sense for you to get in touch with the EMF team to identify the things that should be published and help them with creating the poms.

Comment 185 Alex Blewitt 2011-02-10 19:25:59 EST

(In reply to comment #184)
>  Bob, the answer will still be no if we can't focus our efforts on a subset
> of the problem (see my call in comment #179). On top of that, it may actually
> make sense for you to get in touch with the EMF team to identify the things
> that should be published and help them with creating the poms.

Pascal, this bug is not about 'who should be in the repo' - that's for another bug. This is about providing a Maven repository for those that want in.

We already have EclipseLink publishing to a repo-ette:

http://www.eclipse.org/downloads/download.php?r=1&nf=1&file=/rt/eclipselink/maven.repo

and ECF:

http://download.ecf-project.org/maven

and Jetty, via Maven central:

http://repo2.maven.org/maven2/org/eclipse/jetty/

The problem is none of these have Eclipse branding or the ability for Eclipse projects to share the same resource. Ultimately, if a standard location were made available then it might even filter up into Maven central, which the Sonatype guys manage, which would be great for everyone (and alleviate the bandwith concerns).

What I think it needs is an agreement to provide a maven repository and then leave it up to individual teams (and/or separate bugs) to come on board. But I don't see that we need to convince the world in this bug, just set the infra in place to make it happen.

I'd suggest following Apache's model; they have https://repository.apache.org/ and use that for both releases (https://repository.apache.org/content/repositories/releases/) and snapshots/work-in-progress (https://repository.apache.org/content/repositories/snapshots/). It's managed with a Sonatype Nexus instance (and it's the professional version, with P2 and OBR support).

We could have repository.eclipse.org set up running a Nexus OSS instance (or a Pro instance if the Sonatype guys wanted to donate a license to Eclipse.org) and then enable those using a Maven repository to publish to that site. Get that content feeding into Maven Central, and then Maven Central's regional locations can provide the caching.

If you went down the Nexus route, then you could have another repository flavour which hosted the orbit stuff as well.

If we can get the infrastructure in place, the projects that want to provide a consistent user experience can have a central place at Eclipse to get their content from, much like they can with http://repositories.apache.org and the Apache branding. As time goes on, there may be other bugs presented which aid in other projects getting POM'd up, or generated via Buckminster, or whatever; but let's get the repo set up and in use by those that need it at the moment and leave other issues to separate bugs.

(I mentioned the issues with the mirrors as an example of why the current eclipseLink stuff doesn't work, which would be fixed by having a standard known place for it; I will raise separate bugs related to the redirection and mirroring elsewhere because the details do not belong on this bug)

Comment 186 Pascal Rapicault 2011-02-10 19:28:14 EST

(In reply to comment #181)
> (In reply to comment #179)
> > This discussion has suffered from a lack of scope. 
> > It looks like there is a rally around making available in a Maven repo only the
> > artifacts that can be run without OSGi (e.g. SWT, JFace, Jetty, etc) since for
> > the rest ppl can use Tycho.
> 
> I have no idea where you get that idea from, nor should it be a goal to do so.
> Why should I not be able to get Equinox's DS implementation to consume by a
> non-P2 client? Or Equinox itself? Just because Eclipse is built on OSGi and
> Eclipse uses P2, it does not follow that all OSGi clients use P2. Paremus
> service fabric uses OSGi heavily and only uses OBR; why should you prevent it
> using the JDT baseline APIs?
   I think you are confusing the issues. Tycho is an extension to Maven. It
allows one to build with a manifest first approach or a pom first approach
bundles and features. The most commonly used mode around here is manifest first
approach and in this mode it resolves its dependencies using p2 repositories
(much like maven download stuffs from Maven repos). In the end it produces jars
with manifests (but can also provide p2 repos) with which you can do absolutely
what you want, zip them and send them to your friend, create an OBR repo for
Fabric, etc.. Also those bundles can be used with Felix, with Concierge, you
name it. The requirement that tycho has on p2 repo is no different than the
requirement that Maven has on Maven repos. The repo is just an exchange medium.
  By the way, we are running a tycho tutorial and a tycho conversion workshop
at eclipsecon 2011.

  That aside, I understand that it would be neat to, and we need to, and we
must do, but the real question is who is going to do the work? Are you? 

   I want to be clear, I'm not against solving the issues but again we are not
making progress because the requirements are all over and because nobody who
understands the problem well enough has the bandwidth to take on it. In fact in
December the Arch Council had decided to close this bug as Wontfix to cause
some reaction.
Sonatype has offered to provide a free a copy of Nexus Pro to the Eclipse
Foundation, the Eclipse Foundation is ok to run a server, what is needed is
VOLUNTEERS TO HELP THE INDIVIDUAL PROJECTS create the poms, and deal with the
issues and share lessons learnt.

Comment 187 David Carver 2011-02-10 19:49:37 EST

Uh....I. did volunteer to help manage the nexus instance.

Comment 188 Alex Blewitt 2011-02-10 19:54:38 EST

(In reply to comment #186)
> That aside, I understand that it would be neat to, and we need to, and we
> must do, but the real question is who is going to do the work? Are you?

We need to get the repository set up first; the conversions for non-Maven projects can happen subsequently. I listed a handful in comment #185 of people who already publish Maven artefacts; you can add JGit to that list, too:

http://download.eclipse.org/jgit/maven

> I want to be clear, I'm not against solving the issues but again we are not
> making progress because the requirements are all over and because nobody who
> understands the problem well enough has the bandwidth to take on it.

1. Set up well-known Maven repository, either with a repo manager or dumb file space
2. Tell JGit/Jetty/EclipseLink/ECF to put stuff there
3. Until all/most/some projects are Maven consumable
3a. Help convert them
3b. Publish them
3c. Go to step 3

> Sonatype has offered to provide a free a copy of Nexus Pro to the Eclipse
> Foundation, the Eclipse Foundation is ok to run a server, what is needed is
> VOLUNTEERS TO HELP THE INDIVIDUAL PROJECTS create the poms, and deal with the
> issues and share lessons learnt.

Right, let's get that going then. We have already identified several projects who are capable of making the migration happen immediately. Subsequent projects can migrate as separate bugs or children of these projects.

I'm happy to administer whatever repository solution is provided if the webmaster wants to delegate to me.

Comment 189 Pascal Rapicault 2011-02-10 19:59:41 EST

(In reply to comment #185)
> (In reply to comment #184)
> >  Bob, the answer will still be no if we can't focus our efforts on a subset
> > of the problem (see my call in comment #179). On top of that, it may actually
> > make sense for you to get in touch with the EMF team to identify the things
> > that should be published and help them with creating the poms.
> 
> Pascal, this bug is not about 'who should be in the repo' - that's for another
> bug. This is about providing a Maven repository for those that want in.
   Sorry but it is a problem because when you start converting the OSGi dependencies to the Maven ones, you'd better do it right and coordinate with others otherwise consumers will find themselves with dependencies that are all over the place thus bringing several copies of EMF, JFace, etc on their classpath which will result in even more complaints that Maven downloads the whole internet and worst runtime issues, etc....

Again I'm not saying no, I'm just saying, let's start by doing something that is well understood so ppl are already in pain can be helped and then see about the whole sale conversion (probably using the b3 aggregator).

Comment 190 Alex Blewitt 2011-02-10 20:26:00 EST

(In reply to comment #189)
> (In reply to comment #185)
> > (In reply to comment #184)
> > >  Bob, the answer will still be no if we can't focus our efforts on a subset
> > > of the problem (see my call in comment #179). On top of that, it may actually
> > > make sense for you to get in touch with the EMF team to identify the things
> > > that should be published and help them with creating the poms.
> > 
> > Pascal, this bug is not about 'who should be in the repo' - that's for another
> > bug. This is about providing a Maven repository for those that want in.
>  Sorry but it is a problem 

But not this bug's problem - this is about getting the repo set up. Please file another bug about wholesale b3 migration, so that the repo can get set up and those that already have Maven-compatible POMs have a place to install Maven components instead of the jumbled mess we have already.

Comment 191 Eric Gwin 2011-02-11 10:49:29 EST

I've been asked to provide EclipeLink's 'Maven experience', in terms of this bug I'm interpreting that to mean our external facing experience.

However, based upon the diverging discussion in this bug I want to provide some background info. EclipseLink currently builds with Ant, and uses Mavenant to publish to our Maven repository (on the download site), and Eclipse tools to generate our p2 repositories (download site as well). Our build generates OSGi bundles, POJO jars, and javadoc, and installer archives, and we also make those available via a download page. I am currently investigating moving our OSGi bundle generation over to Tycho, with an eye to migrating the whole system to Maven in the long-term for consistency. 

I've outlined the above because I wanted to point out that using Tycho to build will in no way effect our need to publish our bits to a Maven repository. Other projects need access to our components, both OSGi and POJO, and they use Maven to build. That has meant providing our own Maven repository for them. 

Until I started the work with Tycho I was relatively new to Maven, and unaware of the central Maven repository. However, we were publishing not only what we produce, but the few bundles we needed at runtime as well. While moving our publishing to Maven central may solve the issues Alex has highlighted, it doesn't but resolve issues surrounding our runtime dependencies. Namely: 
- How do we publish an OSGi-ified, Eclipse signed third-party jar to Maven central, if one already exists in its original form? 
- What groupID to use? 
- Are there Licensing restrictions?
- How to publish multiple builds of Orbit bundles, where only the qualifier changes? (Orbit maintains the original libraries' version, and increments the qualifier as fixes are required).
I'm sure there are others.

For us, this is where an Eclipse sponsored Maven repo would come in. The foundation has already done the due diligence. Orbit is publicly available via p2. Eclipse providing those same libraries via a different media shouldn't change its legal status.

Having an Eclipse sponsored Maven repository would:
- provide a centralized place to make our bits, and our dependencies available to groups who build with Maven and want to use them.
- keep us from cluttering our eclipse mirrors with ever growing maven artifacts
- centralize and remove duplication of published dependencies with other eclipse project based Maven repositories 

I have heard that Buckminster and B3 can take our existing p2 repositories and make them Maven usable. I was quite excited by this, as it could mean that rather than publishing our bits to three mediums we could simply publish to two. However, I don't think that resolves the mirror issue, nor is the Eclipse p2 policy regarding milestones in-line with Maven philosophy: Basically if it's not a snapshot and it's been published, it should persist forever.

I think Eclipse could hammer out a policy regarding the publication of milestones (we don't provide them in maven, or they are known to have limited duration, or they are published in a separate transient repository, or we decide to maintain them forever). I'm not certain that tying our Maven support to P2 is a good idea however. 

If we have the software, hardware, and the clearance, let's proceed with setting up an Eclipse Maven server for released versions of Eclipse bundles only. Eclipse can expand its Maven support later, we (eclipse) can also investigate the Buckminister solution, but let's get started. As Pascal has pointed out, that would also require manpower. There have been several offers to help. I am also willing, though with my limited expertise with Maven I'm not certain how much help I'd be. 

Based upon the response to the bug, I'd say there is interest within the foundation to support Maven users. Perhaps we should start by putting together the team. Then identify the issues, potential solutions, initial scope, prefered solution, and milestones to get there.

Comment 192 Bob Fields 2011-02-11 12:59:28 EST

(In reply to comment #185)
> (In reply to comment #184)
> We already have EclipseLink publishing to a repo-ette:
> 
> http://www.eclipse.org/downloads/download.php?r=1&nf=1&file=/rt/eclipselink/maven.repo

This one can't be used as a maven repository location - maven searches all listed repositories for all dependencies, if not found download.php sends a .jar file which is really an .html file telling you the artifact is not available, wrecking the maven build. I probably should report a bug on that one.
 
> Ultimately, if a standard location were
> made available then it might even filter up into Maven central, which the
> Sonatype guys manage, which would be great for everyone (and alleviate the
> bandwith concerns).
> 
> I'd suggest following Apache's model; they have https://repository.apache.org/
> and use that for both releases
> (https://repository.apache.org/content/repositories/releases/) and
> snapshots/work-in-progress
> (https://repository.apache.org/content/repositories/snapshots/). It's managed
> with a Sonatype Nexus instance (and it's the professional version, with P2 and
> OBR support).
> 
> We could have repository.eclipse.org set up running a Nexus OSS instance (or a
> Pro instance if the Sonatype guys wanted to donate a license to Eclipse.org)
> and then enable those using a Maven repository to publish to that site. Get
> that content feeding into Maven Central, and then Maven Central's regional
> locations can provide the caching.

+1.

I'd be happy to start with just the actual releases, maybe one or two updates a year, that shouldn't be too burdensome in terms of size/bandwidth/process. I'd be happy to help with Nexus, it's a great tool we use internally, and I know a lot about maven but know nothing about OSGI or Tycho or building Eclipse projects, my role would probably be limited to verifying that it works correctly. Sonatype will provide the professional version to open source projects, and even some assistance in creating a maven infrastructure and migrating release artifacts to maven Central.

One thing that would help significantly - Central sync requires a -javadoc.jar artifact which is not available as a downloadable artifact as far as I can tell. I had to rebuild all of the jars using maven to create it, which caused some of the aforementioned problems with signed jar conflicts, and wound up simply replacing the rebuilt jar and sources.jar with the original Eclipse artifacts after creating javadocs. Is there an easy way to create that from the online apidocs for each project or download an already created version?

Comment 193 Aaron Digulla 2011-02-13 11:03:53 EST

I've created Bug 337068 to get the infrastructure up and running.

I opt for "maven.eclipse.org" instead of "repository.eclipse.org" because it clearly communicates the intent. "repository" sounds like "p2" in the eclipse ecosystem.

As soon as maven.eclipse.org is available, I suggest to find a simple project (like org.eclipse.equinox or a project which already creates Maven) and see how it needs to be changed to deploy automatically to the new maven repo. This change should be tracked in a new bug.

With that knowledge, we can create bugs for every other project we want to see in Maven.

I also suggest to create some more bugs to help focus the discussion:

- Naming convention in the maven repo for eclipse artifacts (especially how to map nightly builds to Maven snapshots and how to handle qualifiers of releases).

- How to map Orbit bundles to Maven artifacts. For example, should there be two versions of the POM (one with Orbit and the safety of Eclipse's IP process and one which uses the official release from Maven central)?

Comment 194 David Carver 2011-02-22 13:01:01 EST

(In reply to comment #193)
> I've created Bug 337068 to get the infrastructure up and running.
> 
> I opt for "maven.eclipse.org" instead of "repository.eclipse.org" because it
> clearly communicates the intent. "repository" sounds like "p2" in the eclipse
> ecosystem.
> 
> As soon as maven.eclipse.org is available, I suggest to find a simple project
> (like org.eclipse.equinox or a project which already creates Maven) and see how
> it needs to be changed to deploy automatically to the new maven repo. This
> change should be tracked in a new bug.

Once this is done, I'll use the PsychoPath XPath 2.0 component as an example/test project.  I already us Tycho and Maven 3.0.

Comment 195 Alex Blewitt 2011-03-18 18:41:53 EDT

I have started a wiki at http://wiki.eclipse.org/Maven to discuss/record issues/decisions relating to Maven at Eclipse.

Comment 196 Nobody - feel free to take it 2012-06-15 16:41:58 EDT

Are there plans to add all Eclipse Modeling bundles/artifacts of Juno Release?
This will be very important to use Acceleo (M2T) Maven plugin.

Comment 197 Bob Fields 2012-09-21 14:17:38 EDT

We (www.andromda.org) are in the same boat as Acceleo. Our maven project depends on the EMF/UML2 artifacts, used to load models standalone outside of any Eclipse IDE. Three years later, still no stable maven repository of released artifacts, nothing exists in the nexus orbit or releases repositories.

I would be happy to repeat the process of two years ago (manually staging and releasing the eclipse artifacts our project needs), following the new naming guidelines in the wiki page http://wiki.eclipse.org/Maven. I only need 7 artifacts, but had to rebuild about 25 in order to get all of the dependencies. I'm don't have any project at eclipse.org, and people are supposed to ask permission before deploying another project's artifacts to maven central. I would deploy the original unmodified jar and source artifacts. Or perhaps someone from Acceleo can do this? The jars we need are: org.eclipse.uml2 resources, uml, common; org.eclipse.emf common, ecore, ecore.xmi, ecore2xml. 

My earlier question was never answered - Is there a jar download available for an artifact javadocs? Or some way to cobble together the online version into a single jar? Otherwise we have to rebuild the javadocs from the source artifacts, and they may not exactly match the online version (some missing links and aggregated references). Javadocs and source jars are required for release to maven central.

Comment 198 Manuel Blechschmidt 2013-01-01 12:34:36 EST

I am using AndroMDA and I am currently looking into Acceleo.

I am using MagicDraw UML (17.0.2) to export my models to UML2 (http://www.eclipse.org/uml2/2.0.0/UML). Unfortunately all 3 programs are using different versions of UML2.

MagicDraw uses by default: http://www.omg.org/spec/UML/20110701
AndroMDA currently supports: http://www.eclipse.org/uml2/2.0.0/UML
Acceleo uses the most current version: http://www.eclipse.org/uml2/4.0.0/UML

I used MDA because I believe that business concepts are living longer then technical concepts but I am currently making the experience that the UML tools together with the UML are evolving a lot faster then the contents of my models.

A maven repository with the different ecore meta models for UML would be much appreciated. I tried to use the http://www.eclipse.org/uml2/2.0.0/UML metamodel with Acceleo but I was not able to get this working (org.eclipse.uml2.uml_2.0.4.v200707131442.jar). Mainly because I didn't find a update site where I was able to download the correct meta model.

Unfortunately working with UML and MDA technologies is still very complicated because of the fast evolving standards and heterogenous tools.

Comment 199 Wayne Beaton 2013-01-01 17:43:35 EST

(In reply to comment #198)

> A maven repository with the different ecore meta models for UML would be
> much appreciated. I tried to use the http://www.eclipse.org/uml2/2.0.0/UML
> metamodel with Acceleo but I was not able to get this working
> (org.eclipse.uml2.uml_2.0.4.v200707131442.jar). Mainly because I didn't find
> a update site where I was able to download the correct meta model.

Please make sure that the appropriate modeling project (UML2?) is aware of your issue. This bug should not be interfering with any project's ability to create a p2 repository. In fairness to the project, there may be other blockers/point-in-time problems...

Comment 200 Bob Fields 2013-01-02 14:05:30 EST

(In reply to comment #198)
> I am using AndroMDA and I am currently looking into Acceleo.
> 
> I am using MagicDraw UML (17.0.2) to export my models to UML2
> (http://www.eclipse.org/uml2/2.0.0/UML). Unfortunately all 3 programs are
> using different versions of UML2.
> 
> MagicDraw uses by default: http://www.omg.org/spec/UML/20110701
> AndroMDA currently supports: http://www.eclipse.org/uml2/2.0.0/UML
> Acceleo uses the most current version: http://www.eclipse.org/uml2/4.0.0/UML
> 
> A maven repository with the different ecore meta models for UML would be
> much appreciated. I tried to use the http://www.eclipse.org/uml2/2.0.0/UML
> metamodel with Acceleo but I was not able to get this working
> (org.eclipse.uml2.uml_2.0.4.v200707131442.jar). Mainly because I didn't find
> a update site where I was able to download the correct meta model.

If your MagicDraw publishes v2.0.0 UML and Acceleo uses v4.0.0, the older model version should automatically be converted (in memory) to the current version when it is loaded in Acceleo, you don't need to exactly match the UML2 version, only be using something compatible (not UML2 v1.x).

I published the EMF and UML2 artifacts to maven Central 3 years ago so that AndroMDA had the proper maven dependencies. It is amazing that there are still no updated artifacts published since that time. http://maven.eclipse.org/nexus appears to be close but there are no uml.jar artifacts, pom.bak artifacts, and no javadoc artifacts, so it's not possible to synch to maven central, and the artifacts are 1 1/2 years old. It really shouldn't be up to each individual project - Eclipse.org should publish all artifacts on every release, to maven Central, or at least to some stable eclipse maven repository.

Comment 201 John Mark 2013-04-29 03:46:48 EDT

Are there any plans to automate this form the new CBI that is created? Since all the builds are now being done by Hudson it should be pretty simple to push the artifacts to a central repository.

This will really help RCP application developers who depend on specific versions of certain plugins.

An Maven Eclipse repository at  http://maven.eclipse.org seems to exist but does not seem to have updated artifacts. Pushing the updated artifacts should be owned by the CBI and not the individual plugin devs who already are stretched pretty thin.

Comment 202 Pascal Rapicault 2013-04-29 10:29:20 EDT

> This will really help RCP application developers who depend on specific
> versions of certain plugins.
    Could you please detail how a Maven Repo of Eclipse content would help in this particular task? Which artifact are you trying to get, which problem do you have with them? Thx.

Comment 203 Tarun Elankath 2013-10-08 12:57:12 EDT

>     Could you please detail how a Maven Repo of Eclipse content would help
> in this particular task? Which artifact are you trying to get, which problem
> do you have with them? Thx.

I cannot speak for the other commenter, but having eclipse platform plugins on the central maven repository will be a _lifesaver_.  Folks can consume features of the eclipse platform plugins and tools in their maven projects by simply specifying <dependency> on the jars that provide such features. Examples: source code indexers leveraging the classes of the core JDT, etc.  

Providing the platform artifacts as Maven artifacts on the central maven repository will open up the platform for use in a wide variety of non-Eclipse tools.

Comment 204 Jörg Sesterhenn 2014-01-14 07:21:45 EST

(In reply to Tarun Elankath from comment #203)
> >     Could you please detail how a Maven Repo of Eclipse content would help
> > in this particular task? Which artifact are you trying to get, which problem
> > do you have with them? Thx.
> 
> I cannot speak for the other commenter, but having eclipse platform plugins
> on the central maven repository will be a _lifesaver_.  Folks can consume
> features of the eclipse platform plugins and tools in their maven projects
> by simply specifying <dependency> on the jars that provide such features.
> Examples: source code indexers leveraging the classes of the core JDT, etc.  
> 
> Providing the platform artifacts as Maven artifacts on the central maven
> repository will open up the platform for use in a wide variety of
> non-Eclipse tools.

+1

Comment 205 Dave Orme 2014-04-22 09:45:13 EDT

I'm looking to use SWT + JFace data binding outside Eclipse.  Then I'll probably add EMF bits and, ...   Having Eclipse bundles exposed in a classic Maven repository would really help me.

In addition, I'm working in a pretty Maven-centric environment.  It is pretty much impossible to sell Eclipse as a platform here without Eclipse jars available from Maven.  I'd like to see Eclipse grow; hence, I'd like to see this impediment to Eclipse's growth removed.

Comment 206 Aaron Digulla 2014-04-27 06:57:08 EDT

(In reply to Tarun Elankath from comment #203)

> Providing the platform artifacts as Maven artifacts on the central maven
> repository will open up the platform for use in a wide variety of
> non-Eclipse tools.

Please note that you can't simply deploy the bundles to Maven Central. Eclipse uses OSGi which means each bundle gets its own classloader while Maven uses a single classloader for everything.

So in order to use Eclipse bundles in Maven projects, you need to check them for problems and clean them. This is the reason MT4E (https://wiki.eclipse.org/Maven_Tools_4_Eclipse) exists.

Also, you probably want to cut the dependencies to Orbit (which routinely cripples the original projects for legal reasons) and use the original Maven artifact instead.