Community
Participate
Working Groups
Virgo and Gemini Web, for example, use Ant/Ivy as a build system and publish dependencies (3rd party libraries in bundle format and Eclipse bundles) in an external Ivy repository on Amazon S3. This line item is to make Orbit similarly accessible as a remote Ivy repository. Maven users would benefit from support too. The approach outlines in the Orbit FAQ ([1]) is far too labour intensive for a runtime project with many dependencies.The requirements are: 1. to be able to specify Orbit once in a project's build configuration 2. to be able to refer to each dependency using standard Ivy or Maven "coordinates" and download directly from Orbit. The approach taken in the external Ivy repository above is to publish all bundles to a single Ivy repository and to a single Maven repository. An Ant/Ivy build, for example, can then point to the single remote Ivy repository in once piece of configuration and then capture dependencies using normal Ivy coordinates. The absence of this support is the most significant factor in Virgo planning not to join the release train. [1] http://wiki.eclipse.org/index.php/Orbit_Faq#How_to_use_an_Orbit_bundle_in_your_Target_Platform
What's the advantage or need of having on Amazon S3 ... instead of eclipse.org?
(In reply to comment #1) > What's the advantage or need of having on Amazon S3 ... instead of eclipse.org? It's purely historical because Virgo started out outside Eclipse. It would be better for Eclipse projects to use eclipse.org storage and, in that case, Orbit seems like the right source of "blessed" bundles to put in that storage.
Some work has been undertaken to create a Maven/Nexus repository for Eclipse projects at: http://maven.eclipse.org AFAIK, the repository includes Orbit. In an idealized form, does this provide you with what you need (from what I can tell of Ivy, the Maven2 format repositories are supported)? I say "idealized form" above, because I'm not certain what the current state of the repository is. In particular, I'm not sure what the state of Orbit is in that repository (we may need to do some extra work to maintain currency with new Orbit versions). I'm adding Bug 337068 as a blocker based on an assumption that my understanding of the Maven relationship is correct.
(In reply to comment #3) > Some work has been undertaken to create a Maven/Nexus repository for Eclipse > projects at: > > http://maven.eclipse.org > > AFAIK, the repository includes Orbit. In an idealized form, does this provide > you with what you need (from what I can tell of Ivy, the Maven2 format > repositories are supported)? There seem to be a couple of approaches to consuming from a Maven 2 repository from an Ivy build. If anyone's interested, they both involve configuring an ivysettings.xml file. The more basic approach adds this: <url name="maven-lookup" m2compatible="true"> <artifact pattern="http://maven.eclipse.org/maven2/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]"/> </url> Another approach, using ibiblio, is described here: http://stackoverflow.com/questions/4836011/use-public-maven-repository-with-ivy > > I say "idealized form" above, because I'm not certain what the current state of > the repository is. In particular, I'm not sure what the state of Orbit is in > that repository (we may need to do some extra work to maintain currency with > new Orbit versions). I hope to clarify this via https://bugs.eclipse.org/bugs/show_bug.cgi?id=337068#c26 > > I'm adding Bug 337068 as a blocker based on an assumption that my understanding > of the Maven relationship is correct. Thanks Wayne!
From https://bugs.eclipse.org/bugs/show_bug.cgi?id=337068#c32 > Huh! So, far from containing Orbit bundles, maven.eclipse.org actually contains > "mavenised" JARs converted from the corresponding Orbit bundles, which is > precisely the opposite of what we need in bug 364469, e.g. to build OSGi > projects such as Virgo by getting their bundle dependencies directly out of > Orbit. > > Unless someone can definitely state that maven.eclipse.org will eventually > contain all Orbit bundles with no modifications, then I'll need to sever the > dependency of bug 364469 on this bug and look elsewhere. I'm unsure what you're talking about here. Maven cannot consume OSGi bundles since they're missing the POMs. The JARs themselves won't be changed except to fix two issues: 1. Package exploded JARs (plugins that come from an Eclipse installation which were unpacked during the install) 2. Wrapped JARs (library JARs inside of OSGi bundles) will be unwrapped because javac can't load classes from nested JARs. Point #2 is a TODO in my conversion tool. In fact, the resulting files aren't 100% compatible to Maven artifacts since they are missing the information in META-INF/maven/* Can you please explain me your concerns in the light of this information?
(In reply to comment #5) > From https://bugs.eclipse.org/bugs/show_bug.cgi?id=337068#c32 > > > Huh! So, far from containing Orbit bundles, maven.eclipse.org actually contains > > "mavenised" JARs converted from the corresponding Orbit bundles, which is > > precisely the opposite of what we need in bug 364469, e.g. to build OSGi > > projects such as Virgo by getting their bundle dependencies directly out of > > Orbit. > > > > Unless someone can definitely state that maven.eclipse.org will eventually > > contain all Orbit bundles with no modifications, then I'll need to sever the > > dependency of bug 364469 on this bug and look elsewhere. > > I'm unsure what you're talking about here. Maven cannot consume OSGi bundles > since they're missing the POMs. The JARs themselves won't be changed except to > fix two issues: > > 1. Package exploded JARs (plugins that come from an Eclipse installation which > were unpacked during the install) I'm not sure I understand this point, but I think it's moot as far as this bug is concerned. > > 2. Wrapped JARs (library JARs inside of OSGi bundles) will be unwrapped because > javac can't load classes from nested JARs. This is a moot point as far as this bug is concerned. The bundles which Virgo depends on do not contain nested JARs and so they are valid for use with javac. > > Point #2 is a TODO in my conversion tool. > > In fact, the resulting files aren't 100% compatible to Maven artifacts since > they are missing the information in META-INF/maven/* This bug requires that no Maven-specific or other content be added to Orbit bundles. > > Can you please explain me your concerns in the light of this information? To be clear, the requirement here is to be able to build OSGi projects such as Virgo with Ant/Ivy or Maven and download *unmodified* Orbit bundles during the build. The alternative of copying Orbit bundles into the SpringSource Enterprise Bundle Repository (the 'EBR", which provides Maven and Ivy repositories on Amazon S3 referred to in the bug description) and having the build download them from there duplicates effort and is not sustainable. To avoid this duplication and satisfy the release train requirement of using the Orbit version of each dependency, we would like to wean Virgo off the EBR and, in future, add Virgo dependencies to Orbit and consume them directly out of Orbit. Wayne's hope was that maven.eclipse.org could provide that direct access to Orbit, but that would only be true if maven.eclipse.org contains unmodified Orbit bundles.
(In reply to comment #6) > Wayne's hope was that maven.eclipse.org could provide that direct > access to Orbit, but that would only be true if maven.eclipse.org contains > unmodified Orbit bundles. As far as I can see, this is the case except for the bundle org.apache.batik.pdf which is tainted (see bug 350792). I need to fix that because otherwise, it would make maven.eclipse.org unusable for me :-) I'm not 100% sure how to fix it since the Orbit guys don't seem too keep to fix the problem at the source. Comments welcome.
(In reply to comment #7) > (In reply to comment #6) > > > Wayne's hope was that maven.eclipse.org could provide that direct > > access to Orbit, but that would only be true if maven.eclipse.org contains > > unmodified Orbit bundles. > > As far as I can see, this is the case Good. Thanks. We now need maven.eclipse.org to be fully populated from Orbit. > except for the bundle > org.apache.batik.pdf which is tainted (see bug 350792). I need to fix that > because otherwise, it would make maven.eclipse.org unusable for me :-) > > I'm not 100% sure how to fix it since the Orbit guys don't seem too keep to fix > the problem at the source. Comments welcome. (I don't really wish to get involved, but if the problem is in the Apache packaging, have you raised a bug at Apache to get this fixed at source?)
(In reply to comment #8) > (I don't really wish to get involved, but if the problem is in the Apache > packaging, have you raised a bug at Apache to get this fixed at source?) *lol* No one wants to get involved in this one ;-) Apache won't touch this anymore (the project is at version 1.7; 1.6 isn't supported anymore). Orbit didn't notice when they imported the package and now, they don't want to break/fix anything for anyone. Which means the package works (sort of) if you use OSGi (because OSGi isolates the classpaths and no other bundle can see those commons-io packages) but as soon as you use the bundle without OSGi (plain Maven build, i.e. without Tycho, or on the classpath of a non-OSGi app), bad things happen. So I'll put my head on the block here and strip those classes before I deploy the bundle on maven.eclipse.org (I'll add them as dependencies to the POM, so you won't notice). Which boils down to: I'll publish untouched Orbit bundle on maven.eclipse.org but since that server is mainly for Maven (non-Tycho, non-OSGi, non-p2) builds, bundles there must be consumable by the plain Java compiler. This makes maven.eclipse.org consumable by gradle, Ivy+Ant.
(In reply to comment #9) > Which boils down to: I'll publish untouched Orbit bundle on maven.eclipse.org > but since that server is mainly for Maven (non-Tycho, non-OSGi, non-p2) builds, > bundles there must be consumable by the plain Java compiler. This worries me. Virgo is OSGi through and through so the aim of this bug opposed to the aim you state for maven.eclipse.org "mainly for Maven (..., non-OSGi, ...) builds". Unless we can agree that maven.eclipse.org will support OSGi builds, I don't see how I can risk using it to satisfy this bug.
To put this in really simple terms: is maven.eclipse.org suitable for use with a Maven build that needs to download the identical binary content of each bundle that is present in Orbit?
(In reply to comment #11) > To put this in really simple terms: is maven.eclipse.org suitable for use with > a Maven build that needs to download the identical binary content of each > bundle that is present in Orbit? Only if Maven can actually consume that bundle unmodified. That's the point: Most Orbit bundles can be consumed by Maven and by OSGi-based systems (unlike other Eclipse projects). But I'm working on something that makes all of Eclipse available to Maven builds, not only Orbit. If an Eclipse bundle can't be consumed by Maven in unmodified form, the "deploy process" will "translate" it to make the build (and the app created from it) work. Two examples: - The bundle org.apache.batik.pdf contains code that conflicts with code from the bundles org.apache.commons.io and org.apache.commons.logging. If I would take them unmodified, I'd break your build and the resulting application might or might not work (depending on the order of the JARs on the classpath). This is obviously not what anyone wants, so I fix it. - The bundle org.w3c.sac contains the nested flute.jar. Maven can't consume such bundles, so I have to unwrap such bundles. I'm pretty sure that Orbit doesn't contain any of these but I'm not 100% sure; you probably know more about this. That's it. The goal is to have something which works out of the box with Maven (and build systems that are compatible). When you say "Virgo is 100% OSGi" that means for me "can't be built/consumed/used with Maven" because OSGi has features which Maven doesn't support. If you don't use any of those features, you're still 100% OSGi compatible but I have no way of telling. All I can do is tell you what maven.eclipse.org is: It's a *Maven* repository. If that's enough, it's for you. If it isn't, then I'm sorry but this is a *Maven* repository, not an OSGi or p2 or Tycho repository. We already have those. I know little about Ivy (only that it's related to Ant in some way). I don't know your build. I don't know what OSGi features you're using. Right now, all I can say: If you - use nested JARs - rely on exported packages (= if your build breaks when all classes can see each other a.k.a. if you need "Export-Package" in your manifest) - rely on automated bundle discovery (if you use "Import-Package" in your manifest) maven.eclipse.org *might* *not* work for you (mostly because the maven-compiler-plugin can't handle these). If you don't need these features, maven.eclipse.org *should* work for you. Unhappy? Me, too. I'd love if these things were more simple :-) In an ideal world, the code would be in one file and dependency information (MANIFEST.MF/pom.xml) would be in an attached metadata blob where everyone can put their data, so we could all consume from the same source. Isn't going to happen :-)
(In reply to comment #12) > I know little about Ivy (only that it's related to Ant in some way). I don't > know your build. I don't know what OSGi features you're using. Right now, all I > can say: If you > > - use nested JARs We do not consume (Orbit) bundles containing nested JARs, > - rely on exported packages (= if your build breaks when all classes can see > each other a.k.a. if you need "Export-Package" in your manifest) but we rely on exported packages, > - rely on automated bundle discovery (if you use "Import-Package" in your > manifest) and we use Import-Package in our manifests. Or were those the conditions meant to be ANDed rather than ORed, in which case we are ok. Also, we building using Ivy so we simply need to be able to download bundles from maven.eclipse.org as a maven repository and we do not depend on bundles having maven folders inside them.
(In reply to comment #13) > > - rely on exported packages (= if your build breaks when all classes can see > > each other a.k.a. if you need "Export-Package" in your manifest) > > but we rely on exported packages, > > > - rely on automated bundle discovery (if you use "Import-Package" in your > > manifest) > > and we use Import-Package in our manifests. I can't match that with your statement that you use Ivy to build. Ivy (AFAIK) doesn't support Import-Package nor can it hide packages on the classpath. How does Ivy know which bundles to add to the build-time classpath? Why doesn't your build break when Ivy ignores/doesn't support Export-Package?
(In reply to comment #14) > (In reply to comment #13) > > > > - rely on exported packages (= if your build breaks when all classes can see > > > each other a.k.a. if you need "Export-Package" in your manifest) > > > > but we rely on exported packages, > > > > > - rely on automated bundle discovery (if you use "Import-Package" in your > > > manifest) > > > > and we use Import-Package in our manifests. > > I can't match that with your statement that you use Ivy to build. Ivy (AFAIK) > doesn't support Import-Package nor can it hide packages on the classpath. > > How does Ivy know which bundles to add to the build-time classpath? Why doesn't > your build break when Ivy ignores/doesn't support Export-Package? Oh, my mistake. We don't need export-package at build time. A linear classpath of all the bundle dependencies treated as JARs works fine for compilation. We do use Import-Package, but it is irrelevant at build time. So I think we are in good shape to use Orbit bundles from maven.eclipse.org.
I've followed this discussion closely but still can't find a definitive answer to the question "Is there an up-to-date Maven repository based on the content of Orbit?" Can someone state clearly in what shape maven.eclipse.org is? And whether it contains the latest artifacts from Orbit? Also I would like to know how often the Maven repository is updated - is it automatically done on every Orbit build?
