Bugzilla – Full Text Bug Listing
| Summary: | Public Storage for build artifacts | ||
|---|---|---|---|
| Product: | Community | Reporter: | Steve Powell <zteve.powell> |
| Component: | CI-Jenkins | Assignee: | CI Admin Inbox <ci.admin-inbox> |
| Status: | CLOSED WONTFIX | QA Contact: | |
| Severity: | normal | ||
| Priority: | P3 | CC: | d_a_carver, tjwatson |
| Version: | unspecified | ||
| Target Milestone: | --- | ||
| Hardware: | All | ||
| OS: | All | ||
| Whiteboard: | |||
The best way is to produce P2 repositories that you then make available either through the various builds URLs (lastStable, etc). Or publish those repos to download.eclipse.org. Typically at eclipse we have concepts of Nightly, Integration, Milestone, and Release builds. All can produce P2 repos which can be used. It is sounding like you want equivalent Maven based repos as well. Which I think the Buckminster project and the Maven/Tycho project can go and produce. (In reply to comment #1) > The best way is to produce P2 repositories that you then make available either > through the various builds URLs (lastStable, etc). Or publish those repos to > download.eclipse.org. > > Typically at eclipse we have concepts of Nightly, Integration, Milestone, and > Release builds. All can produce P2 repos which can be used. > > It is sounding like you want equivalent Maven based repos as well. Which I > think the Buckminster project and the Maven/Tycho project can go and produce. I would like to publish to p2 on a regular basis, please. However, I'm not sure if I can use p2 directly to resolve dependencies with ivy during build. Can you point me to the mechanisms for publish and resolve/download? I can then modify the build scripts to use them. (In reply to comment #2) > (In reply to comment #1) > > The best way is to produce P2 repositories that you then make available either > > through the various builds URLs (lastStable, etc). Or publish those repos to > > download.eclipse.org. > > > > Typically at eclipse we have concepts of Nightly, Integration, Milestone, and > > Release builds. All can produce P2 repos which can be used. > > > > It is sounding like you want equivalent Maven based repos as well. Which I > > think the Buckminster project and the Maven/Tycho project can go and produce. > > I would like to publish to p2 on a regular basis, please. However, I'm not > sure if I can use p2 directly to resolve dependencies with ivy during build. > Can you point me to the mechanisms for publish and resolve/download? I can > then modify the build scripts to use them. Look into: http://wiki.eclipse.org/Equinox/p2/Publisher > The best way is to produce P2 repositories that you then make available either > through the various builds URLs (lastStable, etc). Or publish those repos to > download.eclipse.org. To help clarify the 'where', build artifacts that are intended to be downloaded by the general public must be placed on download.eclipse.org where they will be replicated to mirror servers worldwide. You should also link to them in a way that will allow users to pull bits from a mirror site. Please see http://wiki.eclipse.org/IT_Infrastructure_Doc#Downloads for more information on the Eclipse download servers. The download links available on Hudson web pages (lastStable, etc) are OK to use for developers. Is there anything left to do here? Denis, Thanks for checking up on this. At present we do not publish to p2 from our release build scripts (we haven't left incubation yet!) and we haven't found an easy way to resolve dependencies with Ivy from p2 repositories. We are currently running with CI builds 'read-only' and committers are running 'ripple' and 'release' builds privately, which publish the downloadables to the eclipse download site. One 'issue' is that our builds aren't run in a (headless or otherwise) eclipse -- we don't need to produce p2 repositories and we are not a 'plugin' -- so that makes us 'strange'. This issue can close, however; we are quite happily producing 'downloadables' and our contributors can build everything they need on their own machines (sourcing dependencies from SpringSource EBR, mostly) so this issue will get resolved later when we have a final (non-incubation) release and someone asks for it. sp |
Virgo builds produce publicly available artifacts which are used to resolve dependencies in other Virgo builds or other projects that use the Virgo Components. The way inter-dependencies are resolved is through ivy, and these currently use Amazon's s3 (simple storage service) for both Virgo-build generated dependencies, and publicly available stuff. We use: s3://repository.springsource.com for the public bundles/libraries -- which we will continue to use for Virgo builds (anyone who takes the repos and builds themselves can use these) and s3://zodiac.springsource.com for Virgo built artifacts. These can also be used by private builders. When we publish artifacts from the Virgo builds we need the equivalent of the zodiac.springsource.com s3 repository to publish to. In our private builds this uses (requires) ssh keys on our servers -- not something we are willing to put on the Eclipse builds; and in any case, the zodiac space doesn't belong to the Eclipse Community. So, where (and how) should we publish the artifacts used to resolve dependencies in our Virgo repositories? Notice that we can continue to build on Eclipse Hudson by using artifacts stored on the build server -- but this doesn't help those who wish to build independently; nor those who want the artifacts as dependencies of their own. It is entirely feasible that the built binaries for the whole of Virgo (output of the web-server build) and the component bundles (for the kernel, for example) goes to the same place -- but his will involve quite a lot of 'current/snapshot' builds and will mean that we ought to manage the lifetime (and version naming) carefully. Part of this 'bug request' is to ask how we publish final built binaries from the Hudson build server.