Bugzilla – Full Text Bug Listing
| Summary: | Xtext artifacts on maven.eclipse.org | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Technology] Dash | Reporter: | Dennis Huebner <dennis.huebner> | ||||
| Component: | Maven | Assignee: | Project Dash Incoming bugs <dash-inbox> | ||||
| Status: | RESOLVED FIXED | QA Contact: | |||||
| Severity: | normal | ||||||
| Priority: | P3 | CC: | alex.blewitt, dj.escay, d_a_carver, jesse.mcconnell, sven.efftinge, webmaster | ||||
| Version: | unspecified | ||||||
| Target Milestone: | --- | ||||||
| Hardware: | PC | ||||||
| OS: | Mac OS X - Carbon (unsup.) | ||||||
| Whiteboard: | |||||||
| Bug Depends on: | |||||||
| Bug Blocks: | 340759 | ||||||
| Attachments: |
|
||||||
|
Description
Dennis Huebner
(In reply to comment #0) > We would like to add Xtext artifacts to maven eclipse.org. > What are the steps to follow? I think I just need a userid and password. So I can link nexus to existing maven artifacts on build.eclipse.org. Added Dave to CC. (Regarding to comment https://bugs.eclipse.org/bugs/show_bug.cgi?id=340759#c0 ) Dave, are you the right person to talk to if we want to have our artifacts published into maven.eclipse.org repository? What do we need to do? (In reply to comment #3) > Dave, > are you the right person to talk to if we want to have our artifacts published > into maven.eclipse.org repository? > What do we need to do? Either myself or Jesse can get you setup. We need a user id setup on maven.eclipse.org/nexus that has deployment rights. That should ideally be an existing committer. I'd also suggest having your projects parent pom inherit from the eclipse parent pom. http://wiki.eclipse.org/Maven/Parent_POM That contains several profiles that can then be enabled during the maven release/deploy process. Let's start by getting the user id you want setup to have deployment rights. (In reply to comment #4) > Let's start by getting the user id you want setup to have deployment rights. User id should be: dhubner Did you already give Dennis the needed privileges? (In reply to comment #6) > Did you already give Dennis the needed privileges? Sorry, no I just saw this message now. Let me try and give denise the appropriate rights. Okay, just set Dennis with access, and set him a separate email with the password. Please let me know if there are any issues. (In reply to comment #8) > Okay, just set Dennis with access, and set him a separate email with the > password. Please let me know if there are any issues. Thank's Dave, we build using buckminster and then convert our p2 repository to maven using b3 aggregator. The repository can be found here: /home/data/httpd/download.eclipse.org/modeling/tmf/xtext/maven/ . Therefore I think the simplest way to get Xtext artifacts available on mavn.eclipse.org is to set up our maven repository as a proxy. I'm able to log in to http://maven.eclipse.org/nexus/ but I can't add any new repositories. Regards, Dennis. (In reply to comment #9) > (In reply to comment #8) > > Okay, just set Dennis with access, and set him a separate email with the > > password. Please let me know if there are any issues. > > Thank's Dave, > we build using buckminster and then convert our p2 repository to maven using b3 > aggregator. The repository can be found here: > /home/data/httpd/download.eclipse.org/modeling/tmf/xtext/maven/ . > Therefore I think the simplest way to get Xtext artifacts available on > mavn.eclipse.org is to set up our maven repository as a proxy. > I'm able to log in to http://maven.eclipse.org/nexus/ but I can't add any new > repositories. What is the public url, as when I try to browse to: http://download.eclipse.org/modeling/tmf/xtext/maven/ I don't see anything. I've added a Xtext repository to the maven.eclipse.org, that is setup as a proxy. Also should this contain both released and snapshots? (In reply to comment #10) > (In reply to comment #9) > > (In reply to comment #8) > > > Okay, just set Dennis with access, and set him a separate email with the > > > password. Please let me know if there are any issues. > > > > Thank's Dave, > > we build using buckminster and then convert our p2 repository to maven using b3 > > aggregator. The repository can be found here: > > /home/data/httpd/download.eclipse.org/modeling/tmf/xtext/maven/ . > > Therefore I think the simplest way to get Xtext artifacts available on > > mavn.eclipse.org is to set up our maven repository as a proxy. > > I'm able to log in to http://maven.eclipse.org/nexus/ but I can't add any new > > repositories. > > > What is the public url, as when I try to browse to: > > http://download.eclipse.org/modeling/tmf/xtext/maven/ > > I don't see anything. That's right download.eclipse.org prevents users to browse folders in ftp manner. I don't sure on which machine maven.eclipse.org is hosted but within build.eclipse.org, downloads.e.o is linked to : lrwxrwxrwx 1 root root 38 9. Mai 2008 downloads -> /home/data/httpd/download.eclipse.org/ So something like file:/home/data/httpd/download.eclipse.org/modeling/tmf/xtext/maven/ as URL should work fine. At least for "Override Local Storage Location" :) I think we should ask a webmaster, how to deal with files on dowload.ecipse.org that should be accessible/browseable using http or ftp protocol. Adding webmaster@eclipse.org to CC list. > > I've added a Xtext repository to the maven.eclipse.org, that is setup as a > proxy. Also should this contain both released and snapshots? I think just releases should be enough. (In reply to comment #11) > I think we should ask a webmaster, how to deal with files on dowload.ecipse.org > that should be accessible/browseable using http or ftp protocol. > Adding webmaster@eclipse.org to CC list. We disallow directory browsing for security and performance reasons. You could perhaps create a static index.html based on the contents? cd <path> echo "<html><head></head><body>" > index.html ; for i in $(find . -type f | sed 's/^\.\///'); do echo "<a href=\"$i\">$i</a><br>" >> index.html; done; echo "</body></html>" >> index.html Unfortunately, I get a 500 error from Nexus when I try to override or point to the downloads directory. Apparently that directory is mounted or seen by maven.eclipse.org's server. Anyways, the best option is to do a mvn deploy if possible of the artifacts up into the next repository. You can do this manually by using the deploy-file goal. You would deploy to the following url. http://maven.eclipse.org/nexus/content/repositories/xtext-repo/ More info on the Maven Deploy mojo is here: http://maven.apache.org/plugins/maven-deploy-plugin/deploy-file-mojo.html (In reply to comment #13) > Unfortunately, I get a 500 error from Nexus when I try to override or point to > the downloads directory. Apparently that directory is mounted or seen by > maven.eclipse.org's server. > > Anyways, the best option is to do a mvn deploy if possible of the artifacts up > into the next repository. You can do this manually by using the deploy-file > goal. > > You would deploy to the following url. > > http://maven.eclipse.org/nexus/content/repositories/xtext-repo/ > > More info on the Maven Deploy mojo is here: > > http://maven.apache.org/plugins/maven-deploy-plugin/deploy-file-mojo.html Thanks a lot for this info. I think we need a separate hudson job to this. @Denis Could you please create a Xtext-Maven-Deploy job for us? An another question, how to pass my nexus user password in hudson, so that it can't be seen by other commiters? You can include the username and password in a settings.xml, and then point your hudson job to that settings.xml file. You'll want to store it somewhere where hudson has read access. More info is here: http://maven.apache.org/settings.html#Servers You can tell maven to use a particular settings.xml by doing. mvn ... -s path/to/settings.xml You can also encrypt the password. See http://maven.apache.org/guides/mini/guide-encryption.html > @Denis Could you please create a Xtext-Maven-Deploy job for us? Voilà: https://hudson.eclipse.org/hudson/job/Xtext-Maven-Deploy (In reply to comment #15) > You can include the username and password in a settings.xml, and then point > your hudson job to that settings.xml file. You'll want to store it somewhere > where hudson has read access. > > More info is here: > > http://maven.apache.org/settings.html#Servers > > You can tell maven to use a particular settings.xml by doing. > > mvn ... -s path/to/settings.xml > > You can also encrypt the password. > > See http://maven.apache.org/guides/mini/guide-encryption.html I successfully deployed my first Xtext bundle!!!! :) http://maven.eclipse.org/nexus/content/repositories/testing/org/eclipse/xtext/org.eclipse.xtext/ You can look into https://hudson.eclipse.org/hudson/job/Xtext-Maven-Deploy/ configuration to see how it works. I granted you all rights, so you can look into or configure. The question is now how to deploy all jars/poms from https://hudson.eclipse.org/hudson/job/Xtext-Maven-Deploy/ws/composite-repository/final/ to nexus? deploy-file can only deploy one jar per request... I asked google but found any solution. Nexus documentation points me to a concept "Deploying to a Staged Repository" but it seems to work only with a web front end. Have you an idea? @Denis Thanks a lot for quick response! (In reply to comment #17) You can look into https://hudson.eclipse.org/hudson/job/Xtext-Maven-Deploy/ > configuration to see how it works. I granted you all rights, so you can look > into or configure. The question is now how to deploy all jars/poms from > https://hudson.eclipse.org/hudson/job/Xtext-Maven-Deploy/ws/composite-repository/final/ > to nexus? deploy-file can only deploy one jar per request... I asked google but > found any solution. Nexus documentation points me to a concept "Deploying to a > Staged Repository" but it seems to work only with a web front end. Have you an > idea? Nexus doesn't handle deploying of P2 update sites, as you need to have a pom to go with it. We only have the Nexus OSS version and not the nexus pro. Nexus Pro allows for mirroring of P2 update sites. So you can only deploy to your site the nexus bundles. If your build was running a maven build, you could use the maven deploy goal to deploy your artifacts, or the maven release mojo. But since you are running Buckminster, you will have to use the deploy-file goal and probably setup a batch script to do the actual deployment. (In reply to comment #18) > (In reply to comment #17) > You can look into https://hudson.eclipse.org/hudson/job/Xtext-Maven-Deploy/ > > configuration to see how it works. I granted you all rights, so you can look > > into or configure. The question is now how to deploy all jars/poms from > > https://hudson.eclipse.org/hudson/job/Xtext-Maven-Deploy/ws/composite-repository/final/ > > to nexus? deploy-file can only deploy one jar per request... I asked google but > > found any solution. Nexus documentation points me to a concept "Deploying to a > > Staged Repository" but it seems to work only with a web front end. Have you an > > idea? > > Nexus doesn't handle deploying of P2 update sites, as you need to have a pom to > go with it. We only have the Nexus OSS version and not the nexus pro. Nexus > Pro allows for mirroring of P2 update sites. > > So you can only deploy to your site the nexus bundles. If your build was > running a maven build, you could use the maven deploy goal to deploy your > artifacts, or the maven release mojo. But since you are running Buckminster, > you will have to use the deploy-file goal and probably setup a batch script to > do the actual deployment. Dave, it's not only a repository it's also a maven repository. Please see i.e. https://hudson.eclipse.org/hudson/job/Xtext-Maven-Deploy/ws/composite-repository/final/org/eclipse/xtext/org.eclipse.xtext/ it contains poms and metadata files. I don't think that a fragile batch script or something similar is the good solution for such important tasks like deploying to public repositories. If there are any other way to deploy the whole _maven_ repository I would prefer to grand maven.eclipse.org read/write access to some build.eclipse.org directory where I can finally promote artifacts produced with Xtext-Maven-Deploy job. So you can mirror it with nexus. Best regards, Dennis. > it's not only a repository it's also a maven repository. Please see i.e. > https://hudson.eclipse.org/hudson/job/Xtext-Maven-Deploy/ws/composite-repository/final/org/eclipse/xtext/org.eclipse.xtext/ > it contains poms and metadata files. > I don't think that a fragile batch script or something similar is the good > solution for such important tasks like deploying to public repositories. If > there are any other way to deploy the whole _maven_ repository I would prefer > to grand maven.eclipse.org read/write access to some build.eclipse.org > directory where I can finally promote artifacts produced with > Xtext-Maven-Deploy job. So you can mirror it with nexus. What I could do is setup a Proxy mirror for the following url: https://hudson.eclipse.org/hudson/job/Xtext-Maven-Deploy/ws/composite-repository/final/ But the artifacts won't actually show up in the nexus repository until they are first requested. Domebody adds the maven.eclipse.org public repository as a repository in their maven file, then they should be fetched. If we had Nexus Pro you could use the maven-nexus-plugin to accomplish what you want. So as this link says, there are only three ways to go about this if the artifacts aren't generated and deployed during a maven build process. http://stackoverflow.com/questions/1410580/nexus-supports-mass-upload-of-artifacts Maybe, Buckminster can add support around this deployment process? One more option: http://stackoverflow.com/questions/3240477/upload-download-entire-directory-to-nexus-through-maven If it helps, directory browsing is enabled on build.eclipse.org, which includes /shared and all the Hudson workspaces: Hudson /shared/jobs: http://build.eclipse.org/jobs/ Build's /shared/technology: http://build.eclipse.org/technology/ Created attachment 207154 [details]
Nexus understand buckminster index
> What I could do is setup a Proxy mirror for the following url: > > https://hudson.eclipse.org/hudson/job/Xtext-Maven-Deploy/ws/composite-repository/final/ > > But the artifacts won't actually show up in the nexus repository until they are > first requested. Domebody adds the maven.eclipse.org public repository as a > repository in their maven file, then they should be fetched. Thats ok for me, as first step. I tried the latest buckminster version which produces also a maven .index folder. So if you set http://build.eclipse.org/jobs/Xtext-Maven-Deploy/maven/ as remote and check "Download Remote Indexes" = true all artifacts should be listed. I tried this with my local nexus and it works fine. Once more... Could you please also create a snapshot repository for text and point it to http://build.eclipse.org/jobs/Xtext-Maven-Deploy/maven-snapshot/ ? > > If we had Nexus Pro you could use the maven-nexus-plugin to accomplish what you > want. > Where can I donate? :) > So as this link says, there are only three ways to go about this if the > artifacts aren't generated and deployed during a maven build process. Maven/Tycho build is not an option for us at least not now. Okay, I think I have this proxy working now. I've got the proxy working at: http://maven.eclipse.org/nexus/content/repositories/xtext-proxy/ I've also added it to the Public repositories, so that it if somebody adds the following url as a repository in their maven build it will pick up the artifacts. http://maven.eclipse.org/nexus/content/groups/public/ I've checked the xtext-proxy index out and it is correctly picking up the artifacts. One item of concern though is that antlr probably shouldn't be provided in the repository as there is already one out there. Unless you provide antlr under your own group id, and it doesn't conflict with the one in maven central. Regardless the one in maven central should take priority. Anyways, please review and let me know if this works. Either of the urls should work to retrieve the artifacts during a build. I'm resolving this one now. I've checked the link I added seems to be working. If we need to change the proxy url, let me know. Thank you! As per bug 365727 I have removed this proxy repository as it was serving out bad data. We will need to revisit this and provide a way of doing it properly. This has been taken out of rotation, but there's another bug 365899 for hosting the released artifacts which supersedes this one. |