Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 311659

Summary: Corrupt maven repo on build.eclipse.org
Product: Community Reporter: Chris Aniszczyk <caniszczyk>
Component: CI-JenkinsAssignee: CI Admin Inbox <ci.admin-inbox>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: denis.roy, d_a_carver, matthias.sohn, nboldt
Version: unspecified   
Target Milestone: ---   
Hardware: PC   
OS: Mac OS X - Carbon (unsup.)   
Whiteboard:

Description Chris Aniszczyk CLA 2010-05-05 02:16:30 EDT
We are having trouble building EGit on build.eclipse.org due to a corrupt jar...

/opt/users/hudsonbuild/.m2/repository/p2/osgi/bundle/org.eclipse.core.expressions/3.4.200.v20100222/org.eclipse.core.expressions-3.4.200.v20100222.jar

I don't have permissions to delete this which isn't good. How can we ensure that we can clean up the local maven repo on build.eclipse.org so this doesn't happen again?

https://build.eclipse.org/hudson/job/egit/181/console

[WARNING] Exception reading bundle manifest
java.util.zip.ZipException: error in opening zip file
       at java.util.zip.ZipFile.open(Native Method)
       at java.util.zip.ZipFile.<init>(ZipFile.java:125)
       at org.codehaus.tycho.osgitools.DefaultBundleManifestReader.doLoadManifest(DefaultBundleManifestReader.java:81)
       at org.codehaus.tycho.osgitools.DefaultBundleManifestReader.loadManifest(DefaultBundleManifestReader.java:43)
       at org.codehaus.tycho.osgitools.EquinoxBundleResolutionState.loadManifest(EquinoxBundleResolutionState.java:289)
       at org.codehaus.tycho.osgitools.EquinoxBundleResolutionState.loadBundleManifest(EquinoxBundleResolutionState.java:304)
       at org.codehaus.tycho.osgitools.EquinoxBundleResolutionState.addBundle(EquinoxBundleResolutionState.java:67)
       at org.codehaus.tycho.osgitools.EquinoxBundleResolutionState.newInstance(EquinoxBundleResolutionState.java:359)
       at org.codehaus.tycho.osgitools.OsgiBundleProject.setTargetPlatform(OsgiBundleProject.java:92)
       at org.codehaus.tycho.maven.TychoMavenLifecycleParticipant.afterProjectsRead(TychoMavenLifecycleParticipant.java:135)
       at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:200)
       at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:95)
       at org.apache.maven.cli.MavenCli.execute(MavenCli.java:430)
       at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:160)
       at org.apache.maven.cli.MavenCli.main(MavenCli.java:124)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:45)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
       at java.lang.reflect.Method.invoke(Method.java:599)
       at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290)
       at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230)
       at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
       at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
[ERROR] Internal error: java.lang.RuntimeException: Unable to initialize BundleResolutionState: Manifest not found in /opt/users/hudsonbuild/.m2/repository/p2/osgi/bundle/org.eclipse.core.expressions/3.4.200.v20100222/org.eclipse.core.expressions-3.4.200.v20100222.jar -> [Help 1]
Comment 1 David Carver CLA 2010-05-05 11:26:18 EDT
A work around for you for now, is to have your builds use Local repositories that are isolated to the particular job.   You can do this by checking the use Local Repository option in your jobs configuration.

The only down side to this is that your workspace will use more diskspace as maven downloads the internet.
Comment 2 Chris Aniszczyk CLA 2010-05-05 11:34:34 EDT
It doesn't work for me because I need to use the input of the JGit build as part of the EGit build. I'd have to modify the EGit build a bit to make this work...
Comment 3 David Carver CLA 2010-05-05 11:41:08 EDT
(In reply to comment #2)
> It doesn't work for me because I need to use the input of the JGit build as
> part of the EGit build. I'd have to modify the EGit build a bit to make this
> work...

Correct, you would need to add the jGit repo as one of the repositories in your eGit POM.

This recommendation comes directly from the Sonatype blog:

http://www.sonatype.com/people/2009/01/maven-continuous-integration-best-practices/
Comment 4 Nick Boldt CLA 2010-05-05 12:26:03 EDT
(In reply to comment #0)
> We are having trouble building EGit on build.eclipse.org due to a corrupt
> jar...
> 
> /opt/users/hudsonbuild/.m2/repository/p2/osgi/bundle/org.eclipse.core.expressions/3.4.200.v20100222/org.eclipse.core.expressions-3.4.200.v20100222.jar
> 
> I don't have permissions to delete this which isn't good. How can we ensure
> that we can clean up the local maven repo on build.eclipse.org so this doesn't
> happen again?

How about using `mvn dependency:purge-local-repository` to purge it (and everything else, I suspect)?

http://maven.apache.org/plugins/maven-dependency-plugin/purge-local-repository-mojo.html

Should we have a nightly repo purge job?
Comment 5 David Carver CLA 2010-05-05 12:38:30 EDT
Isolation of the repos and periodic cleaning of the repos is probably going to be the best thing long term.   Otherwise you have no idea what is getting mixed in there and what some other job adds could affect your job.
Comment 6 Nick Boldt CLA 2010-05-05 12:42:24 EDT
If every component has its own m2 repo (eg., using -Dmaven.repo.local=/opt/users/hudsonbuild/.hudson/jobs/egit/workspace/m2_repository), then provided there's a URL for that repo (eg., https://build.eclipse.org/hudson/job/egit/ws/m2_repository/), your downstream build's parent pom could include something like this:

<pluginRepository>
	<id>tycho</id>
	<url>http://repository.sonatype.org/content/repositories/snapshots/
	</url>
	<snapshots>
		<enabled>true</enabled>
	</snapshots>
	<releases>
		<enabled>true</enabled>
	</releases>
</pluginRepository>

Alternatively, if you publish to a p2 repo, then use:

<repository>
	<id>swtbot-helios</id>
	<url>http://download.eclipse.org/technology/swtbot/helios/dev-build/update-site
	</url>
	<layout>p2</layout>
	<snapshots>
		<enabled>true</enabled>
	</snapshots>
	<releases>
		<enabled>true</enabled>
	</releases>
</repository>
Comment 7 David Carver CLA 2010-05-05 13:22:19 EDT
(In reply to comment #6)
> If every component has its own m2 repo (eg., using
> -Dmaven.repo.local=/opt/users/hudsonbuild/.hudson/jobs/egit/workspace/m2_repository),


If you use the job configuration option for local repo, by default it puts it in the workspace/.repository directory.  Then could use the Archive Artifacts option to make a stable reference to it.
Comment 8 Matthias Sohn CLA 2011-02-28 04:32:56 EST
we are using local maven repos for jgit and egit builds since a long time now, I think we can close this bug ...
Comment 9 Denis Roy CLA 2011-03-10 14:14:20 EST
Closing as per previous comment.