Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 255913

Summary: [mirror] Need ant tasks for mirroring and composite repositories
Product: [Eclipse Project] Equinox Reporter: Andrew Niefer <aniefer>
Component: p2Assignee: DJ Houghton <dj.houghton>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: agcattle, dj.houghton, jeffmcaffer, kim.moir, pascal
Version: unspecified   
Target Milestone: 3.5 M5   
Hardware: PC   
OS: Windows XP   
Whiteboard:
Bug Depends on: 256829    
Bug Blocks: 257783    
Attachments:
Description Flags
Ant Tasks for composite repos and mirroring
none
Update including copyright.
none
Slight improvement over previous patch
none
Missing Builders for previous patch
none
patch none

Description Andrew Niefer CLA 2008-11-19 19:19:29 EST
We need ant tasks for invoking the mirroring applications.

There are two mirroring applications: metadata and artifact.  We may want to consider consolidating them into one application that does both.
Comment 1 Jeff McAffer CLA 2008-11-28 12:15:37 EST
we already have one that does both.  lOok in the tools bundle. It may have been a little neglected but it is there and did work at one point.
Comment 2 Andrew Cattle CLA 2008-11-28 12:51:12 EST
Implementing ant tasks for the mirroring app is almost trivial. The more important thing is to get ant tasks for the composite repositories. I have some tasks ready to go but I can't release them until Bug 256829 is resolved.
Comment 3 Andrew Cattle CLA 2008-12-02 14:02:52 EST
I'm currently putting some finishing touches on the tasks. I'd like to release them all at once so I'm waiting for my changes to Bug 255685, Bug 254747, and Bug 256283 to be released.

Possibly Bug 256909, but I doubt we'll use that too much at this stage.
Comment 4 Andrew Cattle CLA 2008-12-03 09:04:24 EST
Created attachment 119374 [details]
Ant Tasks for composite repos and mirroring

This patch will only not have compile errors if my patch for Bug 256283 is applied.

Additionally, it has to ability to use functionality I introduced in Bug 255685 and will need to be updated once Bug 254747 is resolved.
Comment 5 DJ Houghton CLA 2008-12-05 17:25:25 EST
Andrew, please attach a new patch in which the java files have copyright statements and the code has comments.

Other comments:
- do we need a build.xml file in the metadata.repository project?
- do we need the buildAntTasks.xml file in both projects?
- should we really remove the mirror-app shared launch configuration from the metadata.repository project?
- should put null checks in places. for instance, when we get a service from the service helper.

We should also consider shorter names for the Ant tasks. "eclipse.p2.CompositeMetadataRepository.create", etc are a little long.

What does the Ant code look like when calling these tasks? Did you consider creating a single Ant task for the mirror application and then have different sub-elements for things you want to do to it rather than creating lots of different tasks? Not sure which approach would be best...

Comment 6 Andrew Cattle CLA 2008-12-08 10:28:02 EST
(In reply to comment #5)
> We should also consider shorter names for the Ant tasks.
> "eclipse.p2.CompositeMetadataRepository.create", etc are a little long.
> 

The problem becomes that we have to have ant tasks for both CompositeArtifactRepositories and CompositeMetadataRepositories and they're slightly different for each other. I'm not sure if we can make the name descriptive enough if we make it shorter.

> What does the Ant code look like when calling these tasks? Did you consider
> creating a single Ant task for the mirror application and then have different
> sub-elements for things you want to do to it rather than creating lots of
> different tasks? Not sure which approach would be best...
> 

Pascal and I were trying to figure out if there was a way to merge all the composite tasks into a single task and then somehow call appropriate methods. However we couldn't figure out how to do this and we decided that for now the "building block" approach was best.

I like to keep the MirrorApplication tasks separate because I often find myself performing only artifact or only metadata. I could write a third mirror application task that does both but 1) what package would I put it in? and 2) the tasks are similar enough in most instances that invoking the second type fo mirroring is almost only a straight copy/paste away.

As for what the ant code looks like, I have several test scripts I created to ensure my tasks were correct. A simple script would look something like this:

<?xml version="1.0" encoding="UTF-8"?>
<project name="test" default="main">

	<target name="main">
		<eclipse.p2.CompositeArtifactRepository.create
			location="file:C:/ant_test"
		/>
		<eclipse.p2.CompositeArtifactRepository.addChild
			location="file:C:/ant_test"
			child="file:c:/foobar"
		/>
		<eclipse.p2.artifactMirrorApplication
			source="file:C:/ant_test"
			destination="file:c:/foobarMirror"
			verbose="true"
		/>
	</target>
</project>
Comment 7 Andrew Cattle CLA 2008-12-11 11:12:27 EST
Created attachment 120212 [details]
Update including copyright.

Added copyright information. Removed some unnecessary files. Modified the tasks to reflect changes introduced by Bug 254747.

Will most likely need to be updated again for Bug 257961.

I left the names as they were before because, as I said above, I'm not sure I can make them descriptive enough if they are any shorter. I am open to suggestions.
Comment 8 Andrew Cattle CLA 2008-12-18 14:25:28 EST
Created attachment 120876 [details]
Slight improvement over previous patch

I edited how I handle the writeMode in the mirror application tasks to make it easier to extend in the future.
Comment 9 Andrew Cattle CLA 2008-12-19 10:03:21 EST
Created attachment 120947 [details]
Missing Builders for previous patch
Comment 10 DJ Houghton CLA 2009-01-13 18:26:06 EST
Created attachment 122471 [details]
patch

New patch with added comments and bug fixes.
I will release to HEAD but keep this report open pending further testing.
Comment 11 DJ Houghton CLA 2009-01-19 17:25:26 EST
Added documentation to the wiki here: http://wiki.eclipse.org/Equinox/p2/Composite_Repositories

I talked to Kim and she is going to look at adding support in our builds.

Closing.