Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 342809

Summary: Move tycho to eclipse.org
Product: z_Archived Reporter: Jan Sievers <jan.sievers>
Component: TychoAssignee: Project Inbox <tycho-inbox>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: akurtakov, caniszczyk, gunnar, igor, matthias.sohn, pascal, t-oberlies, wayne.beaton
Version: unspecified   
Target Milestone: ---   
Hardware: All   
OS: All   
Whiteboard:
Attachments:
Description Flags
strict test-only dependencies
none
modified tycho build script which parses 'mvn dependency:tree' output
none
list of unique compile time dependencies
none
193602: modified tycho build script which parses 'mvn dependency:tree' output
none
list of artifacts which come with maven 3.0
none
list of artifacts which come with maven 3.0
none
strict test-only dependencies
none
dependency master list - g:a:v format
none
list of compile time dependencies
none
list of compile time dependencies
none
build script which generates list of compile-time dependencies
none
list of compile time dependencies (maven core artifacts removed)
none
list of artifacts which come with maven 3.0
none
list of compile time dependencies (maven core artifacts removed)
none
Details about how the JARs in the Tycho sources were processed for the initial contribution
none
list of compile time dependencies (maven core artifacts removed)
none
list of compile time dependencies (maven core artifacts removed)
none
list of runtime-only dependencies
none
strict test-only dependencies none

Description Jan Sievers CLA 2011-04-14 05:14:52 EDT
This is an umbrella bug to help coordinate all steps necessary to move the tycho codebase to the Eclipse Foundation.

Necessary steps:

- Move tycho groupIds and java packages to the org.eclipse.tycho.* namespace - DONE
- Add copyright headers to source files - DONE
- use eclipse.org mailing lists only - DONE
- No new bugs in https://issues.sonatype.org/browse/TYCHO , use bugs.eclipse.org only - DONE
- File CQs for 3rd party dependencies - OPEN
- Submit codebase to git.eclipse.org - OPEN

I probably forgot lots of things so feel free to add steps and report progress here.
Comment 1 Jan Sievers CLA 2011-04-14 05:16:56 EDT
leftovers for the file headers/copyright notices:

- Following java files have a non-EPL license header (these are all apache license):

maven-osgi-compiler-plugin/src/main/java/org/eclipse/tycho/osgicompiler/AbstractOsgiCompilerMojo.java
maven-osgi-compiler-plugin/src/main/java/org/eclipse/tycho/osgicompiler/OsgiCompilerMojo.java
maven-osgi-compiler-plugin/src/main/java/org/eclipse/tycho/osgicompiler/OsgiTestCompilerMojo.java
maven-osgi-compiler-plugin/src/main/java/org/eclipse/tycho/osgicompiler/copied/AbstractCompilerMojo.java
maven-osgi-compiler-plugin/src/main/java/org/eclipse/tycho/osgicompiler/copied/CompilationFailureException.java
maven-osgi-source-plugin/src/main/java/org/eclipse/tycho/source/AbstractSourceJarMojo.java
tycho-surefire/org.eclipse.tycho.surefire.junit/src/org/eclipse/tycho/surefire/junit/JUnitDirectoryTestSuite.java
tycho-surefire/org.eclipse.tycho.surefire.junit/src/org/eclipse/tycho/surefire/junit/JUnitTestSet.java
tycho-testing-harness/src/main/java/org/eclipse/tycho/testing/EmptyLifecycleExecutor.java

We will probably have to add a "3rd party about file template" [1] to these projects.
I propose we do this together with the 3rd party CQs for the dependencies.

- Following projects have checked-in binaries under apache license and will also need a 3rd-party about notice:

tycho-surefire/org.eclipse.tycho.surefire.junit
tycho-surefire/org.eclipse.tycho.surefire.junit4

[1] http://www.eclipse.org/legal/epl/about.php
Comment 2 Wayne Beaton CLA 2011-04-14 07:53:03 EDT
We need a break-down of the many third-party dependencies. 

Which of the dependencies are strictly for testing? (these can be lumped into a single WORKSWITH CQ)

Which of the dependencies are part of Maven? (these can be lumped into a single Exempt Pre-req CQ)

What's left over?
Comment 4 Wayne Beaton CLA 2011-04-18 10:57:41 EDT
(In reply to comment #3)
> see also
> https://docs.sonatype.org/display/TYCHO/Moving+Tycho+codebase+to+Eclipse+Foundation

To quote:

> At this point I do not know how to apply 
> http://www.eclipse.org/org/documents/Eclipse_Policy_and_Procedure_for_3rd_Party_Dependencies_Final.pdf 
> to Tycho dependencies brought via default lifecycle

I can help. But I need you to answer the questions from comment 2. Or is there something that makes providing an answer difficult?
Comment 5 Jan Sievers CLA 2011-04-18 11:23:12 EDT
(In reply to comment #4)
> (In reply to comment #3)
> > see also
> > https://docs.sonatype.org/display/TYCHO/Moving+Tycho+codebase+to+Eclipse+Foundation
> 
> To quote:
> 
> > At this point I do not know how to apply 
> > http://www.eclipse.org/org/documents/Eclipse_Policy_and_Procedure_for_3rd_Party_Dependencies_Final.pdf 
> > to Tycho dependencies brought via default lifecycle
> 
> I can help. But I need you to answer the questions from comment 2. Or is there
> something that makes providing an answer difficult?

I am currently working on finding out the strict test-only dependencies.

My kind of "black-box" approach is:

0. clean local maven repo
1. build tycho sources with test compilation and execution switched off, integration tests switched off; then scan local repo for jars
2. build tycho sources with test compilation and execution switched on, ITs switched on; then scan local repo for jars

The diff of the jar lists between 1. and 2. provides the stric test-only dependencies.

For the question

"Which of the dependencies are part of Maven?"

Unfortunaltely with maven being a plugin system with only a minimal kernel I am afraid this is not very well defined. Maybe we can ask it the other way round "which dependencies are originating from tycho dependency roots?"

My test for this would be to create an artificial project (packaging type pom) which configures all possible tycho plugins so that transitive dependency resolution for them is triggered.
Resulting local repo content is directly or transitively triggered by tycho dependencies.
The tricky case which Igor mentioned is that tycho also defines so-called "default build lifecycles" for each of its packaging types which declaratively reuse some of the standard maven plugins. So technically tycho does not have a compile dependency on these but maven will download and resolve them when you build a project with a tycho packaging type.

I will post the results of my local repo scans/diffs here as soon as available.
Comment 6 Igor Fedorenko CLA 2011-04-18 11:39:16 EDT
I think in this case "Maven" can be classified as any maven dependencies that has been cleared through Eclipse IP process as part of m2e, which you can see in http://git.eclipse.org/c/m2e/m2e-core.git/tree/org.eclipse.m2e.maven.runtime/jars .

Everything else will have to be submitted as separate new CQs, so I do not think it matters much if it is "maven" or "not maven".

We still need to know pure test dependencies, as these can use simplified IP clearance process as far as I understand.
Comment 7 Wayne Beaton CLA 2011-04-18 12:06:59 EDT
I don't think we need to restrict the definition of "Maven" to just those bits that are part of m2e. We may be able to include "Standard maven plugins" in our definition. Once we have the restructured list of dependencies, we can make a judgement call whether or not this avenue is worth pursuing with the IP team.

I am going to point the Technology PMC at this bug to initiate the public discussion required prior to declaring a Works-with or Exempt Pre-req.
Comment 8 Jan Sievers CLA 2011-04-18 12:09:01 EDT
Created attachment 193496 [details]
strict test-only dependencies

list of test-only dependencies obtained as diff from local maven repo content as described in comment #5 .
Comment 9 Igor Fedorenko CLA 2011-04-18 12:52:06 EDT
One this I forgot to mention, virtually all tycho dependencies and almost certainly all "maven" dependencies are of "works-with" variety, as version of *everything* tycho depends on can be overridden by the user in their pom.xml files. For example, even though Tycho is built against maven 3.0, most of the users will likely run one of more recent 3.0.x releases.
Comment 10 Tobias Oberlies CLA 2011-04-19 08:31:10 EDT
(In reply to comment #7)
> We may be able to include "Standard maven plugins" in our definition. 

Obviously, all the stuff that comes from the archive that users need to download when installing Maven is a pre-requisite. However I don't really know what you mean with "standard Maven plugins". Everything with groupId org.apache.maven.plugins? Or even all Maven plug-ins available from Maven central? Tycho aims to work well with all kinds of Maven plug-ins...

I don't think that this approach will take us very far. Instead we should focus on the following categories:

- Things that will be delivered via eclipse.org: I guess this could be almost exclusively the Tycho binaries - and (almost) no third party content. This would mean that you can't use Tycho without having Maven download additional artifacts from Maven central - but since Maven does this on its own, I don't think that the Tycho users will care.

- Things that the Tycho sources are compiled against. I guess we'll need individual CQs for that.

- Things that the Tycho tests are compiled against. This goes into one "workswith" CQ, right?

- Things that is used to build Tycho. Do we need anything for this at all? How did other projects built with Maven handle this?

The tricky question from my point of view is: How do we handle third party artifact that Tycho has a declarative dependency on? Tycho provides so called "packaging types" that combine a set of Maven plug-ins into something useful. Do we need CQs for everything that is pointed at through this mechanism?
Comment 11 Wayne Beaton CLA 2011-04-19 08:48:23 EDT
(In reply to comment #10)
> (In reply to comment #7)
> > We may be able to include "Standard maven plugins" in our definition. 
> 
> Obviously, all the stuff that comes from the archive that users need to
> download when installing Maven is a pre-requisite. However I don't really know
> what you mean with "standard Maven plugins". Everything with groupId
> org.apache.maven.plugins? Or even all Maven plug-ins available from Maven
> central? Tycho aims to work well with all kinds of Maven plug-ins...

I have no idea what "standard Maven plugins" means. I was parroting a previous comment.

> I don't think that this approach will take us very far. Instead we should focus
> on the following categories:
> 
> - Things that will be delivered via eclipse.org: I guess this could be almost
> exclusively the Tycho binaries - and (almost) no third party content. This
> would mean that you can't use Tycho without having Maven download additional
> artifacts from Maven central - but since Maven does this on its own, I don't
> think that the Tycho users will care.

Strictly, speaking, if Tycho needs something at runtime--regardless of whether or not we are directly distributing it--a CQ is required. We need a CQ of some form (pre-req, exempt pre-req, or works with) for every dependency. In the case of dynamic dependencies (e.g. some version range of a particular library), we can go the "works with" route.
 
> - Things that the Tycho sources are compiled against. I guess we'll need
> individual CQs for that.

This is likely a subset of the previous category.
 
> - Things that the Tycho tests are compiled against. This goes into one
> "workswith" CQ, right?

Yes. All "test" dependencies can be lumped together into a single works with.
 
> - Things that is used to build Tycho. Do we need anything for this at all? How
> did other projects built with Maven handle this?

In general, anything that is needed to build Tycho, but is not distributed and not generally considered a dependency of the code itself doesn't need a CQ. Essentially, this is stuff that lives on the build server exclusively.

> The tricky question from my point of view is: How do we handle third party
> artifact that Tycho has a declarative dependency on? Tycho provides so called
> "packaging types" that combine a set of Maven plug-ins into something useful.
> Do we need CQs for everything that is pointed at through this mechanism?

Can you give a "for instance"? I think these are "works with". i.e. we create a CQ for one of the packaging types.

I'm still waiting for the complete breakdown of the master dependency list
Comment 12 Jan Sievers CLA 2011-04-19 13:05:10 EDT
Created attachment 193602 [details]
modified tycho build script which parses 'mvn dependency:tree' output

Trying to figure out the list of compile time dependencies of tycho.

The attached script uses 'mvn dependency:tree' to list all transitive dependencies of tycho. The log output is filtered and parsed to come up with a list of unique dependencies.

Dependencies in the master list but not in this list must be coming in via

a) test scoped dependencies (see my earlier attachment)
b) runtime dependencies, e.g. via build lifecycle

If I get Wayne right this list is a candidate for the "prereq" type of dependencies.

Do you think this is a valid approach?
Comment 13 Jan Sievers CLA 2011-04-19 13:07:03 EDT
Created attachment 193603 [details]
list of unique compile time dependencies

this is the result of running the build script which parses dependency:tree output
Comment 14 Jan Sievers CLA 2011-04-19 17:36:41 EDT
Created attachment 193627 [details]
193602: modified tycho build script which parses 'mvn dependency:tree' output

fixed typo in script
Comment 15 Jan Sievers CLA 2011-04-19 18:32:09 EDT
Created attachment 193630 [details]
list of artifacts which come with maven 3.0

artifacts which are part of the maven 3.0 installation.
I.e. what you get when you unpack http://archive.apache.org/dist/maven/binaries/apache-maven-3.0-bin.tar.gz
Comment 16 Jan Sievers CLA 2011-04-19 19:19:47 EDT
I still need to reformat the attachments to use the same

groupId:artifactId:version

syntax so we can easily diff the files.

However we should have more data now to try to do the breakdown of dependencies.

In theory we should now have:

1. the master list of dependencies which is a superset of all following lists. It is obtained by running all tycho integration tests on an empty local repo and then scanning the local repo. Since the ITs also configure and use other maven plugins, this list is not strictly tycho-related and maybe we should think of a better way to generate this list (less pollution from unrelated maven plugins => smaller list)

2. list of dependencies used for testing only

3. list of dependencies which come with maven 3.0

4. list of compile-time dependencies (obtained via 'mvn dependency:tree')

I also still need to do some sanity checking whether 2-4 are actually subsets of 1 as expected and remove the intersection of 3 and 4 from 4.
Comment 17 Jan Sievers CLA 2011-04-20 05:04:47 EDT
Created attachment 193670 [details]
list of artifacts which come with maven 3.0

add 2 forgotten nekohtml artifacts
Comment 18 Jan Sievers CLA 2011-04-20 05:08:54 EDT
Created attachment 193671 [details]
strict test-only dependencies

converted to g:a:v format
Comment 19 Jan Sievers CLA 2011-04-20 05:50:14 EDT
Created attachment 193673 [details]
dependency master list - g:a:v format

list is obtained by running all tycho integration tests with empty local repo, then scanning the local repo for jars, removing tycho and eclipse artifacts, collapsing multiple versions of one artifact to the latest version only
Comment 20 Jan Sievers CLA 2011-04-20 05:55:38 EDT
Created attachment 193674 [details]
list of compile time dependencies

converted to g:a:v format
Comment 21 Jan Sievers CLA 2011-04-20 06:46:54 EDT
Created attachment 193681 [details]
list of compile time dependencies

- remove runtime/noaop flags
- remove duplicate version
Comment 22 Igor Fedorenko CLA 2011-04-20 07:39:44 EDT
Comment on attachment 193681 [details]
list of compile time dependencies

The list of compile-time dependencies can be trimmed further...

Out of all maven-scm-provider* artifacts, only maven-scm-provider-cvs* are actually used by Tycho mapfile import support. And, frankly, I am happy to remove mapfile import support altogether, so none of maven-scm* artifact would be necessary.

maven-plugin-testing-harness is only used by tycho integration tests

org.eclipse.jdt.core and org.eclipse.osgi are Eclipse artifacts, I do not believe we need CQs to use them
Comment 23 Wayne Beaton CLA 2011-04-20 10:02:30 EDT
(In reply to comment #22)
> Comment on attachment 193681 [details]
> list of compile time dependencies
> 
> The list of compile-time dependencies can be trimmed further...

How many of these compile-time dependencies are "just Maven"? I'm thinking that all the "org.apache.maven*" and most/all the plexus ones are "just Maven".

Let me put this another way... Which of these would be (potentially) loaded if a user (who already has Maven installed on their workstation) runs a Tycho-based build?

> Out of all maven-scm-provider* artifacts, only maven-scm-provider-cvs* are
> actually used by Tycho mapfile import support. And, frankly, I am happy to
> remove mapfile import support altogether, so none of maven-scm* artifact would
> be necessary.

Can we argue that these artifacts are all part of Maven (i.e. expected to be on the user's workstation when the run a Tycho-based build)?

> maven-plugin-testing-harness is only used by tycho integration tests

We need to lump this in with the "test" dependencies. Or, again, can we consider this part of Maven?
 
> org.eclipse.jdt.core and org.eclipse.osgi are Eclipse artifacts, I do not
> believe we need CQs to use them

Correct
Comment 24 Jan Sievers CLA 2011-04-20 10:19:02 EDT
(In reply to comment #23)
> > The list of compile-time dependencies can be trimmed further...
> How many of these compile-time dependencies are "just Maven"? I'm thinking that
> all the "org.apache.maven*" and most/all the plexus ones are "just Maven".
> Let me put this another way... Which of these would be (potentially) loaded if
> a user (who already has Maven installed on their workstation) runs a
> Tycho-based build?

We can simply do a diff on the two attached files "maven 3.0 artifacts" and "compile-time-deps". Those showing up in both come with a maven installation already.

> maven-scm-provider* artifacts
> Can we argue that these artifacts are all part of Maven (i.e. expected to be on
> the user's workstation when the run a Tycho-based build)?

they don't come with the maven installation. But since Igor proposed to remove the dependency, we should just remove it and get rid of all the maven-scm-* stuff.

> > maven-plugin-testing-harness is only used by tycho integration tests
> 
> We need to lump this in with the "test" dependencies. Or, again, can we
> consider this part of Maven?

tycho-testing-harness and its transitive deps should go into the "test-only" dependency list. Probably we have to comment out the tycho-testing-harness module when computing the compile-time deps.
Comment 25 Jan Sievers CLA 2011-05-03 08:52:34 EDT
Created attachment 194571 [details]
build script which generates list of compile-time dependencies

modified tycho build script which parses 'mvn dependency:tree' output and generates a g:a:v list of compile time dependencies
Comment 26 Jan Sievers CLA 2011-05-03 09:12:11 EDT
Created attachment 194574 [details]
list of compile time dependencies (maven core artifacts removed)

list of compile-time dependencies with maven core artifacts removed.
Here is wht I did to come up with this list:

- Remove dependency to maven-scm-client in tycho sources ( https://github.com/sonatype/sonatype-tycho/commit/bed5809ab5dcae90ca924096891f2c964e810712 )
- comment out tycho-testing-harness module as this belongs to test-only dependencies
- run the build script attached
- manually add
  org.apache.maven.surefire:surefire-junit:2.4.3
  org.apache.maven.surefire:surefire-junit4:2.4.3
  as these are not covered by the dependency:tree analysis
- remove all artifacts which already come with a maven 3.0 installation (see list of maven core artifacts attached)


If there are no vetos, I consider this list as well as the maven core artifacts list final.

@Wayne going forward, can we start with the maven core and compile time dependency lists now and finalize the test-only list and leftovers to complete the master list as we go? Or should we finalize all lists before we file any CQs.
Comment 27 Igor Fedorenko CLA 2011-05-03 09:25:45 EDT
org.sonatype.sisu:sisu-guice and org.codehaus.plexus:plexus-classworlds are part of maven core too. users always get them as part of maven install.


(In reply to comment #26)
> Created attachment 194574 [details]
> list of compile time dependencies (maven core artifacts removed)
> 
> list of compile-time dependencies with maven core artifacts removed.
> Here is wht I did to come up with this list:
> 
> - Remove dependency to maven-scm-client in tycho sources (
> https://github.com/sonatype/sonatype-tycho/commit/bed5809ab5dcae90ca924096891f2c964e810712
> )
> - comment out tycho-testing-harness module as this belongs to test-only
> dependencies
> - run the build script attached
> - manually add
>   org.apache.maven.surefire:surefire-junit:2.4.3
>   org.apache.maven.surefire:surefire-junit4:2.4.3
>   as these are not covered by the dependency:tree analysis
> - remove all artifacts which already come with a maven 3.0 installation (see
> list of maven core artifacts attached)
> 
> 
> If there are no vetos, I consider this list as well as the maven core artifacts
> list final.
> 
> @Wayne going forward, can we start with the maven core and compile time
> dependency lists now and finalize the test-only list and leftovers to complete
> the master list as we go? Or should we finalize all lists before we file any
> CQs.
Comment 28 Jan Sievers CLA 2011-05-03 09:42:31 EDT
Created attachment 194579 [details]
list of artifacts which come with maven 3.0

forgot sisu-guice (jar has no maven metadata). correct plexus-classworlds version.
Comment 29 Jan Sievers CLA 2011-05-03 09:44:37 EDT
Created attachment 194580 [details]
list of compile time dependencies (maven core artifacts removed)

removed suisu-guice and plexus-classworlds which come with maven
Comment 30 Wayne Beaton CLA 2011-05-03 15:40:38 EDT
I recommend that you:

1) Open a CQ for Maven; add a comment requesting that the CQ be considered an "exempt pre-req". Note that IPZilla will badger you to attach code. Just ignore those requests, the IP team will silence them.

2) Open a CQ for each of the artifacts listed in the compile time dependencies attachment.

> @Wayne going forward, can we start with the maven core and compile time
> dependency lists now and finalize the test-only list and leftovers to complete
> the master list as we go? Or should we finalize all lists before we file any
> CQs.

Works for me.

For completeness, the last bit will be to create a CQ for the test dependencies and request that it be considered a "works with" dependency.

(In reply to comment #7)
> I am going to point the Technology PMC at this bug to initiate the public
> discussion required prior to declaring a Works-with or Exempt Pre-req.

I will follow up with the Technology PMC to get explicit buy-in for the "exempt pre-req" and "works with" dependencies.
Comment 31 Tobias Oberlies CLA 2011-05-04 03:19:51 EDT
Wayne, when should we do the CQ for the Tycho code itself?
Comment 32 Gunnar Wagenknecht CLA 2011-05-04 04:57:33 EDT
(In reply to comment #31)
> Wayne, when should we do the CQ for the Tycho code itself?

AFAIK it can happen in parallel. Well, the IP team has some resource constraints. But I don't think that should you prevent from submitting the CQ. Wayne?
Comment 33 Wayne Beaton CLA 2011-05-04 07:26:55 EDT
(In reply to comment #32)
> (In reply to comment #31)
> > Wayne, when should we do the CQ for the Tycho code itself?
> 
> AFAIK it can happen in parallel. Well, the IP team has some resource
> constraints. But I don't think that should you prevent from submitting the CQ.
> Wayne?

Right. I knew I forgot something ;-)


0) create the CQ for the Tycho code as soon as you're ready.
Comment 34 Wayne Beaton CLA 2011-05-04 07:28:29 EDT
The Technology PMC approves of declaring Maven as an exempt pre-req.

http://dev.eclipse.org/mhonarc/lists/technology-pmc/msg03113.html

This link will have to be included on the corresponding CQ when it is created.
Comment 35 Igor Fedorenko CLA 2011-05-04 07:35:31 EDT
I need to double-check this, but I believe we can remove biz.aQute:bndlib and org.apache.maven.shared:maven-osgi dependencies. They are used by generate-bundle goal only, which does not belong in tycho core. I plan to either move generate-bundle to tycho-extras or remove altogether as it largely overlaps with maven-bundle-plugin.
Comment 36 Jan Sievers CLA 2011-05-06 08:33:44 EDT
opened CQs for all compile-time dependencies which are already in orbit (reusing existing orbit CQs):

CQ#  bundle
-------------------------------------------------------
5138 javax.servlet:2.5.0.v200910301333
5139 org.apache.commons.codec:1.3.0.v20100518-1140
5140 org.apache.commons.httpclient:3.1.0.v201005080502
5141 org.apache.commons.logging:1.1.1.v201005080502
5148 org.hamcrest.core:1.1.0.v20090501071000
5143 org.junit:junit.jar:3.8.2.v3_8_2_v20100427-1100
5142 org.junit:junit.jar:4.8.1.v4_8_1_v20100427-1100
5144 org.mortbay.jetty.server:6.1.23.v201004211559
5144 org.mortbay.jetty.util:6.1.23.v201004211559
5146 org.sat4j.core:2.2.0.v20100429
5146 org.sat4j.pb:2.2.0.v20100429
Comment 37 Jan Sievers CLA 2011-05-06 09:40:33 EDT
opened CQ 5149 for maven 3.0 as exempt-prereq as per Wayne's last comment
Comment 38 Tobias Oberlies CLA 2011-05-06 14:47:34 EDT
Created attachment 194971 [details]
Details about how the JARs in the Tycho sources were processed for the initial contribution

(In reply to comment #33)
> 0) create the CQ for the Tycho code as soon as you're ready.

I created the CQ for the Tycho sources: CQ 5150.

Also, I attached the Tycho sources from revision c3cf2e2 to the CQ. For this I had to remove the archive files in the sources (according to comment 1 in the CQ). We only have JAR files checked in, and only as test data. The JARs fall into one of three categories:

- Zero byte JAR files -> deleted; I guess we can simply re-create them once we have the sources at Eclipse
- Bundle/feature JARs from other Eclipse projects or Orbit -> deleted; we'll need to request the use of these files in the test CQ and then we should be able to re-add them
- JARs with test data -> unpacked into ${original-name}-unzipped; we'll need to JAR them back together for the initial commit (or as second commit) at Eclipse.

See attachment for a detailed log of the processing and the lists of affected files.
Comment 39 Jan Sievers CLA 2011-05-16 11:33:15 EDT
Created attachment 195744 [details]
list of compile time dependencies (maven core artifacts removed)

commits a2e95c and 0a74d2 remove compile-time dependencies to

bcel:bcel
biz.aQute:bndlib
org.apache.maven.shared:maven-osgi
Comment 40 Jan Sievers CLA 2011-05-16 11:45:07 EDT
(In reply to comment #36)
> opened CQs for all compile-time dependencies which are already in orbit
> (reusing existing orbit CQs):
> 
> CQ#  bundle
> -------------------------------------------------------
> 5138 javax.servlet:2.5.0.v200910301333
> 5139 org.apache.commons.codec:1.3.0.v20100518-1140
> 5140 org.apache.commons.httpclient:3.1.0.v201005080502
> 5141 org.apache.commons.logging:1.1.1.v201005080502
> 5148 org.hamcrest.core:1.1.0.v20090501071000
> 5143 org.junit:junit.jar:3.8.2.v3_8_2_v20100427-1100
> 5142 org.junit:junit.jar:4.8.1.v4_8_1_v20100427-1100
> 5144 org.mortbay.jetty.server:6.1.23.v201004211559
> 5144 org.mortbay.jetty.util:6.1.23.v201004211559
> 5146 org.sat4j.core:2.2.0.v20100429
> 5146 org.sat4j.pb:2.2.0.v20100429

added

CQ#  bundle
-------------------------------------------------------
5173 commons-io:commons-io:1.4
5174 commons-lang:commons-lang:2.1
Comment 41 Jan Sievers CLA 2011-05-17 09:48:22 EDT
Created attachment 195856 [details]
list of compile time dependencies (maven core artifacts removed)

commit c0922e8 replaces alpha versions of plexus-archiver and plexus-io with stable versions
Comment 42 Jan Sievers CLA 2011-05-17 10:40:40 EDT
opened CQs for all remaining compile-time dependencies (these are new, no reuse of old CQs possible):

CQ#  artifact
==================================================================
5178 de.pdark.decentxml:1.3
5179 org.apache.maven.plugins:maven-source-plugin:maven-plugin:2.1
5180 org.apache.maven.surefire:surefire-api:2.4.3
5181 org.apache.maven.surefire:surefire-booter:2.4.3
5182 org.apache.maven.surefire:surefire-junit4:2.4.3
5183 org.apache.maven.surefire:surefire-junit:2.4.3
5184 org.apache.maven:maven-archiver:2.4
5185 org.codehaus.plexus:plexus-archiver:1.2	
5186 org.codehaus.plexus:plexus-io:1.0.1
5187 org.codehaus.plexus:plexus-compiler-api:1.6
5188 org.codehaus.plexus:plexus-compiler-manager:1.6
Comment 43 Jan Sievers CLA 2011-05-24 07:01:37 EDT
to capture runtime-only dependencies coming in via default build lifecycle of tycho packaging types [1], I created an "empty" sample project for each of tycho's packaging types and ran 'mvn clean deploy' on these.

The content of the local maven repo after running this build yields the following runtime-only dependencies for which I created CQs:

CQ#  maven artifact
========================================================
5224 commons-cli:commons-cli:1.0
5225 junit:junit:3.8.1
5226 org.apache.maven.doxia:doxia-sink-api:1.0-alpha-7
5227 org.apache.maven.plugins:maven-clean-plugin:2.4.1
5228 org.apache.maven.plugins:maven-deploy-plugin:2.5
5229 org.apache.maven.plugins:maven-install-plugin:2.3.1
5230 org.apache.maven.plugins:maven-resources-plugin:2.4.3
5231 org.apache.maven.reporting:maven-reporting-api:2.0.6
5232 org.apache.maven.shared:maven-filtering:1.0-beta-4
5233 org.codehaus.plexus:plexus-digest:1.0
5234 org.codehaus.plexus:plexus-interactivity-api:1.0-alpha-4
5235 org.codehaus.plexus:plexus-interpolation:1.13
5236 org.codehaus.plexus:plexus-utils:1.5.6
5237 org.codehaus.plexus:plexus-utils:2.0.5
5238 org.sonatype.plexus:plexus-build-api:0.0.4

With this, there is only one CQ left to open for the test-only dependencies.
I will open a WORKSWITH CQ for these as Wayne proposed in comment #2.

---
[1] https://github.com/sonatype/sonatype-tycho/blob/master/tycho-maven-plugin/src/main/resources/META-INF/plexus/components.xml
Comment 44 Jan Sievers CLA 2011-05-24 07:03:39 EDT
Created attachment 196422 [details]
list of runtime-only dependencies

these are coming in via default build lifecycle for tycho packaging types:
https://github.com/sonatype/sonatype-tycho/blob/master/tycho-maven-plugin/src/main/resources/META-INF/plexus/components.xml
Comment 45 Tobias Oberlies CLA 2011-05-24 07:32:49 EDT
I have a question about Tycho's IP Log: Can we take patches from Bugzilla and mark them with iplog+ although our sources are still in review and not yet approved for parallel IP? Will this work technically? 

I think this would work from a legal perspective: With the patches being EPL-licensed through Bugzilla, we can collect the patches in our current non-Eclipse Git and copy the commits to the Eclipse Git later.
Comment 46 Wayne Beaton CLA 2011-05-24 08:00:02 EDT
(In reply to comment #45)
> I have a question about Tycho's IP Log: Can we take patches from Bugzilla and
> mark them with iplog+ although our sources are still in review and not yet
> approved for parallel IP? Will this work technically? 

Yes and yes.

> I think this would work from a legal perspective: With the patches being
> EPL-licensed through Bugzilla, we can collect the patches in our current
> non-Eclipse Git and copy the commits to the Eclipse Git later.

Agreed.
Comment 47 Jan Sievers CLA 2011-05-26 11:35:56 EDT
opened WORKSWITH CQ 5252 for all test-only dependencies
Comment 48 Jan Sievers CLA 2011-05-26 11:37:22 EDT
Created attachment 196674 [details]
strict test-only dependencies
Comment 49 Jan Sievers CLA 2011-05-26 11:38:55 EDT
All CQs should be created now - phew.
Comment 50 Tobias Oberlies CLA 2011-06-09 11:57:02 EDT
Just to let you all know: We are also planning to move the tycho-extras sources to Eclipse.

And one small question: Can we already start to set up our CI build at Eclipse although the Tycho sources are still at Github?
Comment 51 Wayne Beaton CLA 2011-06-09 12:04:31 EDT
(In reply to comment #50)
> And one small question: Can we already start to set up our CI build at Eclipse
> although the Tycho sources are still at Github?

Yes, as long as everything stays on the build server (which is not accessible by the general public). 

You cannot actually distribute any builds until the sources are approved by the IP team and hosted in an eclipse.org repository.
Comment 52 Tobias Oberlies CLA 2011-07-14 10:13:31 EDT
(In reply to comment #51)
> > And one small question: Can we already start to set up our CI build at Eclipse
> > although the Tycho sources are still at Github?
> 
> Yes, as long as everything stays on the build server (which is not accessible by
> the general public).

I've opened bug 352086 to track all activities around building Tycho at eclipse.org.
Comment 53 Jan Sievers CLA 2011-07-15 05:30:59 EDT
(In reply to comment #49)
> All CQs should be created now - phew.
As a follow-up in CQ 5150, created CQs 5375, 5376
Comment 54 Tobias Oberlies CLA 2011-07-15 07:09:29 EDT
(In reply to comment #47)
> opened WORKSWITH CQ 5252 for all test-only dependencies

Wayne, do we need CQs for test-only dependencies to Eclipse projects? For example, we have a couple of bundles like org.eclipse.core.jobs checked in in our sources as test data. I had removed them for the Tycho source CQ - but are we allowed to check them back in at eclipse.org, or do we need another WORKSWITH CQ for them?
Comment 55 Wayne Beaton CLA 2011-07-15 12:31:09 EDT
(In reply to comment #54)
> (In reply to comment #47)
> > opened WORKSWITH CQ 5252 for all test-only dependencies
> 
> Wayne, do we need CQs for test-only dependencies to Eclipse projects? For
> example, we have a couple of bundles like org.eclipse.core.jobs checked in in
> our sources as test data. I had removed them for the Tycho source CQ - but are
> we allowed to check them back in at eclipse.org, or do we need another
> WORKSWITH CQ for them?

If you're checking in copies of code from another Eclipse project, then we need a CQ to track that. Essentially, we treat it as a fork.

Is there some reason why you can't just obtain the code from the host project?
Comment 56 Tobias Oberlies CLA 2011-07-19 11:44:26 EDT
(In reply to comment #55)
> If you're checking in copies of code from another Eclipse project, then we need
> a CQ to track that. Essentially, we treat it as a fork.
> 
> Is there some reason why you can't just obtain the code from the host project?

We don't check in the code, but binaries (i.e. bundles). The binaries are used for testing Tycho, and for the sake of stability and speed, we try to avoid remote connections during tests.
Comment 57 Gunnar Wagenknecht CLA 2011-07-19 14:40:31 EDT
(In reply to comment #56)
> We don't check in the code, but binaries (i.e. bundles). The binaries are used
> for testing Tycho, and for the sake of stability and speed, we try to avoid
> remote connections during tests.

FWIW, if you run the tests on Eclipse.org servers you have local file system access to source + downloads.
Comment 58 Jan Sievers CLA 2011-08-03 07:51:56 EDT
filed bug 353741 for git repo creation.
we can then go ahead and check in the initial contribution as of CQ5150.
Comment 59 Tobias Oberlies CLA 2011-08-04 05:30:20 EDT
(In reply to comment #57)
> FWIW, if you run the tests on Eclipse.org servers you have local file system
> access to source + downloads.
But then this would prevent running the tests locally.

I'll see if I can somehow download the test data during the build, but if it is too much effort, I'll just file the CQs for checking in the binaries.
Comment 60 Jan Sievers CLA 2011-09-22 11:45:59 EDT
After 59 comments on this bug and 43 CQs opened, it's done.

I just pushed our initial contribution to 

http://git.eclipse.org/c/tycho/org.eclipse.tycho.git/

This is an important step forward for the Tycho project and I would like to thank everybody who helped making this happen.
Special thanks to Wayne and Sharon from the IP team who have been very supportive throughout this whole process.

Regards,
Jan