Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 361019

Summary:	3rd party dependency management using Git and Orbit
Product:	Community	Reporter:	Gunnar Wagenknecht <gunnar>
Component:	Architecture Council	Assignee:	eclipse.org-architecture-council
Status:	CLOSED MOVED	QA Contact:
Severity:	enhancement
Priority:	P3	CC:	david_williams, denis.roy, dj.houghton, glyn.normington, janet.campbell, john.arthorne, mike.milinkovich, mober.at+eclipse, wayne.beaton
Version:	unspecified
Target Milestone:	---
Hardware:	All
OS:	All
Whiteboard:

Description Gunnar Wagenknecht

2011-10-14 14:19:19 EDT

Based on a comment in bug 360994 I think we should streamline the process for managing 3rd party dependencies especially when migrating to Git.

Bug 360994 comment 0 says:
> [...] If a component must be removed later, this involves a non-trivial
> expenditure of effort on the part of both the project team and the webmaster
> team. [...]

A while back when the Git discussion started I proposed to not allow 3rd party dependencies at all in individual project Git repositories. Instead all should go to Orbit. 

This proposal was based on the assumption that Orbit would stay on CVS for some time and removing files from CVS is easy. If Orbit also migrates to Git than I propose to have a separate Git repository for every 3rd party package (which may contain multiple bundles, eg. a Lucene Git repo contains all and only Lucene bundles).

I'd like to bring this up on the AC table again for discussion. I think that we have room here for improving the process by streamlining and consolidation. If there is only one common place where all 3rd party dependencies are managed it will be easier to find, use and manage them. Separating the 3rd party packages from the project Git repos also reduces administrative overhead for webmasters.

When looking at bug 360994 I think that such a policy should be made mandatory for 3rd party packages approved under parallel IP. 

As an exception, only finally reviewed and approved 3rd party dependencies may be committed into a project Git repo. But I would even go so far and propose to not allow this at all.

Comment 1 Denis Roy

2011-10-14 15:04:31 EDT

I strongly +1 what is proposed here.

Actually, I would even commit to maintaining CVS indefinitely if it is used exclusively for Orbit 3rd party packages.  The overhead of maintaining CVS for a single project would likely pale compared to the ease of removing code that doesn't belong.

Also, I can't help but assume there is wasted disk space from the same 3rd party code appearing in project repos.

Comment 2 Janet Campbell

2011-10-14 15:07:37 EDT

+1

I like the idea of retaining all approved third party libraries in one central
location such as Orbit.  In addition to the benefits cited, it also reduces the
risk that a Project uses a version of a third party library that has not been
fully approved.

Janet

Comment 3 David Williams

2011-10-14 15:20:20 EDT

Allowing (or ... forcing?) Orbit to continue to use CVS lines up well with some discussion and conclusions in bug 349048. 

Forcing everyone to put 3rd party code into Orbit's CVS repository (even if not a "shared" 3rd party bundle) is a little different. 

In fact, I think there are some projects that have a slightly different version of an Orbit bundle in their repos, for some legacy reasons, never intended to be "shared" by others.  (Such as see CQ 3932). Some of this type of complication, though, could be handled by having a "non-shared" portion of Orbit's repo ... which would not be that much different that saying all projects should continue to use CVS for their (binary) 3rd party dependencies, if not in Orbit.

Comment 4 David Williams

2011-10-14 15:25:54 EDT

Adding Glyn, as I've heard Virgo has about 150 3rd party bundles "not in Orbit" ... so, they'd likely deserve (require?) similar "solution" as Orbit ... that is, keep binary 3rd party bundles in their own Virgo specific CVS repository. (I say this not even knowing where they currently are ... SVN? Git already?)

Comment 5 Wayne Beaton

2011-10-14 15:28:25 EDT

(In reply to comment #4)
> Adding Glyn, as I've heard Virgo has about 150 3rd party bundles "not in Orbit"
> ... so, they'd likely deserve (require?) similar "solution" as Orbit ... that
> is, keep binary 3rd party bundles in their own Virgo specific CVS repository.
> (I say this not even knowing where they currently are ... SVN? Git already?)

Virgo is entirely Git.

Comment 6 Gunnar Wagenknecht

2011-10-14 16:59:17 EDT

(In reply to comment #3)
> Forcing everyone to put 3rd party code into Orbit's CVS repository (even if not
> a "shared" 3rd party bundle) is a little different. 

Well, it's easier for webmaster to handle CVS when it comes to removing things. Why not centralize all this in one single place? That simplifies administration even further. I think it's also a big improvement for the Eclipse community at large. All 3rd party libraries, OSGi-ified bundles contributed and supported by all Eclipse projects in one single place.

> In fact, I think there are some projects that have a slightly different version
> of an Orbit bundle in their repos, for some legacy reasons, never intended to
> be "shared" by others. 

So there is still the IP process which requires projects to submit CQs. Thus, no immediate "re-use" will start. Projects still need to file CQs.

> (Such as see CQ 3932).

Frankly, I don't like that particular example. IMHO it represents a decision made by fear. The project didn't trust what was in Orbit. However, instead of reviewing, fixing and improving what was in Orbit (for the benefit of the whole community) it went away with creating its own copies. It surely was a win for the project. But I think the bigger win for the community is to help improving what is there and used by many.

Comment 7 Glyn Normington

2011-10-17 04:10:33 EDT

(In reply to comment #5)
> (In reply to comment #4)
> > Adding Glyn, as I've heard Virgo has about 150 3rd party bundles "not in Orbit"
> > ... so, they'd likely deserve (require?) similar "solution" as Orbit ... that
> > is, keep binary 3rd party bundles in their own Virgo specific CVS repository.
> > (I say this not even knowing where they currently are ... SVN? Git already?)
> 
> Virgo is entirely Git.

Correct.

Furthermore, Virgo uses Ant/Ivy to build, so none of its dependencies are checked into git. Ivy dependency definitions are checked in which reference the actual dependencies in a remote Ivy repository (which we control).

One inhibitor to Virgo using Orbit is that we would need to perform duplicate maintenance on Orbit bundles by publishing them to an Ivy repository so Virgo could reference them. So for dependencies which are new to Orbit, we would need to get them into Orbit *and* publish them to Ivy.

Comment 8 Frederic Gurr

2021-12-22 10:50:39 EST

This issue has been migrated to https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/143.