Community
Participate
Working Groups
Currently we have 2 update sites... the real one and the one which is produced by integration builds. This one is deleted every few days by a cron job. When a build is promoted from an i-build to a release, then the contents of the update site are manually copied to the real site, and the contents of the 2 site.xml files are merged by hand. Throw in a little metadata and artifact repository index and there are too many manual steps to ensure that we get it right. We should have a tool which will allow us to merge the contents of update sites. Note that this might be as simple as ensuring we can run the "mirroring application" in "append" mode instead of always creating a new site.
the mirror app already supports -append. we also need a way of trimming or "slicing" artifact repos to remove old unwanted stuff.
To start with the slicer could be implemented by using the resolver (or the planner). However depending on what the resulting slice is being used for, there may be problems due to the loss of content / context. The example that comes to mind is the bug #208599 where the slice we would not necessarily capture the problematic setup since the problem here is in the resolver itself. Of course this is only a chicken and egg pb until everything stabilize but still...
yeah, I'm also thinking that there is a case here for some sort of root set. That is, when adding a bunch of things to the repo, you might want to be able to remember them as a "collection/group" and then be able to remove them, mirror them, ... without having to do resolve etc. This may be another case the bill of materials kind of structure.
Adding Andrew to the CC since he is interested in this sort of thing. This was the bug report that I first opened when we were thinking about having composite repositories... the content.xml just had reference URLs to other files/sites.
Andrew, are you doing any work on this?
(In reply to comment #5) > Andrew, are you doing any work on this? I was planning on starting on this today.
Then I'm assigning to you :-)
Do you guys think that a content.xml with reference URLs is better than actually merging multiple repos into one? If we had the former, it would be nicer for me since I could somehow denote sub-sets of the "base".
After thinking about this some more, I've come to the conclusion that what I really need for the shared install scenario is the ability to merge repositories into one - or at least display multiple sub-repositories as components of a larger whole.
Created attachment 90422 [details] composite repository patch Here's a cut at a composite repository. It uses the "subRepos" directory. That name can be changed. I've tested manually with generated mylyn and sdk sub-directories (obviously with content.xml in both) and it seems to work. I get all IUs from both presented under the "Composite Repository" MR in the admin UI. There's still a FIXME or two and I'll write some tests as well. Suggestions and criticisms welcome :)
Created attachment 96877 [details] updated patch (2008-04-21) Here's an updated patch. I just morphed the constructor to match the new API.
AndrewC is going to take a look at composite repositories. Re-assigning...
Created attachment 116446 [details] Code for partially implemented Composite Metadata Repository Currently this is unable to write. Also I suspect that CompositeRepositoryFactory.java might be useless. I still need to implement Composite Artifact Repository as well as writers for both the artifact and metadata and make a few tweaks but it's a start.
Created attachment 117253 [details] Implementation and tests for composite repositories This implements composite repositories for both artifacts and metadata. By design they share the same parser, and thus the same xml syntax. See the following example: (repository information and properties) <children size="1"> <child location="URL of file's location. Will be encoded to URI"/> </children> Also note that CompositeArtifactRepository uses the file compositeArtifacts.xml and CompositeMetadataRepository similarly uses compositeContent.xml. Could the committer please verify the copyright information. Specifically in the CompositeArtifactRepository.java and CompositeMetadataRepository.java files. They were created from a previous attachement to this bug but were heavily modified to the point of being almost completely different. I didn't know what the copyright information should be.
If you've used none of my original code, feel free to remove Red Hat from the copyright statements.
Created attachment 117443 [details] Updated Composite Repo implementation Adds duplicate checking for artifact descriptors now that Bug 249035 is closed. Adds logging when a child repo fails to load (is this the desired behaviour?). Possible future improvement: prioritized children repositories (use faster connections first).
Created attachment 117594 [details] Minor updates to previous patch
Created attachment 117676 [details] Minor modifications to previous patch Repositories now assertModifiable before attempting to save. Updated copyright information (removed all non-IBM copyright information in files I heavily modified) Improved removeAll implementation And a few other minor changes.
Created attachment 117706 [details] Updated agian to reflect latest changes in HEAD Reflects releases for Bug 248269 and Bug 254955
Released. Thanks Andrew.
