Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 355582

Summary:	Non-existent reference repo results in over-inclusion of "changed" bundles.
Product:	[Tools] Orbit	Reporter:	David Williams <david_williams>
Component:	releng	Assignee:	David Williams <david_williams>
Status:	RESOLVED FIXED	QA Contact:	Project Inbox <orbit.releng-inbox>
Severity:	normal
Priority:	P3
Version:	unspecified
Target Milestone:	---
Hardware:	PC
OS:	Linux
Whiteboard:

Description David Williams

2011-08-23 17:51:02 EDT

While creating the Indigo Maintenance build, I realized the "reference repository" for our Orbit build still pointed to some previous S build, which no longer exists on the file system, so, nothing to compare to, so the Orbit builds have been including everything as if "completely new" ... that is, even if qualifiers are exactly the same (but, content differs in minor ways, such as including a "SourceReference" directive, due to changes in build scripts, then the "new" bundle gets published, even though qualifier is exactly the same as the version without the qualifier. That's bad. And, wouldn't be too bad, except that some bundles such as these were likely published as part of Juno M1 ... so, if anyone is keeping a "cumulative repository", then they will have incorrect content (or, two bundles with same qualifier, with different content). 

So, 1) we should "fail build" if reference repository does not exist, and 
2) we need to sort out what to do about Juno M1; can we "move backwards? Or, should we just retag bundles so, all from Orbit would be "new", but at least "unique".

Comment 1 David Williams

2011-08-23 23:22:24 EDT

I'll leave this bug open as the place to put in a releng fix to fail if comparator repo does not exist, so we'd know to fix it. 

My initial inclination is not to do anything special about the fact that some minor changes (such as changes in "date signed") my have snuck into Juno M1. We can reconsider if issues noticed before Juno release, but it's far enough away I suspect it will all be "refreshed" and not cause any confusion or ambiguity in the wild.

Comment 2 David Williams

2011-08-23 23:24:23 EDT

Oh, and meant to say, I have fixed the reference repo "link" to point to a valid repo (the Indigo release) for both Juno and Indigo maintenance builds, so the spurious changes (such as "date signed") will no longer be picked up.

Comment 3 David Williams

2011-10-18 01:14:19 EDT

I've improved the build so it will "fail" if there is no repository where we say there should be one. This assume our reference repo is on the file system, not HTTP, even though HTTP would work with the comparator/mirror part of the build scripts. In practice, on "build.eclipse.org" we'd always be working from file system ... though on "local" test build, it may result in a "failed" message ... but, after all, a "local" build isn't really valid anyway (since not signed) so no great harm. If someone really cared about it, they could reproduce the reference repo on their local file system so it could be "found" on the file system.

It's important to emphasize we catch (fail) only pretty obvious errors (repo directory is missing). It is possible we are using the _wrong_ repository as reference, and this fix would still not "fail" the build. But, to help combat the wrong repo problem, I made it part of the download page display, so it will be easier to see (though, it is also listed in the comparator log).

Examples, if the repo path can not be found, the build will "fail" with a "failed" message sent to orbit-dev list as usual, and the download page will contain a line that sames something like

[red x] Bad build. Repo to use while mirroring was not found: /home/data/httpd/download.eclipse.org/tools/orbit/downloads/drops/ABCD/repository

where as if is does find a repo, the line would say something like

[check mark] Repo used for comparison during mirroring: /home/data/httpd/download.eclipse.org/tools/orbit/downloads/drops/R20110523182458/repository

So, again, it just means that repo exists ... might not be the _right_ one, but should be easier to notice.

Note, above, when I say the "build failed", the build actually does complete, and produces all the usual artifacts ... they would just not be quite the right ones if the wrong comparator repo was used (that is, might include "new" ones with same qualifier as old ones, but have some minor change like signed on different date, etc.).

I have put the change in the Juno stream builds and the maintenance (Indigo SR2) builds.