Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 402063

Summary: [CBI] Generate Deployable build directory including repo
Product: [Eclipse Project] Platform Reporter: David Williams <david_williams>
Component: RelengAssignee: Platform-Releng-Inbox <platform-releng-inbox>
Status: RESOLVED DUPLICATE QA Contact:
Severity: normal    
Priority: P3 CC: david_williams, irbull, Lars.Vogel, pascal.rapicault, pwebster, tjwatson
Version: 4.2.1   
Target Milestone: ---   
Hardware: PC   
OS: Linux   
Whiteboard:
Bug Depends on: 402693    
Bug Blocks: 372792, 393922    
Attachments:
Description Flags
smallest diff output. Show's "what IUs are missing or extra"
none
shows "all" IUs that are differnt
none
diff "deep" comparison, of IUs with same id and version none

Description David Williams CLA 2013-02-28 16:08:08 EST
+++ This bug was initially created as a clone of Bug #393927 +++

One of the items I wanted to do first, before automatically "deploying" our Tycho/Maven repos was to look at the differences in previous I-build PDE repos. 

I'll use this bug to capture some of that output, and solicit comments, but I don't see anything in my untrained p2 eyes that is anything we didn't already know. But, experts with p2 might still spot some important differences that need to be fixed. 

But, bottom line, to me, is that there's nothing to prevent automatically publishing the repos. That those with an PDE I-build install will be able to update fairly normally ... perhaps just missing a few things (such as emf is not in our Tycho/Maven repos yet, so "updates" to EMF would not be found there). Some source missing .. not sure if important ones, or not ... lots to look at in this attachments. :/

I used the handy "p2diff" tool from Ian, 
http://eclipsesource.com/blogs/2012/10/10/introducing-p2diff/

And set up and ran on my local machine (which, I mention, since the URLs look very similar to "download.eclipse.org" ... but ... just symbolic links). 

These are three command lines for the different "diffs" I'll attach: 

./p2diff -query=all -mode=ignoreVersions -onlylatest file:///data/httpd/download.eclipse.org/eclipse/updates/4.3-I-builds/I20130219-1600/ file:///data/httpd/download.eclipse.org/eclipse/updates/4.3-I-builds/I20130226-0800/ > ~/temp/p2diffout.txt

./p2diff -query=all -mode=deep file:///data/httpd/download.eclipse.org/eclipse/updates/4.3-I-builds/I20130219-1600/ file:///data/httpd/download.eclipse.org/eclipse/updates/4.3-I-builds/I20130226-0800/ > ~/temp/p2diffdeepout.txt

./p2diff -query=all -mode=all file:///data/httpd/download.eclipse.org/eclipse/updates/4.3-I-builds/I20130219-1600/ file:///data/httpd/download.eclipse.org/eclipse/updates/4.3-I-builds/I20130226-0800/ > ~/temp/p2diffallout.txt
Comment 1 David Williams CLA 2013-02-28 16:10:57 EST
Created attachment 227755 [details]
smallest diff output. Show's "what IUs are missing or extra"
Comment 2 David Williams CLA 2013-02-28 16:13:07 EST
Created attachment 227756 [details]
shows "all" IUs that are differnt

Shows all IUs that are "different" ... does not really show which "increased" in versions (as they should) and which "decreased" ... would have to visually inspect.
Comment 3 David Williams CLA 2013-02-28 16:15:32 EST
Created attachment 227757 [details]
diff "deep" comparison, of IUs with same id and version

This one may be most interesting to the p2 experts. 

You'll have to ignore or filter out the "maven" stuff (as, that should be insignificant) ... but some others seem more important ... singleton, versus not a singleton? If my quick glance was correct.
Comment 4 David Williams CLA 2013-02-28 16:21:58 EST
Pascal, Ian, and Tom, 

I think if anyone would be interested in this "p2diff" output, you three would be. 

If you get a chance to study, please open bugs on anything "really bad" you see that needs to be corrected. I know you have already opened some, and some of the "obvious" differences are already listed in bug 393927 comment 2, but if you notice anything "deep", please say so. (That doesn't mean I'll have the first clue how to fix it ... but ... at least we'll know).
Comment 5 David Williams CLA 2013-03-02 13:16:43 EST
I happened across some bugs which explain the difference in "tooling" names. 

Bug 390203 - Duplicate IU error during building
which in turn was related to 
Bug 390184 - p2.inf files must match the product file

I'm assuming at worse this means someone could not reliably update from Juno to Kepler? But, otherwise, doubt if the exact name matters? Anyone know for sure what implications are?
Comment 6 David Williams CLA 2013-03-06 18:28:36 EST
My local test seemed good (better) with this fix to include EMF in our own repository, as we have been doing: 

http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?id=4624c63ee2e584ecc7b0794be5c015715d7b28c9

Will at least help us "aggregate cleanly", if not improve some of the packages? 

If this is "good" in master, the EMF fix should be back ported to 4.2.2+ but NOT to 3.8.2+.
Comment 7 David Williams CLA 2013-03-12 16:27:26 EDT
One of the issues was that we include a "master" category in our repository (in new CBI based build) whereas in old one, we did not include any categories ... but, "added" them later, via composite site). 

I'm going to go ahead and remove that for M6 because a) it reveals other problems, and b) gets us closer to the end-goal. 

http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?id=0137e658981b9a09f5c77e4b95aeb1cab5446512

Doing this, Tycho/Maven apparently creates its own "uncategorized" category, which we'll have to later remove, if Tycho doesn't fix. 

The things that show up under "uncategorized" are pretty ugly, but one of them might point out that we are including org.eclipse.pde.api.tools.ee.feature twice. I'll have to look closer and see if one is the older "custom category" feature that PDE contributed, in the past, or ... what.
Comment 8 Lars Vogel CLA 2014-05-14 18:44:11 EDT
As a note, this Bug seems to be the last bug clocking the super CBI build Bug 372792.
Comment 9 David Williams CLA 2014-05-14 19:31:34 EDT
not sure how we ended up with two of these ... but think bug 393927 has the concluding information.

*** This bug has been marked as a duplicate of bug 393927 ***