Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 349267

Summary: Investigate and consider Indigo respin for file handle regression
Product: Community Reporter: David Williams <david_williams>
Component: Cross-ProjectAssignee: David Williams <david_williams>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: achilles.sabine, bluesoldier, Chokri.Mraidha, christian.campo, daniel_megert, dennis.huebner, g.watson, gunnar, hmalphettes, holger.staudacher, kim.moir, mknauer, pwebster, rsternberg, stephan.herrmann, tjwatson, wayne.beaton, yjiang
Version: unspecified   
Target Milestone: ---   
Hardware: PC   
OS: Linux   
Whiteboard:
Attachments:
Description Flags
console log from the original RC4 run.
none
log from RC4 re-run, prior to any new contributions from Eclipse Platform, etc.
none
console log from latest (midnight) run
none
log from latest aggregation, showing Gyrex added none

Description David Williams CLA 2011-06-14 02:01:03 EDT
As has been mentioned recently on cross-project list, there's some consideration to respin the Indigo common repository, and EPP packages ... with, of course, the fix itself in the org.eclipse.osgi bundle, as described in bug 349105. 

A couple of considerations have been mentioned to investigate, to see if we could respin and if other projects are effected and if they can react. 

So, this bug is document those issues (not the original bug).
Comment 1 David Williams CLA 2011-06-14 02:06:20 EDT
One step in the investigation, was to respin the Indigo common repository aggregation, and see if anything has changed. 

I've done that and, judging from the logs created during the run, only the PTP project has new changes showing up. I'm not sure if these are "early maintenance" (in which case, something will have to be reverted, to get the intended released features versions listed explicitly in the b3aggrcon file. 

I will attached the original log from the RC4 run, and a run from this evening. by importing these into an Eclipse IDE, it's fairly easy to "compare with ..." each other, and see there is not much difference. A few dates, and job numbers, and then many of the PTP features and bundles. Some of the differences MIGHT be simply "order effects" in the listing ... but I did look directly at some patching features and can see different version numbers list.
Comment 2 David Williams CLA 2011-06-14 02:07:04 EDT
Created attachment 197937 [details]
console log from the original RC4 run.
Comment 3 David Williams CLA 2011-06-14 02:07:59 EDT
Created attachment 197938 [details]
log from RC4 re-run, prior to any new contributions from Eclipse Platform, etc.
Comment 4 David Williams CLA 2011-06-14 02:10:24 EDT
So ... 

1. We'll need to hear from PTP project if they need to revert something ... or if the currently picked-up features/bundles are what they intend to release for Indigo. 

and then, eventually, 

2. need to hear from RAP runtime team (and, possibly others) to see if/when they can react to platform change in versioning for org.eclipse.osgi bundle and any features it's contained in (directly, or indirectly).
Comment 5 Christian Campo CLA 2011-06-14 02:36:48 EDT
Riena is planing on delivering a new build anyway today or early tomorrow morning. So I am fine with any direction this takes. However the Riena p2 repo only contains equinox.core.net and equinox.core.variables so we are probably not affected.
Comment 6 Markus Knauer CLA 2011-06-14 03:14:43 EDT
(In reply to comment #0)
> As has been mentioned recently on cross-project list, there's some
> consideration to respin the Indigo common repository, and EPP packages ...

I think this bug is severe enough to justify a respin of the EPP packages. I'll watch this bug and as soon as the fix is available from /releases/staging I'll (manually!) start a rebuild of the packages.
Comment 7 Markus Knauer CLA 2011-06-14 03:20:50 EDT
We should be *very* careful. This bug is meant to fix bug 349105, it is not the wild card for new submissions to the common repo.
Comment 8 Hugues Malphettes CLA 2011-06-14 05:37:07 EDT
Jetty will need to respin: we publish an equinox based product.
As a consequence we do need to respin everytime there is a new version of equinox (I wish there was a way to loosely require equinox and the p2 bundles in the product files instead of embedded them but this is certainly out of the scope of this bug).

It is no problem for jetty to make a new build once platform is ready.
Comment 9 Greg Watson CLA 2011-06-14 07:52:06 EDT
PTP changed some packages in preparation for a bug fix release, but we would be fine if these get released with Indigo. We're happy to go either way.
Comment 10 Kim Moir CLA 2011-06-14 09:08:29 EDT
I submitted I20110613-1736 to the Indigo build.
Comment 11 David Williams CLA 2011-06-14 11:37:28 EDT
(In reply to comment #9)
> PTP changed some packages in preparation for a bug fix release, but we would be
> fine if these get released with Indigo. We're happy to go either way.

It is up to you. Normally, I'd scold you about not maintaining a constant repository, etc., ... but, I'm sure you know. In this case, if you are _completely confident_ that what is there is safe, stable, etc. then it is probably less risk (and less work for you) to simply leave current content. But, please, leave constant now, don't try to update it to sneak in a few more bug fixes. On the other hand, if it's simply a routine or experimental build, and would need lots of testing to confirm its safe, then it might be easier for you (and less risky) to revert to the previously expected versions. And remember, all the usual requirements are still required ... signed, pack'd files, etc. Maybe you do that automatically now for each build/repository you produce, but thought I'd mention it to cover all basses.
Comment 12 Holger Staudacher CLA 2011-06-14 11:39:00 EDT
Hi David,
today is the best day for this ;) Ralf is on vacation and I'm traveling for the democamps ;)

Anyway, I kick off another build and we will publish the repository tomorrow latest. Is this enough time?

I assume that the rebuild is located in the drops: ...downloads/eclipse/downloads/drops/I20110613
Comment 13 David Williams CLA 2011-06-14 11:45:14 EDT
(In reply to comment #10)
> I submitted I20110613-1736 to the Indigo build.


Thanks Kim. This failed with the expected RAP Runtime clash, so we are waiting to hear from them, if/when they can re-contribute. 

Others saying they need to respin: 
Gyrex, 
Jetty, 
Riena

Please report here status and outlook, so we we all know.
Comment 14 David Williams CLA 2011-06-14 11:50:08 EDT
(In reply to comment #12)

> 
> Anyway, I kick off another build and we will publish the repository tomorrow
> latest. Is this enough time?

Sooner the better, but we'll wait. But, I think I will disable RAP Runtime then, and let the aggregator crunch its bits as best it can, just to see if there are other problems. 

> 
> I assume that the rebuild is located in the drops:
> ...downloads/eclipse/downloads/drops/I20110613

Maybe, but their "repo" contributed to Indigo Aggregator is called 
eclipse/updates/3.7milestones/S-3.7RC5-201106131736
Comment 15 Kim Moir CLA 2011-06-14 11:57:36 EDT
Holger

The build is http://download.eclipse.org/eclipse/downloads/drops/I20110613-1736

David, that's the same build I contributed to Indigo
Comment 16 Greg Watson CLA 2011-06-14 12:02:12 EDT
(In reply to comment #11)
> (In reply to comment #9)
> > PTP changed some packages in preparation for a bug fix release, but we would be
> > fine if these get released with Indigo. We're happy to go either way.
> 
> It is up to you. Normally, I'd scold you about not maintaining a constant
> repository, etc., ... but, I'm sure you know. In this case, if you are
> _completely confident_ that what is there is safe, stable, etc. then it is
> probably less risk (and less work for you) to simply leave current content.
> But, please, leave constant now, don't try to update it to sneak in a few more
> bug fixes. On the other hand, if it's simply a routine or experimental build,
> and would need lots of testing to confirm its safe, then it might be easier for
> you (and less risky) to revert to the previously expected versions. And
> remember, all the usual requirements are still required ... signed, pack'd
> files, etc. Maybe you do that automatically now for each build/repository you
> produce, but thought I'd mention it to cover all basses.

These bug fixes are important enough that we'd prefer to leave them in. We were
planning to release a bug fix version at the same time as Indigo because of
this, but it shouldn't be required any longer.
Comment 17 David Williams CLA 2011-06-14 15:07:03 EDT
Status as of 3 PM Eastern. 
The respin succeeded ... minus RAP Runtime and Gyrex. 

I'd prefer to wait to be more complete, and more "ready" confirmations here, before promoting a new "staging" release candidate repo ... but anyone (esp. Markus :) can tell me you'd prefer a new staging repo right now. 

Thanks everyone .... so far.
Comment 18 Stephan Herrmann CLA 2011-06-14 15:38:09 EDT
We in Object Teams have a regression that was detected just yesterday.
see bug 349247.

It is a simple NPE that makes a typical use case impossible.
It's not a stop-ship per se, but an embarrassing regression.

We would be grateful if we get permission to contribute a new build
to the next aggregator run. Our build is ready, I'm just waiting
for the download of SDK RC 5 for final testing.

The fix is low risk, a trivial null check, and affects only one feature
and one of its contained bundles. No EPP package is affected.

Naturally, I'll always keep the last successful contribution, and
promise to revert should the re-contribution cause any issues.

Please let me know if you have any objections.
Comment 19 Markus Knauer CLA 2011-06-14 15:49:01 EDT
(In reply to comment #17)
> The respin succeeded ... minus RAP Runtime and Gyrex. 

I prefer to wait for the RAP contribution based on the new Equinox build, because it is included in the RCP/RAP package. Without this contribution it is not possible to build this package anyway. (And I know first hand that the RAP team is working on it...)
Comment 20 Christian Campo CLA 2011-06-14 16:28:41 EDT
As posted on the rt-pmc mailing list yesteday, we found a problem in Riena which we just fixed and supply a new build for. The build is already uploaded, I will switch versions in the riena.build file and monitor the aggregation in indigo. If anything goes wrong it easy to just switch back.

Riena is not in the EPP packages as far as I know. So we are "just" contributing to the composite repo.
Comment 21 Ralf Sternberg CLA 2011-06-14 18:31:30 EDT
I just re-built and re-contributed RAP to the aggregator.
Comment 22 David Williams CLA 2011-06-14 18:51:43 EDT
Status as of 6:50 Eastern ... I stopped the build and restarted it, to pickup the rap contribution. 

So, it that it? All in? 

I've lost track :) but this will be out last spin ... unless someone asks for me to restart it?
Comment 23 David Williams CLA 2011-06-14 19:02:03 EDT
status as of 7:00. That didn't take long :( 

It was me that re-enabled Gyrex, thinking it was ready ... but, guess not, since "conflict" reported with Gyrex in there. I'm assuming we'll hear from Gunnar soon, but have temporarily disabled it, again.
Comment 24 Hugues Malphettes CLA 2011-06-14 19:48:24 EDT
On the jetty side it is early Wednesday morning here.
As long as jetty does not crash the aggregator I would say that the version of the jars from equinox that were are embedding have not changed from platform-RC4 to platform-RC5.

I am checking on that manually.
I have restarted a build for jetty against the RC5 of the platform repo.
Comment 25 Hugues Malphettes CLA 2011-06-14 19:52:22 EDT
(In reply to comment #24)
> I would say that the version of
> the jars from equinox that were are embedding have not changed from
> platform-RC4 to platform-RC5.
> 
> I am checking on that manually.
Well, jetty-RC4 publishes RC4's org.eclipse.osgi_3.7.0.v20110524
But platform-RC5 publishes org.eclipse.osgi_3.7.0.v20110613

> I have restarted a build for jetty against the RC5 of the platform repo.
I'll:
- wait for the jetty-RC5 build to complete
- prepare the updated contribution
- wait for confirmation here that we should contribute jetty-RC5
Comment 26 David Williams CLA 2011-06-14 19:59:13 EDT

> - wait for confirmation here that we should contribute jetty-RC5

Confirmed. We do still want to be coordinated :) 

Just post a note here when you have re-contributed, and we'll crunch the bits through the aggregator to get all the latest. 

Thanks,
Comment 27 Hugues Malphettes CLA 2011-06-14 20:28:42 EDT
(In reply to comment #26)
> 
> > - wait for confirmation here that we should contribute jetty-RC5
> 
> Confirmed. We do still want to be coordinated :) 
> 
Thanks, jetty-RC5 is contributed.
Comment 28 David Williams CLA 2011-06-14 22:42:38 EDT
Status 10:40 PM, 6/14 (Eastern). 

All built (aggregated) cleanly, except for the fact that gyrex has been disabled, 'till Gunnar can tell us he's ready to respin. 

If I don't hear anything soon, (e.g. within 30 minutes) I'll go ahead and promote what we have to 'staging', and we can decide later, once gyrex is 'in' if we need to recreate EPP packages, etc.
Comment 29 David Williams CLA 2011-06-15 00:08:22 EDT
Status midnight, 6/14 (Eastern). 

I did promote, even without Gyrex, just to take another step forward. 

Another issue ... I hate to be picky ... is that now there is a file that says it is missing an about.html file. See 

http://build.eclipse.org/indigo/simrel

Missing about.html in file: org.pushingpixels.trident_1.2.0.v20100204-1500.jar

How could that happen?
Comment 30 David Williams CLA 2011-06-15 00:10:06 EDT
Created attachment 197997 [details]
console log from latest (midnight) run

From what I can tell from a quick scan, all the projects expecting to be changed in this re-run did show up as different in the console log. I'm attaching here, just if anyone's interested.
Comment 31 David Williams CLA 2011-06-15 01:27:44 EDT

> 
> Missing about.html in file: org.pushingpixels.trident_1.2.0.v20100204-1500.jar
> 
> How could that happen?

From what I can tell, there must be two version of this jar, with exact same qualifier, one with about.html, one without ... and there's probably some element of chance as to which p2/aggregator picks. 

org.pushingpixels.trident_1.2.0.v20100204-1500.jar

and now, in the respin, there is even an additional, new version, that was not there in RC4, apparently with the about.html file. 

org.pushingpixels.trident_1.2.0.201106090835.jar

Sigh ...
Comment 32 Christian Campo CLA 2011-06-15 01:35:42 EDT
pushingpixels.trident is a file Riena is contributing. since it had a license issue in the past, we had e4 (where its coming from) fix it and we are just contributing org.pushingpixels.trident_1.2.0.201106090835.jar

org.pushingpixels.trident_1.2.0.v20100204-1500.jar is what we had in RC4. The difference between the two tags is just the about.html file.

I guess I assumed that the new Riena build would overwrite the old RC4 content. But thats probably only true if the filename is the same.

So what do we do ?


(In reply to comment #31)
> 
> > 
> > Missing about.html in file: org.pushingpixels.trident_1.2.0.v20100204-1500.jar
> > 
> > How could that happen?
> 
> From what I can tell, there must be two version of this jar, with exact same
> qualifier, one with about.html, one without ... and there's probably some
> element of chance as to which p2/aggregator picks. 
> 
> org.pushingpixels.trident_1.2.0.v20100204-1500.jar
> 
> and now, in the respin, there is even an additional, new version, that was not
> there in RC4, apparently with the about.html file. 
> 
> org.pushingpixels.trident_1.2.0.201106090835.jar
> 
> Sigh ...
Comment 33 Markus Knauer CLA 2011-06-15 02:00:39 EDT
(In reply to comment #29)
> I did promote, even without Gyrex, just to take another step forward. 

I started a full EPP build.
Comment 34 Markus Knauer CLA 2011-06-15 02:12:38 EDT
(In reply to comment #33)
> (In reply to comment #29)
> > I did promote, even without Gyrex, just to take another step forward. 
> 
> I started a full EPP build.

New packages will appear soon here:
http://build.eclipse.org/technology/epp/epp_build/indigo/download/20110615-0608/
Comment 35 Gunnar Wagenknecht CLA 2011-06-15 02:22:33 EDT
(In reply to comment #28)
> Status 10:40 PM, 6/14 (Eastern). 
> 
> All built (aggregated) cleanly, except for the fact that gyrex has been
> disabled, 'till Gunnar can tell us he's ready to respin. 

Good Morning and sorry for the delay. I didn't take the RAP and Jetty re-spin into account and submitted our build too early. I just updated our target platform and started a new build.


(In reply to comment #32)
> So what do we do ?

The thing is:
  1.2.0.v20100204-1500  >  1.2.0.201106090835

I think if you change the qualifier to also start with a 'v' then p2 should pick up the corrected version because it would be "higher".
Comment 36 David Williams CLA 2011-06-15 02:40:48 EDT
> (In reply to comment #32)
> > So what do we do ?
> 
> The thing is:
>   1.2.0.v20100204-1500  >  1.2.0.201106090835
> 
> I think if you change the qualifier to also start with a 'v' then p2 should
> pick up the corrected version because it would be "higher".

I think it is a little bit more complicated than that, I don't think reina is "contributing old and new" versions. From a quick scan of the log, it appears papyrus also uses this jar ... and indeed, the one in their repository appears to have no about.html file. 

/home/data/httpd/download.eclipse.org/modeling/mdt/papyrus/updates/milestones/0.8
 
$ find ./ -name org.pushingpixels.trident*
./plugins/org.pushingpixels.trident_1.2.0.v20100204-1500.jar.pack.gz
Comment 37 Christian Campo CLA 2011-06-15 03:25:32 EDT
To make  things worse, their is only the CQ from e4 and the piggyback from "Riena" for the trident lib (and a withdrawn CQ from windowbuilder). I have sent an email to Wayne, so he can double check this.

(In reply to comment #36)
> > (In reply to comment #32)
> > > So what do we do ?
> > 
> > The thing is:
> >   1.2.0.v20100204-1500  >  1.2.0.201106090835
> > 
> > I think if you change the qualifier to also start with a 'v' then p2 should
> > pick up the corrected version because it would be "higher".
> 
> I think it is a little bit more complicated than that, I don't think reina is
> "contributing old and new" versions. From a quick scan of the log, it appears
> papyrus also uses this jar ... and indeed, the one in their repository appears
> to have no about.html file. 
> 
> /home/data/httpd/download.eclipse.org/modeling/mdt/papyrus/updates/milestones/0.8
> 
> $ find ./ -name org.pushingpixels.trident*
> ./plugins/org.pushingpixels.trident_1.2.0.v20100204-1500.jar.pack.gz
Comment 38 Markus Knauer CLA 2011-06-15 06:42:04 EDT
(In reply to comment #34)
> (In reply to comment #33)
> > I started a full EPP build.
> 
> New packages will appear soon here:
> http://build.eclipse.org/technology/epp/epp_build/indigo/download/20110615-0608/

EPP Build 20110615-0608-indigo-RC5 is ready, see URL above.
Comment 39 Wayne Beaton CLA 2011-06-15 09:18:08 EDT
(In reply to comment #37)
> To make  things worse, their is only the CQ from e4 and the piggyback from
> "Riena" for the trident lib (and a withdrawn CQ from windowbuilder). I have
> sent an email to Wayne, so he can double check this.

Papyrus may be using this library indirectly through some eclipse.incubator.e4 bundles. I need to take a harder look to make sure that the right set of piggyback CQs are in place. In any case, there is no IP exposure as the library has been IP cleared (we may still need a piggyback CQ for tracking purposes).

The missing about.html file is worrisome, however, and really does need to be there.
Comment 40 David Williams CLA 2011-06-15 09:21:06 EDT
Lets see if Papyrus can respin, to pickup/use a "corrected" org.pushingpixels.trident jar. 

And, Christian, I suggest you respin to use a qualifer > 1.2.0.v20100204-1500. Otherwise, I fear, there might be some chance your users (or others) _might_ end up getting the one without the about.html file? (depending). 

And ... can ?someone?, open an Orbit bug/and cq to add to Orbit for next development cycle? That wouldn't automatically solve current sort of problem, but would make it easier to solve. 
 



(In reply to comment #37)
> To make  things worse, their is only the CQ from e4 and the piggyback from
> "Riena" for the trident lib (and a withdrawn CQ from windowbuilder). I have
> sent an email to Wayne, so he can double check this.
> 
> (In reply to comment #36)
> > > (In reply to comment #32)
> > > > So what do we do ?
> > > 
> > > The thing is:
> > >   1.2.0.v20100204-1500  >  1.2.0.201106090835
> > > 
> > > I think if you change the qualifier to also start with a 'v' then p2 should
> > > pick up the corrected version because it would be "higher".
> > 
> > I think it is a little bit more complicated than that, I don't think reina is
> > "contributing old and new" versions. From a quick scan of the log, it appears
> > papyrus also uses this jar ... and indeed, the one in their repository appears
> > to have no about.html file. 
> > 
> > /home/data/httpd/download.eclipse.org/modeling/mdt/papyrus/updates/milestones/0.8
> > 
> > $ find ./ -name org.pushingpixels.trident*
> > ./plugins/org.pushingpixels.trident_1.2.0.v20100204-1500.jar.pack.gz
Comment 41 Christian Campo CLA 2011-06-15 09:26:24 EDT
Just to be clear…..

You want me in the Riena feature to require a version higher than 1.2.0.v20100204-1500 for trident so I pick the right version ?

Because I am only contributing the right version into the repo with the about.html. Once the papyrus thing is respun and the invalid plugin is erased we would be good with any respin from my side ?


(In reply to comment #40)
> Lets see if Papyrus can respin, to pickup/use a "corrected"
> org.pushingpixels.trident jar. 
> 
> And, Christian, I suggest you respin to use a qualifer > 1.2.0.v20100204-1500.
> Otherwise, I fear, there might be some chance your users (or others) _might_
> end up getting the one without the about.html file? (depending). 
> 
> And ... can ?someone?, open an Orbit bug/and cq to add to Orbit for next
> development cycle? That wouldn't automatically solve current sort of problem,
> but would make it easier to solve. 
> 
> 
> 
> 
> (In reply to comment #37)
> > To make  things worse, their is only the CQ from e4 and the piggyback from
> > "Riena" for the trident lib (and a withdrawn CQ from windowbuilder). I have
> > sent an email to Wayne, so he can double check this.
> > 
> > (In reply to comment #36)
> > > > (In reply to comment #32)
> > > > > So what do we do ?
> > > > 
> > > > The thing is:
> > > >   1.2.0.v20100204-1500  >  1.2.0.201106090835
> > > > 
> > > > I think if you change the qualifier to also start with a 'v' then p2 should
> > > > pick up the corrected version because it would be "higher".
> > > 
> > > I think it is a little bit more complicated than that, I don't think reina is
> > > "contributing old and new" versions. From a quick scan of the log, it appears
> > > papyrus also uses this jar ... and indeed, the one in their repository appears
> > > to have no about.html file. 
> > > 
> > > /home/data/httpd/download.eclipse.org/modeling/mdt/papyrus/updates/milestones/0.8
> > > 
> > > $ find ./ -name org.pushingpixels.trident*
> > > ./plugins/org.pushingpixels.trident_1.2.0.v20100204-1500.jar.pack.gz
Comment 42 David Williams CLA 2011-06-15 10:04:19 EDT
I was suggesting that, but agree it's debatable. Would not be good if caused churn elsewhere (e.g. if Gyrex had to respin?). 

But, generally, we _always_ want _all_ our bundles to have increasing qualifiers/versions when they change. 

And, I'll confess ... this is open development ... I was thinking/hoping if your bundle increased version/qualifier like it should, to be greater than 1.2.0.v20100204-1500, then your version _might_ be used by the Papyrus requirement, hence solving the "common repo" problem. 

It would not solve the fact that Papyrus has the wrong version in their repo ... but ... we'd be closer to solving, maybe. 

So, its up to your and the group. Just a suggestion. 


(In reply to comment #41)
> Just to be clear…..
> 
> You want me in the Riena feature to require a version higher than
> 1.2.0.v20100204-1500 for trident so I pick the right version ?
> 
> Because I am only contributing the right version into the repo with the
> about.html. Once the papyrus thing is respun and the invalid plugin is erased
> we would be good with any respin from my side ?
>
Comment 43 Christian Campo CLA 2011-06-15 10:28:31 EDT
(In reply to comment #42)
> I was suggesting that, but agree it's debatable. Would not be good if caused
> churn elsewhere (e.g. if Gyrex had to respin?). 
> 
> But, generally, we _always_ want _all_ our bundles to have increasing
> qualifiers/versions when they change. 
> 
> And, I'll confess ... this is open development ... I was thinking/hoping if
> your bundle increased version/qualifier like it should, to be greater than
> 1.2.0.v20100204-1500, then your version _might_ be used by the Papyrus
> requirement, hence solving the "common repo" problem. 
> 
> It would not solve the fact that Papyrus has the wrong version in their repo
> ... but ... we'd be closer to solving, maybe. 
> 
> So, its up to your and the group. Just a suggestion. 
> 
> 
> (In reply to comment #41)
> > Just to be clear…..
> > 
> > You want me in the Riena feature to require a version higher than
> > 1.2.0.v20100204-1500 for trident so I pick the right version ?
> > 
> > Because I am only contributing the right version into the repo with the
> > about.html. Once the papyrus thing is respun and the invalid plugin is erased
> > we would be good with any respin from my side ?
> >
I was a little confused but now I see the thing you are getting at…..Its the missing 'v' in front of the timestamp……of the trident .jar file

I would have to do that later tonight…..I keep you posted.
Comment 44 Gunnar Wagenknecht CLA 2011-06-15 10:29:05 EDT
(In reply to comment #42)
> I was suggesting that, but agree it's debatable. Would not be good if caused
> churn elsewhere (e.g. if Gyrex had to respin?). 

I submitted a new Gyrex build. However, we aren't in any EPP packages.
Comment 45 Paul Webster CLA 2011-06-15 10:32:02 EDT
I had a look at our e4 0.11.

download.eclipse.org/e4/updates/0.11 contains our RC4 which contains:
org.pushingpixels.trident_1.2.0.v20100204-1500.jar

John A fixed the about.html in our version with Bug 348690

download.eclipse.org/e4/updates/0.11-I-builds (where we're working towards our GA) then contains our bundle with our fix:
org.pushingpixels.trident_1.2.0.v20110609-1700.jar

In e4 it's built as part of the e4.ui feature and consumed by org.eclipse.e4.xwt.

PW
Comment 46 David Williams CLA 2011-06-15 11:09:52 EDT
Created attachment 198032 [details]
log from latest aggregation, showing Gyrex added


> I submitted a new Gyrex build. However, we aren't in any EPP packages.

Yes, our current /releases/staging contains the rebuild for Gyrex. I've attached console log if anyone wants to "diff". I do, with Eclipse IDE "compare with", and shows the main changes we expected. There are a few other small stray changes that might only be "order effects" (not true differences in the repository) ... and none look significant to me ... but kind of hard for me to "see" or know, for sure. [If anyone knows a better (easy) way to "compare repositories", that'd be great to teach us :) ]
Comment 47 Chokri Mraidha CLA 2011-06-15 11:12:03 EDT
Hi Paul,

I have used /e4/updates/0.11/S-0.11RC4-201106042201 in my last respin of Papyrus. Does this location contain the right version?

Chokri
Comment 48 Christian Campo CLA 2011-06-15 11:47:41 EDT
(In reply to comment #47)
> Hi Paul,
> 
> I have used /e4/updates/0.11/S-0.11RC4-201106042201 in my last respin of
> Papyrus. Does this location contain the right version?
> 
> Chokri

cant be right is my guess since the correct trident version starts with 20110609 in the bundle. id
Comment 49 Chokri Mraidha CLA 2011-06-15 11:56:40 EDT
I have taken the location you have indicated last Thursday: https://bugs.eclipse.org/bugs/show_bug.cgi?id=348742#c7 

I don't find any bundle begining with 20110609 in the download/e4 repository, can you give me the right location please?
Comment 50 Chokri Mraidha CLA 2011-06-15 12:07:07 EDT
More recent builds are in the 0.11-I-builds directory. But it seems that builds in that directory are deleted after 5 days. I cannot base Papyrus build on an integration build.
Comment 51 Paul Webster CLA 2011-06-15 12:13:25 EDT
(In reply to comment #49)
> I have taken the location you have indicated last Thursday:
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=348742#c7 
> 
> I don't find any bundle begining with 20110609 in the download/e4 repository,
> can you give me the right location please?

yes, this bundle was not under discussion in that bug.

The e4 fixed version of org.pushingpixels.trident_1.2.0.v20110609-1700.jar  is in http://download.eclipse.org/e4/updates/0.11-I-builds

(In reply to comment #50)
> More recent builds are in the 0.11-I-builds directory. But it seems that builds
> in that directory are deleted after 5 days. I cannot base Papyrus build on an
> integration build.

yes, they'll be deleted in a few days, as well as that entire I build repo will be deleted when we start e4 0.12

We're not at the point where we would normally promote another stable build, but we are producing GA candidates.  David, if you think this is important and it's holding up Indigo I can promote I20110614-1545 to a semi-fictional stable build in http://download.eclipse.org/e4/updates/0.11

PW
Comment 52 David Williams CLA 2011-06-15 12:28:44 EDT
> 
> We're not at the point where we would normally promote another stable build,
> but we are producing GA candidates.  David, if you think this is important and
> it's holding up Indigo I can promote I20110614-1545 to a semi-fictional stable
> build in http://download.eclipse.org/e4/updates/0.11
> 

Yes, it's important. I think either you should promote to semi-fictional stable build, so it will be slightly longer lived. 

But, the responsibility is primarily on the Papyrus team to work this out with the e4 team ... any time someone is building on (much less releasing) some dependency which is not yet released, it's really up to them to figure out how to "retain" versions they might need long term.
Comment 53 Chokri Mraidha CLA 2011-06-15 13:07:44 EDT
Ok, thanks.
A new build of Papyrus is launched with the following e4 update: e4/updates/0.11/S-0.11RC5-201106141545
Comment 54 Paul Webster CLA 2011-06-15 13:21:10 EDT
I've promoted our build to our stable update site:
download.eclipse.org/e4/updates/0.11

This now contains both versions:
org.pushingpixels.trident=1.2.0.v20100204-1500
org.pushingpixels.trident=1.2.0.v20110609-1700

Just a comment.  download.eclipse.org/e4/updates/0.11 is the stable composite repo, the child repo S-0.11RC5-201106141545 is little better than an I build site.

PW
Comment 55 David Williams CLA 2011-06-15 14:36:04 EDT
For what its worth ... just to show how there can be indirect effects of the "content of the repo" on EPP package builds ... I've discovered some bundles (listed below) are not in our EPP packages ... at least JEE and JavaScript, and that they are contributed by Gyrex (which had no contribution in the repo, at the time the packages were built). 

So, someone somewhere must have an optional dependency on them, so the current behavior of p2 (and repo publishing) is to include all optional bundles it finds in a repository (whether required by a feature, or not). 

Hence, I think best to respin the EPP packages once final repository is ready. 

> org.apache.lucene.highlighter_2.9.1.v20100421-0704.jar
> org.apache.lucene.memory_2.9.1.v20100421-0704.jar
> org.apache.lucene.misc_2.9.1.v20100421-0704.jar
> org.apache.lucene.queries_2.9.1.v20100421-0704.jar
> org.apache.lucene.snowball_2.9.1.v20100421-0704.jar
> org.apache.lucene.spellchecker_2.9.1.v20100421-0704.jar

I've no idea what function these bundles provide ... but since users will have them installed if installing from the repo, I think the EPP packages should have them too. Sound right?
Comment 56 David Williams CLA 2011-06-15 14:47:55 EDT
> A new build of Papyrus is launched ...

Ok, not that I mean to pressure you (well, maybe a little :) what's the outlook of re-contributing to the Indigo aggregation build? 

(That would include, you checking/testing your results, at least partially, to make sure they are as expected, has the updated, correct org.pushingpixels.trident bundle ... oh, and yes, still works :) 

Thanks,
Comment 57 Gunnar Wagenknecht CLA 2011-06-15 15:07:23 EDT
(In reply to comment #55)
> I've no idea what function these bundles provide ... but since users will have
> them installed if installing from the repo, I think the EPP packages should
> have them too. Sound right?

Very good catch David! The optional dependency very likely comes from the Orbit Lucene bundle itself (org.apache.lucene). This is because the bundle re-exports all Lucene packages without split-packages header. They all provide "add-on" functionality to Lucene which *in theory* should not be required by the Eclipse Help System (which is very likely the consumer of org.apache.lucene & org.apache.lucene.core).

-Gunnar
Comment 58 Chokri Mraidha CLA 2011-06-15 15:23:33 EDT
David,

Papyrus contribution is published.

Chokri
Comment 59 David Williams CLA 2011-06-15 16:09:23 EDT
> ... The optional dependency very likely comes from the Orbit
> Lucene bundle itself (org.apache.lucene). This is because the bundle re-exports
> all Lucene packages without split-packages header. They all provide "add-on"
> functionality to Lucene which *in theory* should not be required by the Eclipse
> Help System (which is very likely the consumer of org.apache.lucene &
> org.apache.lucene.core).
> 

I think you are right. I could find nothing, in JEE (RC4) package that required it, even optionally, except Lucene itself. So ... what to do? What's implications? If found earlier, I think I'd have asked eclipse help to consider a more specific pre-req? And/or ask Gyrex not to "pull" from general Orbit repo, but to trim to what is really desired and useful to users. 

But, for where we are ... should we just go with what we have, and not care about "mismatching" or respin EPP packages? 

Are you interest in respining Gyrex? And filter-out optional bundles, with an extra mirror step? (I'm not recommending it ... just making sure it is not important to you, or if it is).

But, if its pretty clear it would make no difference to end-user function, I could live with the discrepancy between EPP package bundles, and "install from repo" bundles. (We could,, to be explicit, always change EPP build procedure so there we would filter out optional-only bundles .. which we currently do not). Only odd thing is that during SR1 update, people may "automatically" get these extra bundles ... but, if they really do not matter ... then seems minor.
Comment 60 Markus Knauer CLA 2011-06-15 16:15:58 EDT
(In reply to comment #59)

Cross-referencing to my very similar comment on bug 349464 comment #c7.

Before deciding whether to rebuild all EPP packages or not, I'd like to hear more opinions.
Comment 61 Gunnar Wagenknecht CLA 2011-06-15 16:42:50 EDT
(In reply to comment #59)
> Are you interest in respining Gyrex? And filter-out optional bundles, with an
> extra mirror step? (I'm not recommending it ... just making sure it is not
> important to you, or if it is).

The thing is, we actually like shipping *all* the Lucene bundles from Orbit as we also include Solr. If EPP can't live with the bundles being included than the filtering would have to happen during the EPP package build.
Comment 62 David Williams CLA 2011-06-15 18:10:39 EDT
> The thing is, we actually like shipping *all* the Lucene bundles from Orbit as
> we also include Solr. If EPP can't live with the bundles being included than
> the filtering would have to happen during the EPP package build.

Thanks Gunnar, I see that after reading Markus's excellent analysis. 

For those interested, this problem of EPP packages pulling in less than desired optional bundles has been reported before (such as bug 348357) and there is ongoing (long) discussion in bug 247099 of how to improve p2 or metadata publishing in Juno to provide the right level of control, for providers, builders, and users.
Comment 63 David Williams CLA 2011-06-15 18:21:32 EDT
I think we are done. There is a new /releases/staging repo that has the right bundles (that has the right about.html files). 

So, to summarize, besides the Eclipse Platform, (to get the fix we started off repining for) the following projects also had some changes; some to include their own fixes while they had the chance, and some as a pure reaction to other changes: 

ptp
objectteam
jetty
riena
gyrex
papyrus (and e4, sort of)

Let us know if I missed anyone. 
Thanks to all.
Comment 64 David Williams CLA 2011-06-15 18:22:50 EDT
> Let us know if I missed anyone. 

Whoops, missed RAP runtime :)
Comment 65 Christian Campo CLA 2011-06-15 18:32:59 EDT
(In reply to comment #64)
> > Let us know if I missed anyone. 
> 
> Whoops, missed RAP runtime :)

Its ok for me. The only thing is that we have two files for the pushingpixels.jar. They are both ok, have the right about.html. Its just that I wasnt fast enough to fix the Riena feature to include the newest.

We have org.pushingpixels.trident,1.2.0.201106090835 (Riena) and we have
org.pushingpixels.trident,1.2.0.v20110609-1700 (Papyrus)

Its ok to leave it at that since the content is identical. I have just finished a p2 repo for Riena that also uses org.pushingpixels.trident,1.2.0.v20110609-1700.

But its just a little thing and we leave it at that. The p2 repo is ready and uploaded but we it would mean another 2 hours…… :-)
Comment 66 David Williams CLA 2011-06-15 19:42:47 EDT
> But its just a little thing and we leave it at that. 

Yes, to be explicit. We'll leave it at that (I didn't forget you ... just got impatient :). 

But, thanks for updating. Now about that Orbit CQ ... ? I've opened bug 349512 in Orbit to track, but one of the 3 currently consuming projects will need to step forward to "own" it. Any volunteers?