Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 414591

Summary: two versions of 'httpcore' included in repo
Product: [Eclipse Project] Platform Reporter: David Williams <david_williams>
Component: RelengAssignee: Platform-Releng-Inbox <platform-releng-inbox>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P2 CC: bugs.eclipse.org, irbull, pascal.rapicault, sebastian.schmidt2, slewis, steffen.pingel, tjwatson
Version: 4.3   
Target Milestone: 4.4 M2   
Hardware: PC   
OS: Linux   
Whiteboard: might need cq filed by p2?

Description David Williams CLA 2013-08-07 11:30:07 EDT
According to some of the first "Luna repo reports" we have two versions of "httpcore" in our repo. Not sure this hurts much, but should be avoided. 

One might "come from" ECF and the other from "us"? But should check to make sure there are no pom's that "constrain" which version to use. 

org.apache.httpcomponents.httpcore
	4.1.4.v201203221030
	4.2.4.v201305222326
org.apache.httpcomponents.httpclient
	4.1.3.v201209201135
	4.2.5.v201305222326
Comment 1 David Williams CLA 2013-08-07 21:50:06 EDT
Tracked down the fundamental cause of this. 

In Luna, we are getting the 4.1.x version from ECF ... and that's what we use in Kepler, as that was the most recent in the "released" version of Orbit, 

http://download.eclipse.org/tools/orbit/downloads/drops/R20130517111416/repository/

But, for Luna, to get Ant 1.9.2, we moved up to an Orbit I build, 
http://download.eclipse.org/tools/orbit/downloads/drops/I20130724151342/repository/

and it happens to have the more recent 4.2.x versions of "apache http" bundles. 

At a minimum, we will at least have to file a new CQ saying we are using the 4.2.x versions. (Assuming we want most recent versions, which can't imagine why not). 

But, more so, we'll have to be sure to coordinate with ECF better, say for M2, so we both use same version. 

Scott, 
  Are you doing "Luna builds" yet? Plan to for Luna M2? I ask since even if you haven't changed anything else (in "base" components, we use) it would be good to coordinate our use of these "most recent" http bundles. I'm optimistically assuming for the moment that at runtime the most recent one gets used so the 4.1.x versions are merely wasted space, but in theory, such "slightly different versions" could be loaded (wired) by different segments of code and eventually get their wires crossed. 

Pascal, Ian, 
  Added you to CC just FYI, since p2 is the primary consumer of these bundles in Eclipse SDK stack. 

I'm not too concerned about "M1" ... unless any of you are :/ ... but just wanted to be sure we get synchronized in Luna stream before too long. 

An alternative is for us ("sdk") to put "restrictions" in pom ?target environment? to get only 4.1 version ... but that seems less than ideal for several reasons.
Comment 2 Scott Lewis CLA 2013-08-07 23:21:30 EDT
(In reply to comment #1)
<stuff deleted>
> At a minimum, we will at least have to file a new CQ saying we are using the
> 4.2.x versions. (Assuming we want most recent versions, which can't imagine
> why not). 

In general we/ECF are fine with moving up to more recent versions, but we need some time in advance to test for possible regression, etc.

> 
> But, more so, we'll have to be sure to coordinate with ECF better, say for
> M2, so we both use same version. 

Sure.

> 
> Scott, 
>   Are you doing "Luna builds" yet? Plan to for Luna M2? 

No, not yet.   When is M2?  :)

Like I said, we are usually going to be game for using more recent versions of stuff in Orbit, but we would like lead time...to schedule sufficient my/our time to test, make our build changes, etc.
Comment 3 David Williams CLA 2013-08-08 02:18:46 EDT

> > Scott, 
> >   Are you doing "Luna builds" yet? Plan to for Luna M2? 
> 
> No, not yet.   When is M2?  :)
> 

Our +0 day for M2 is 9/20. (Most likely, still being formalized). 

So ... if we could get an update at least a week before that, that'd be great.
Comment 4 Scott Lewis CLA 2013-08-08 10:47:40 EDT
(In reply to comment #3)
> 
> > > Scott, 
> > >   Are you doing "Luna builds" yet? Plan to for Luna M2? 
> > 
> > No, not yet.   When is M2?  :)
> > 
> 
> Our +0 day for M2 is 9/20. (Most likely, still being formalized). 
> 
> So ... if we could get an update at least a week before that, that'd be
> great.

Ok, I can't commit to that yet, but I'll investigate.

To verify...exactly which versions is the non-ECF usage of httpcomponents going to use?
Comment 5 David Williams CLA 2013-08-08 20:56:03 EDT
> 
> Ok, I can't commit to that yet, but I'll investigate.
> 
> To verify...exactly which versions is the non-ECF usage of httpcomponents
> going to use?

Not exactly sure what you mean ... we just "get latest" from Orbit ... which we can "coordinate" better for M2 ... (Orbit didn't hasn't officially declared an "S build" for Luna M1 ... yet, but imagine we'll be more organized and timely for M2.
Comment 6 Scott Lewis CLA 2013-08-09 11:03:38 EDT
(In reply to comment #5)
> > 
> > Ok, I can't commit to that yet, but I'll investigate.
> > 
> > To verify...exactly which versions is the non-ECF usage of httpcomponents
> > going to use?
> 
> Not exactly sure what you mean ... we just "get latest" from Orbit ... which
> we can "coordinate" better for M2 ... (Orbit didn't hasn't officially
> declared an "S build" for Luna M1 ... yet, but imagine we'll be more
> organized and timely for M2.

Ok, I've opened two reuse CQs for these versions

org.apache.httpcomponents.httpclient 4.2.5
https://dev.eclipse.org/ipzilla/show_bug.cgi?id=7525

org.apache.httpcomponents.httpcore 4.2.4
https://dev.eclipse.org/ipzilla/show_bug.cgi?id=7526

from http://download.eclipse.org/tools/orbit/downloads/drops/S20130724151342/
Comment 7 Scott Lewis CLA 2013-08-09 16:39:32 EDT
(In reply to comment #3)
> 
> > > Scott, 
> > >   Are you doing "Luna builds" yet? Plan to for Luna M2? 
> > 
> > No, not yet.   When is M2?  :)
> > 
> 
> Our +0 day for M2 is 9/20. (Most likely, still being formalized). 
> 
> So ... if we could get an update at least a week before that, that'd be
> great.


Today I was able to build and test ECF filetransfer against the newer versions of httpcomponents...and all ECF filetransfer tests passed fine (no regression).  Good news.

Given this...and some regression testing assistance from the p2 folks once we've updated our own build (probably next week for that)...I'll commit to Luna M2 delivery.
Comment 8 Scott Lewis CLA 2013-08-28 17:58:08 EDT
<stuff deleted>
> Today I was able to build and test ECF filetransfer against the newer
> versions of httpcomponents...and all ECF filetransfer tests passed fine (no
> regression).  Good news.
> 
> Given this...and some regression testing assistance from the p2 folks once
> we've updated our own build (probably next week for that)...I'll commit to
> Luna M2 delivery.

We've successfully integrated apache httpcomponents 4.2.4/4.2.5 into our httpclient4 provider, and have produced a source repo to use for regression testing (by p2 I assume):

https://build.ecf-project.org/jenkins/job/C-HEAD-filetransfer.httpclient4.feature/lastSuccessfulBuild/artifact/

I will post a notice on the p2-dev mailing list, and they can let me know how they want to get it, do the regression testing, etc.

I'll let you/David resolve this bug, but from my chair we're done with this one.  New version of filetransfer has to get into p2/platform...but I assume that's another bug.
Comment 9 David Williams CLA 2013-08-28 18:42:44 EDT
(In reply to comment #8)

> 
> I'll let you/David resolve this bug, but from my chair we're done with this
> one.  New version of filetransfer has to get into p2/platform...but I assume
> that's another bug.

No, I think we can use this bug, and I don't think you need to wait for assessment from p2 team. You might be waiting a long time :) 

I suggest you go ahead and prepare/copy a repo we can use in our build, such as the current one: 

<ecf-repo.url>http://download.eclipse.org/rt/ecf/int7/site.p2/</ecf-repo.url>

And, we'll give it a go .. there is plenty of of time to fix things if we see errors. 

And, I'm saying this partially because I assume you didn't change much, except to get the 4.2.x versions of httpcomponents ... that, and, if you look at EPP packages from M1, they actually picked up the 4.2.x version ... such as "Eclipse Standard", at 
http://www.eclipse.org/downloads/download.php?file=/technology/epp/downloads/release/luna/M1/eclipse-standard-luna-M1-linux-gtk-x86_64.tar.gz&mirror_id=346

So I think if very much was wrong with it, we would have heard about it by now.
Comment 10 Steffen Pingel CLA 2013-08-29 03:45:42 EDT
I scheduled bug 381543 for the Mylyn Luna release to ensure that we also consume the latest HttpCore/Client version.
Comment 11 Scott Lewis CLA 2013-08-30 13:12:59 EDT
(In reply to comment #9)
<stuff deleted>

> No, I think we can use this bug, and I don't think you need to wait for
> assessment from p2 team. You might be waiting a long time :) 
> 
> I suggest you go ahead and prepare/copy a repo we can use in our build, such
> as the current one: 
> 
> <ecf-repo.url>http://download.eclipse.org/rt/ecf/int7/site.p2/</ecf-repo.url>
> 
> And, we'll give it a go .. there is plenty of of time to fix things if we
> see errors. 

Ok, here's a new repo with these changes:

http://download.eclipse.org/rt/ecf/3.7.int1/site.p2


> 
> And, I'm saying this partially because I assume you didn't change much,

Yeah, that's right.  I believe that for the filetransfer plugins that you are using, that was/is the only change.

<stuff deleted>

> 
> So I think if very much was wrong with it, we would have heard about it by
> now.

Hopefully true.

Please let me know if you have any difficulties with this new repo.  I'll monitor the p2-dev mailing list as usual as it gets integrated/tested.
Comment 12 David Williams CLA 2013-08-30 22:41:08 EDT
Thanks Scott, 

I've updated our parent pom to fetch from your latest repo so we will "match". 

http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?id=5651f6032c5208f0eee44427b75c1920cf07d2d4

I will assume for now this is the only/last change you will make for Luna M2 ... if that is an invalid assumption, feel free to reopen or open new bug if we should update ECF again for M2. 

Ian, Pascal ... don't forget p2 needs a "reuse from Orbit" CQ open for this version of httpcore and httpclient (CQs 7240 and 7268). [To summarize, these are "new" in the Luna stream of Orbit ... and a) we'd just as well be current and b) I'm sure they made some fixes/improvements that some client might need). 

And, actually, now that I look at their home page, 
http://hc.apache.org/httpcomponents-core-ga/
it appears they have released a version 4.3 just within the past few days so someone (Mylyn? if not ourselves?) may want to upgrade again before Luna releases.
Comment 13 David Williams CLA 2013-09-02 12:06:56 EDT
For what's it worth, I asked on Mylyn list if they planned to move to "4.3" versions once available and the reply was "probably" ... so we may have to go through this again in a future milestone. 

http://dev.eclipse.org/mhonarc/lists/mylyn-commons-dev/msg00027.html
Comment 14 Scott Lewis CLA 2013-09-02 22:57:01 EDT
(In reply to David Williams from comment #13)
> For what's it worth, I asked on Mylyn list if they planned to move to "4.3"
> versions once available and the reply was "probably" ... so we may have to
> go through this again in a future milestone. 
> 
> http://dev.eclipse.org/mhonarc/lists/mylyn-commons-dev/msg00027.html

Did you ask them if they are prepared to help out with regression testing for p2/Equinox?

I believe Steffen is on this bug...Steffen...could you communicate with someone at Mylyn to see:  if Mylyn plans on moving to 4.3...could someone allocate some resources to help test p2 regression?...i.e. if you want there to be only one version...i.e. the latest one...in the Luna repo.
Comment 15 Scott Lewis CLA 2013-09-02 23:01:00 EDT
(In reply to David Williams from comment #12)
> Thanks Scott, 
> 
> I've updated our parent pom to fetch from your latest repo so we will
> "match". 
> 
> http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/
> commit/?id=5651f6032c5208f0eee44427b75c1920cf07d2d4
> 
> I will assume for now this is the only/last change you will make for Luna M2
> ... if that is an invalid assumption, feel free to reopen or open new bug if
> we should update ECF again for M2. 

So for filetransfer...this is our last expected change for Luna M2 (unless there is some serious regression discovered).

But we/ECF might have another build for Luna M2...for the rest of ECF.

> 
> Ian, Pascal ... don't forget p2 needs a "reuse from Orbit" CQ open for this
> version of httpcore and httpclient (CQs 7240 and 7268). [To summarize, these
> are "new" in the Luna stream of Orbit ... and a) we'd just as well be
> current and b) I'm sure they made some fixes/improvements that some client
> might need). 

Does p2 need a separate-from-our/ECF CQs?  We have submitted CQs for ECF filetransfer usage see comment 6.  Also...BTW, httpcomponents has a dependency on Apache codec 1.6...so we had to have a reuse CQ for that too.
Comment 16 David Williams CLA 2013-09-03 01:19:30 EDT
(In reply to Scott Lewis from comment #15)
> (In reply to David Williams from comment #12)

> > Ian, Pascal ... don't forget p2 needs a "reuse from Orbit" CQ open for this
> > version of httpcore and httpclient (CQs 7240 and 7268). [To summarize, these
> > are "new" in the Luna stream of Orbit ... and a) we'd just as well be
> > current and b) I'm sure they made some fixes/improvements that some client
> > might need). 
> 
> Does p2 need a separate-from-our/ECF CQs?  We have submitted CQs for ECF
> filetransfer usage see comment 6.  Also...BTW, httpcomponents has a
> dependency on Apache codec 1.6...so we had to have a reuse CQ for that too.

Ah, good point. Thanks! Unless p2 calls the APIs directly (from httpcomponents or codec) then they do not need a CQ, which I imagine is the case -- but if "call/manipulates the APIs directly, then they do. I believe that's the rule.