Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 329191 - [releng] Hard to install into a milestone build
Summary: [releng] Hard to install into a milestone build
Status: RESOLVED FIXED
Alias: None
Product: e4
Classification: Eclipse Project
Component: UI (show other bugs)
Version: unspecified   Edit
Hardware: PC Linux
: P3 critical (vote)
Target Milestone: 4.1 M4   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-11-01 09:55 EDT by Paul Webster CLA
Modified: 2010-12-15 15:59 EST (History)
10 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Paul Webster CLA 2010-11-01 09:55:30 EDT
M2 wasn't really installable or updatable, almost 1 week after we created it.  In the last 2 weeks, I've seen a lot of people trying out M2 (probably getting ready for M3) and it was not pretty :-)

Our current builds come with 3 sites defined:
http://download.eclipse.org/releases/indigo
http://download.eclipse.org/eclipse/updates/4.1-I-builds
http://download.eclipse.org/e4/updates/0.11-I-builds

For M3, we need to do better, but there are a couple of things it looks like we need to address:


1) The Indigo M3 version of the EMF SDK.  We're going to respin our M3 candidate this week with the indigo M3 candidate.

2) Installing the e4 tools (and e4 UI source).  0.11-I-builds tracks the I builds, and after about 2 weeks you can no longer add the e4 tools.

3) 4.1-I-builds also becomes useless after about 2 weeks for the milestone build.


For 2 and 3, do we need to create a 0.11-milestones and 4.1-milestones respectively?  Create the composite repos, and then re-seed them with the M2 repos (still available on the individual e4/4.1 M2 pages).  Then add the 2 milestone update sites?  Do we add them just to the milestone respin, or generally to all builds?

PW
Comment 1 Paul Webster CLA 2010-11-01 13:46:31 EDT
uh, OK, now that I've looked at the Planning Council schedule, looks +1 M3 isn't until Nov 8th.  Since we can't wait, can we come up with a solution that will allow e4/4.1 M3 to work with the EMF SDK that comes out of http://download.eclipse.org/releases/indigo whether it's M2 or M3?

Andrew, what do we have to do here?

In e4 it looks like we specify EMF as Compatible on the Dependeny tab.  That seems to imply the ui feature will install on any version of EMF.

In 4.1 org.eclipse.e4.rcp specifies org.eclipse.emf.common and org.eclipse.emf.ecore with no version numbers in the Included Features tab.

Ideally we want to be able to build and distribute our 4.1 M3 with the M2 version of those 2 features.  But when Indigo M3 comes out, installing the M3 EMF SDK should update those 2 features within the 4.1 build.

PW
Comment 2 John Arthorne CLA 2010-11-01 16:03:16 EDT
Here is a quick summary of the situation: Platform 4.1 depends on EMF. 4.1's M3 date is a week earlier than the EMF M3 date. This gap reduces to a single day in M4 and later. For M3, our options are:

1) Declare Platform M3 immediately using EMF from M2
2) Wait a week, and declare Platform M3 using EMF M3
3) Change our build structure so that we require EMF rather than including it. That would allow us to declare Platform M3 on EMF M2, but then the user could install EMF M3 into Platform M3 later. That also means someone installing Platform M3 today would get something different that installing it next week (get a different version of EMF).

My vote is option 1). Although it prevents anyone from installing tools based on EMF M3 into Platform M3, it allows us to ship close to our schedule. More importantly it means we ship the actual configuration that we tested last week. 

For M4 and beyond, I think we need to put 4.1 builds onto the release train +1 schedule along with EMF. Currently we are attempting to be a +0 project, but with +1 dependencies.
Comment 3 Eric Moffatt CLA 2010-11-01 16:13:21 EDT
John, the concern is that being +1 essentially eliminates 4.1 as a viable platform for folks that are doing Indigo development. They'd have to wait a week after the milestone before the build would contain their M3 contributions...

Note that we use only the most basic elements of EMF so there shouldn't be any (technical) issue with using the previous milestone's EMF should there ? What's broken from a consumer's perspective if we where to just use an earlier EFM build? Done this way only the EMF folks using 4.1 would be affected rather than the whole community.
Comment 4 John Arthorne CLA 2010-11-01 16:26:05 EDT
(In reply to comment #3)
> John, the concern is that being +1 essentially eliminates 4.1 as a viable
> platform for folks that are doing Indigo development. They'd have to wait a
> week after the milestone before the build would contain their M3
> contributions...

I was suggesting this for M4 and beyond, where the gap between +0 and +1 is one *day* instead of the current one *week* gap.


> Note that we use only the most basic elements of EMF so there shouldn't be any
> (technical) issue with using the previous milestone's EMF should there ? What's
> broken from a consumer's perspective if we where to just use an earlier EFM
> build? Done this way only the EMF folks using 4.1 would be affected rather than
> the whole community.

The end user problem is that if they install 4.1 M3, and open up the indigo release train M3 repository, they will not be able to install certain things because of the mismatch in EMF dependencies. A core purpose of the simultaneous release is that the stuff shipped together can actually install and run together.
Comment 5 Mike Wilson CLA 2010-11-01 16:33:28 EDT
If Eclipse 4.x has a dependency on a +1 project, then to me the best we can do is also deliver Eclipse 4.x as +1. I think we would likely continue to lock down our 4.x content as though we were +0, and then take the appropriate version of EMF when it becomes available, sniff test to make sure we aren't busted at the last second, and deliver our 4.x build.

For this to work, it assumes that the EMF team is not going to destabilize us in the last week, which is similar to what happens with the pieces of ECF that we consume (although in that case, they provide them early so that we can declare at +0). In any case, we will need to make sure that we have appropriate levels of communication/early testing.

Note that we currently only upgrade to a new EMF once per milestone, so the "delayed" version of 4.x only needs to happen for the first week after each milestone.
Comment 6 Dani Megert CLA 2010-11-02 04:59:33 EDT
I'm not sure we have a strict dependency on EMF and that the +1 is needed for 4.1 M4 and beyond, given only basic EMF functionality is used i.e. 4.1 might even work with EMF from Helios.

Can't we ship with what works and got tested but specify the requirements in a way that we also work with newer versions? That way I would expect that p2 can later install a newer version of EMF if it is required by a component that gets added later. Also, when installing that component I would expect p2 to fetch the latest EMF version that's required by that component.

While we discuss this further I suggest that we now declare 4.1 M3 immediately (option 1 from comment 2).
Comment 7 Jeff McAffer CLA 2010-11-02 10:59:31 EDT
My 2c.  ship on schedule and sort out the issue for the next milestone.  

Going forward we can't ship before our prereqs.  Note 1 week, 1 day, 1 second. Beyond testing etc we need to know their version number.

While we technically can setup a requires dependency to get later things as they come available, this is very problematic for product install / build reproducibility scenarios.  It is also against proposed (adopted?) Indigo must do guidelines.

One possibility is to ship a +0 and +1 version of 4.1. Given that the diff is one day (soon) not sure that is worth the effort/confusion but it might be something to consider now (immediate problem) and for future trains when the gap is larger.
Comment 8 Andrew Niefer CLA 2010-11-02 17:36:54 EDT
M3 is now up.
Comment 9 Martin Oberhuber CLA 2010-11-03 00:01:20 EDT
I agree with Dani. It doesn't feel right to lock the adopters of Platform 4.1 into a particular version of EMF. Just like adopters of the Helios Modeling Package should be able to update to a newer EMF if they want.

I can't see how loosening the version constraints for adopters would affect build reproducability or cause product install problems. The build can always obtain a specific well-defined version. And for product install, users can always refer to a specific well-defined repository that holds the bits they expect.

In terms of release schedule / testing, sure it makes sense to ship what we test. I don't know the exact dependencies, is there a circle (Platform->EMF->Platform) or could EMF freeze the parts that we consume early?
Comment 10 Jeff McAffer CLA 2010-11-03 10:09:18 EDT
This is the same discussion we have been having in the planning council and elsewhere.  If you say your product P version 1.2.3 *requires* EMF [2.5.0, 3.0.0) then at install time, p2 will find and use the highest version available at that time with the given set of repos available to that user.  So you and I could both install P 1.2.3 but get very different results. In the lab we can control the set of repos (and thus versions) available but in the wild, not so much. Even in our own repos it will be a challenge.  For example, once we do Indigo SR1 users will not be able to install SR0 because the later version will be used.
Comment 11 Paul Webster CLA 2010-11-12 13:18:56 EST
For our M3 we need the M2 EMF:

org.eclipse.emf.sdk_2.7.0.v20100928-1843

When indigo offers org.eclipse.emf.sdk_2.7.0.v20101108-0959 we have to remember to uncheck "show latest version only"

PW
Comment 12 Andrew Niefer CLA 2010-11-26 14:29:34 EST
Have we decided to change the EMF dependency to a version range?  If we are doing this for M4 the builder changes will need to be in place this coming week.
Comment 13 John Arthorne CLA 2010-11-26 15:10:06 EST
(In reply to comment #12)
> Have we decided to change the EMF dependency to a version range?  If we are
> doing this for M4 the builder changes will need to be in place this coming
> week.

No. EMF is considering declaring early on +0 date, so we can consume it. See bug 329935. Until that point, 4.1 will be operating as a +1 project (one day after 3.7 milestones).
Comment 14 Andrew Niefer CLA 2010-12-09 16:37:30 EST
For M4, EMF is contributing an "emf.base" at +0.  We will be recompiling the 4.1 SDK to target this.

The e4 incubator currently builds first and requires more EMF which will not be available at +0.  These pieces are not required by the SDK.

At this point we will end up compiling the incubator against an EMF almost-m4-I-Build and the 4.1 SDK will target M4.  The incubator pieces will still be able to be installed into 4.1M4 because they require the EMF features using a compatible matching rule.

Starting early in M5, we will switch to building the 4.1 SDK before the incubator (bug 331237).  At this point we will be able to properly build 4.1 at +0 against EMF +0 pieces, and the e4 incubator is really +1 and can be built separately.
Comment 15 John Arthorne CLA 2010-12-15 15:59:26 EST
4.1 builds now consume EMF from the same milestone.