Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 223920 - [ui] Installing EMF SDK feature from zip does not add artifact repo, install fails
Summary: [ui] Installing EMF SDK feature from zip does not add artifact repo, install ...
Status: RESOLVED FIXED
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: p2 (show other bugs)
Version: 3.4   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: 3.4 M6   Edit
Assignee: Susan McCourt CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 223931
  Show dependency tree
 
Reported: 2008-03-25 13:48 EDT by Nick Boldt CLA
Modified: 2008-03-26 00:20 EDT (History)
5 users (show)

See Also:


Attachments
eclipse install problem log (10.33 KB, text/plain)
2008-03-25 13:48 EDT, Nick Boldt CLA
no flags Details
patch to ExternalFileHandler (1.60 KB, patch)
2008-03-25 17:26 EDT, Susan McCourt CLA
no flags Details | Diff
patched bundle (89.29 KB, application/octet-stream)
2008-03-25 17:41 EDT, Simon Kaegi CLA
no flags Details
console log (10.95 KB, text/plain)
2008-03-25 18:16 EDT, Nick Boldt CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Boldt CLA 2008-03-25 13:48:37 EDT
Created attachment 93453 [details]
eclipse install problem log

Steps to reproduce:

0. start up Eclipse 3.4 I20080324-1300 (linux gtk), with Sun 5.0
1. download EMF 2.4 All-in-one SDK zip (latest I or N build from http://emf.torolab.ibm.com/modeling/emf/downloads/?project=emf&sortBy=date&hlbuild=0#latest)
2. Help > Software Updates... > 
   Available Features > Manage sites... > 
   Add... > Archive... >
   Browse for emf-sdo-xsd-SDK-*.zip
   Expand? [Yes]. 
   Click [OK]
4. Expand "Other" twisty.
5. Pick "Eclipse Modeling Framework (EMF) Extender SDK". Click Install...
6. Only one feature is shown.
7. Next > Agree > Finish
8. See attached error log.
Comment 1 Simon Kaegi CLA 2008-03-25 14:25:11 EDT
I'll take a look at this
Comment 2 Nick Boldt CLA 2008-03-25 14:42:16 EDT
I should mention that the latest N builds on the emf.torolab site are built with the same Eclipse version - I20080324-1300; last week's I build was built w/ the 03/05 platform driver. I'm not sure yet if the version used to build is relevant to the issues seen or not.
Comment 3 Simon Kaegi CLA 2008-03-25 16:09:18 EDT
Perhaps getting the builds in synch has something to do with it.

I tried this out with a clean SDK I20080324-1300 and emf-sdo-xsd-SDK-N200803251300.zip on Windows XP. I literally unzipped the SDK and then followed the script (thanks).

I however didn't run into any problems and didn't see any errors written to the log.

After finishing the install and restarting I also saw only the EMF Extender SDK in the Installed Features dialog in the p2 update ui. If I look at Help->About->Feature Details I see about 13 EMF Features have been installed.

This is perhaps a bit misleading as p2 "really" is installing the features but is only showing what it sees as the "root" feature that triggered the installation of all the other pieces.

I'm going to try this scenario again as I saw this all work correctly with a debugger attached which might have altered timing.
Comment 4 Simon Kaegi CLA 2008-03-25 16:18:48 EDT
I'm hitting this now. It looks like we're only adding the metadatarepository and not the associated artifact repository for the contents of an archive.

If this is blocking you and you want a workaround... remove the site and then re-add it using the local folder where the contents were unzipped.
Comment 5 Susan McCourt CLA 2008-03-25 16:54:13 EDT
This is a three-liner to fix.  Working on it right now (I'm on a slow connection so it will just take awhile to download the specific builds mentioned).
Comment 6 Susan McCourt CLA 2008-03-25 16:55:20 EDT
Another workaround is to drag the zip into the available features tab.  That code path (and the workaround Simon mentioned) both assume colocated artifact/metadata repos.
Comment 7 Susan McCourt CLA 2008-03-25 17:26:08 EDT
Created attachment 93500 [details]
patch to ExternalFileHandler

Simon, can you test this patch (I'm still downloading...)
Comment 8 Simon Kaegi CLA 2008-03-25 17:41:11 EDT
Created attachment 93502 [details]
patched bundle

Thanks Susan. With your patch things work correctly for Nick's script. I've attached the bundle jar with the correct version for SDK I20080324-1300 if others want to just overwrite the bundle in their plugins folder.
Comment 9 Nick Boldt CLA 2008-03-25 18:16:35 EDT
Created attachment 93507 [details]
console log

(In reply to comment #8)
> Thanks Susan. With your patch things work correctly for Nick's script. I've
> attached the bundle jar with the correct version for SDK I20080324-1300 if
> others want to just overwrite the bundle in their plugins folder.

Patch doesn't work here, using new jar to replace existing one:

cp -f org.eclipse.equinox.p2.ui.sdk_0.1.0.v20080317-1820.jar eclipse/plugins/; ./34clean.sh again 2>&1 | tee eclipse_I20080324-1300.bug223920.patch.txt

(my 34clean.sh script can be seen in bug 223931 comment #0)

After installing the SDK failed, I just tried installing a single feature, the first on the list. That failed too, as per the log.
Comment 10 Susan McCourt CLA 2008-03-25 18:31:02 EDT
I just verified the scenario with Simon's bundle.
Released to HEAD.  Tagged for next rebuild.

Renamed bug to mention the implementation problem.

One thing I do not like is that the busy indicator doesn't seem to work during the unzip, so it's pretty confusing what is going on (the unzip takes a little while).  Opened bug #223996 for this part of the problem.
Comment 11 Susan McCourt CLA 2008-03-25 18:33:30 EDT
Nick, I committed the fix before seeing your note.  I will check your scenario again.  The fix definitely fixes the problem for me.  

For the fix to work, you need to remove the previously added site before trying the scenario again?  If you did not, the problem would not be fixed.
Comment 12 Nick Boldt CLA 2008-03-25 19:48:17 EDT
(In reply to comment #11)
> For the fix to work, you need to remove the previously added site before trying
> the scenario again?  If you did not, the problem would not be fixed.

That must be what I did, because with a clean start it works. I get 29 plugins and 19 features when I pick just the main EMF SDK feature. I can also install some random xsd feature  (eg., xsd.ecore.converter) and p2 appears to correctly installs its plugin dependencies.

However, if I pick, say, the SDO SDK feature, I'd expect that I would end up with at least 6 features installed, including:

   <includes id="org.eclipse.emf.ecore.sdo" version="0.0.0"/>
   <includes id="org.eclipse.emf.ecore.sdo.edit" version="0.0.0"/>
   <includes id="org.eclipse.emf.ecore.sdo.editor" version="0.0.0"/>
   <includes id="org.eclipse.emf.ecore.sdo.source" version="0.0.0"/>
   <includes id="org.eclipse.emf.ecore.sdo.doc" version="0.0.0"/>

and org.eclipse.emf.ecore.sdo.sdk. (The XML above is from /cvsroot/modeling/org.eclipse.emf/org.eclipse.emf.ecore.sdo/features/org.eclipse.emf.ecore.sdo.sdk-feature/feature.xml)

Instead, I get:

org.eclipse.emf
org.eclipse.emf.mapping.ecore
org.eclipse.emf.mapping.ecore.editor
org.eclipse.emf.ecore.sdo
org.eclipse.emf.ecore.sdo.doc
org.eclipse.emf.ecore.sdo.source

... the emf.ecore.sdo.edit, emf.ecore.sdo.editor, and emf.ecore.sdo.sdk features are not installed!

Should contained features be installed, as they used to be with Update Manager? Or does p2 intentionally not care about this extraneous metadata?

--

I have a few usability questions that I've opened as separate bugs:

bug 224003: [Usability] report progress in Update dialog as well as Progress widget / view
bug 224004: [Usability] Closing Software Updates dialog causes "Restart Eclipse?" prompt to self-dispose
bug 224006: [Usability] Filter already-installed features from Available Features
Comment 13 Susan McCourt CLA 2008-03-25 20:00:24 EDT
Thanks for rechecking, Nick.

>Should contained features be installed, as they used to be with Update Manager?
>Or does p2 intentionally not care about this extraneous metadata?

Pascal, can you comment?  With all the late breaking swirl on optional features and other changes, I'm not really sure of the answer.  
Comment 14 John Arthorne CLA 2008-03-25 20:32:12 EDT
Nick, what exactly do you mean by "not installed" for those other features? You mean the feature.xml, etc, do not appear in the features directory on disk, or they don't show up in the installed features list in the install dialog?  You may be misled by the fact that only the "root" item you installed appears in the installed features list in the install dialog. I.e., if you select "X" and install it, you will only see "X" in the installed features list, even if the installation of "X" caused a wad of other things to be installed. This is by design (although I can understand confusion).
Comment 15 Nick Boldt CLA 2008-03-25 21:05:32 EDT
(In reply to comment #14)
> Nick, what exactly do you mean by "not installed" for those other features? > they don't show up in the installed features list in the install dialog?  

Right, or in the Help > About Features dialog, either.

You
> may be misled by the fact that only the "root" item you installed appears in
> the installed features list in the install dialog. I.e., if you select "X" and
> install it, you will only see "X" in the installed features list, even if the
> installation of "X" caused a wad of other things to be installed. This is by
> design (although I can understand confusion).

I understand the geek side of this design, and agree, this makes sense. 

But consider the user side. If I, Joe User, install the Foo SDK feature, which I know includes the Foo runtime, Foo doc, Foo source, and Foo examples features, I expect to see those features installed too, or I'd think my install is busted. Just because the plugins are there and work... shouldn't the features show up too? Or are we intentionally trying to convince people that because features suck (heh), they should be considered second-class IUs, and in terms of functionality, we shouldn't care if all the features get installed?

(My fear here is that new users, when developing, may think they don't need features anymore, when in reality they are still useful for providing the optional granularity and choices when installing via the old UM or the new p2 IM.)
 
And while I'm ranting about the new UI and the user experience w.r.t. auditing what I think is installed, what happened to the Manage Configuration view? I love(d) that thing for quickly verifying installed/broken features! 
Comment 16 John Arthorne CLA 2008-03-25 21:51:52 EDT
> But consider the user side. If I, Joe User, install the Foo SDK feature, which
> I know includes the Foo runtime, Foo doc, Foo source, and Foo examples
> features, I expect to see those features installed too, or I'd think my install
> is busted.

This is assuming Joe User understands the feature structure of the thing they chose to install. Joe who doesn't understand this structure is likely to be surprised that they installed one thing, and ten things showed up. Now they install two other things that bring along ten others each, and they have 30 things in the list of installed features. Now they realize they didn't want one of things that they installed, and they have to figure out which ten of the thirty need to be removed.

> Or are we intentionally trying to convince people that
> because features suck (heh), they should be considered second-class IUs, and in
> terms of functionality, we shouldn't care if all the features get installed?

Well, yes.  p2 actually has no knowledge of features in the Update Manager sense. We can model feature-like behaviour with p2 metadata, but we can also install other things that are not UM features (random plugins, launchers, license files, VMs, etc). UM Features are just boxes to put stuff in, and the end user doesn't (or shouldn't) care about the box that was used to deliver the things they have installed.

For people familiar with UM, there's an unfortunate terminology overload in that we use the word "feature" in the GUI when we really just mean "stuff that can be installed".
Comment 17 Nick Boldt CLA 2008-03-26 00:20:18 EDT
(In reply to comment #16)
> This is assuming Joe User understands the feature structure of the thing they
> chose to install. Joe who doesn't understand this structure is likely to be
> surprised that they installed one thing, and ten things showed up. Now they
> install two other things that bring along ten others each, and they have 30
> things in the list of installed features. Now they realize they didn't want one
> of things that they installed, and they have to figure out which ten of the
> thirty need to be removed.

Well, I might argue that the p2 installer should show ONLY the installed IU that I installed, but Help > About Features should show all the features and their contained children, like it has for the past 4 years. 

I'm less concerned about installing an SDK IU and only seeing that one item than I am if I install something like emf.ecore.editor and it pulls in a lot of other plugins, but not the associated features like emf.ecore.edit or emf.ecore. 

Why? Because if I want to remove the editor, but not ecore, I can't do that without uninstalling the whole shebang and then installing a different feature. 

(If I installed the whole SDK, chances are I'm not as "power user" as the guy who's picking and choosing smaller features and installing them one by one.)

This is sort of the problem I ran into recently w/ apt-get where I uninstalled noatun (a media player) and was told I could safely autoremove a bunch of no longer required other .debs. Doing so, I lost kde, kdm, kmix, an a whole bunch of suddenly-disabled UI tools/widgets... including my desktop manager and display environment. Removing an optional tool (like emf.ecore.editor) should NOT remove shared, required libraries like emf.ecore. But if there's only one touchpoint to remove, the user experience will be less than ideal.
 
> For people familiar with UM, there's an unfortunate terminology overload in
> that we use the word "feature" in the GUI when we really just mean "stuff that
> can be installed".

Whatever you call the box (feature, functional group, IU, hoosifudge) the fact is that these need to be chunked in logical pieces for people to manage their lifecycle, especially since we now -- finally! -- have a story for uninstalling them. (And thanks ever so much for that!!) 

Sure, it's a bit of a pain when I install some .deb and it pulls in 5 other .debs that it requires (eg., shared libraries), but I'd prefer to know what's been installed, if there are shared pieces involved. If it's a self-contained unit, then it's one box... but as soon as multiple IUs depend on the same sub-box, there ought to be a way to allow me to uninstall X (which depends on A and B) without uninstalling A and B because I also use Y, which depends on A and B). However, to be able to do so, the GUI has to be able to tell me that I've installed X, A, and B, not just X.

And if the developers have gone to the trouble to define features, there's probably a reason for them, and a value in their being exposed to the end-user. (Or maybe that's just my POV because I was intimately involved in creating all of EMF's tiny little features last year and am worried that their disappearance will introduce new problems down the line.) ;-)