Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 276884

Summary: [ui] Newly created update site doesn't show up in the "Work with" list
Product: [Eclipse Project] Equinox Reporter: Tomasz Zarna <tomasz.zarna>
Component: p2Assignee: Susan McCourt <susan>
Status: VERIFIED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: henrik.lindberg, john.arthorne, pascal, remy.suen, susan
Version: 3.5   
Target Milestone: 3.6 M5   
Hardware: PC   
OS: Windows XP   
Whiteboard:
Attachments:
Description Flags
Install wizard and Available Sites pref page opened from it
none
slider that controls event responsiveness
none
patch to RepositorySelectionGroup none

Description Tomasz Zarna CLA 2009-05-19 09:18:05 EDT
3.5RC1

I added an update site[1] via "Add..." button next to "Work with" list. But the site didn't show up in the list of available sites in the drop down list. I can see it on "Available Software Sites" pref page though. I need to close the "Install" wizard and open it again to see the JDT site.


[1] "JDT Optional Plug-ins" located under http://www.eclipse.org/jdt/ui/update-site
Comment 1 Susan McCourt CLA 2009-05-19 12:33:56 EDT
I just tried this (added the JDT site via the Add... button) and I immediately see the site in the combo, the "Pending..." node in the list, and then the content.

Is there anything in the log indicating a problem?  It sounds like something failed as the UI updated.
Comment 2 Tomasz Zarna CLA 2009-05-20 04:39:46 EDT
Created attachment 136452 [details]
Install wizard and Available Sites pref page opened from it

There is nothing in the log, sorry. Let me provide you more detailed steps:

1. Make sure you haven't installed anything from the site you're about to add.
2. Open Help > Install New Software
3. Click "Add..." button next to "Work with" list
4. Enter "JDT Optional Plug-ins" as Name and "http://www.eclipse.org/jdt/ui/update-site" as Location
=> In my case the list flickered like it was updating but nothing actually happened ie the site was not in the combo.
=> I do admit it sometimes works as you described it in comment 1.

Anyway, I don't know what's happening under the hood, but maybe the fact that the "strange entry" mentioned on bug 276886 is not added to the list on the pref page is relevant.
Comment 3 Susan McCourt CLA 2009-05-20 12:49:37 EDT
(In reply to comment #2)
> Created an attachment (id=136452) [details]
> Install wizard and Available Sites pref page opened from it
> 
> There is nothing in the log, sorry. Let me provide you more detailed steps:
> 
> 1. Make sure you haven't installed anything from the site you're about to add.
> 2. Open Help > Install New Software
> 3. Click "Add..." button next to "Work with" list
> 4. Enter "JDT Optional Plug-ins" as Name and
> "http://www.eclipse.org/jdt/ui/update-site" as Location
> => In my case the list flickered like it was updating but nothing actually
> happened ie the site was not in the combo.
> => I do admit it sometimes works as you described it in comment 1.
> 
> Anyway, I don't know what's happening under the hood, but maybe the fact that
> the "strange entry" mentioned on bug 276886 is not added to the list on the
> pref page is relevant.
> 

I'll play with this a bit more.  It could be that the failure occurs while processing the referenced site, so that the UI code doesn't complete the population of the combo.  The appearance in the pref page means that the site was successfully "registered" with p2, but while "loading" it (which involves generating metadata and processing those site references), something went wrong and whatever went wrong did not get logged or raised as an exception.
Comment 4 DJ Houghton CLA 2009-05-25 13:35:08 EDT
This has happened to me as well but I don't have a reproducible test case for it yet,
Comment 5 John Arthorne CLA 2009-05-26 12:12:47 EDT
This happened to me today while testing RC2. It occurred on my Vista box, but not on my XP box. It clearly seems timing related. In the failure case, I saw the works with combo flicker several times, and then settled on the default "type or select a site". In both cases I was adding the 3.5-I-builds site to a fairly clean 3.5 RC2 install (not upgraded from any previous build).

I suspect it is initially selecting the new repository, but them some event occurs during loading of the repository that causes the selection to change/reset. Perhaps it happens when referenced repositories are added from the loaded repo?
Comment 6 Susan McCourt CLA 2009-05-26 12:23:40 EDT
>Perhaps it happens when referenced repositories are added from
> the loaded repo?

Yes, I think it's related to that.  See comment #2 and comment #3...it appears that the referenced site did not get added successfully when the error is observed.  So I think something fails unexpectedly/quietly in that case and the load does not complete.  

It doesn't seem to be a composite repo issue, since in Tomasz' case, it's a update site referenced in one of the features. 

Comment 7 Susan McCourt CLA 2009-10-21 11:10:22 EDT
will investigate in M4 and look at the feasibility of a patch for 3.5.2
Comment 8 Nathan Gervais CLA 2009-10-21 11:15:49 EDT
Pascal asked me to post my experience here.

I'm building some functionality into the Eclipse Marketplace site that will allow users to access a composite repository of there favorite plugins from the site.

That repo is available here http://marketplace.eclipse.org/user/896/p2/repo

On my test machine i downloaded the latest copy of the PDT Package unzipped and ran it.  Created a new workspace and then launched P2.   I added this url but it is not displayed in teh Work With list, however it does show up if i click the "Available Software Sites" link underneath the list.
Comment 9 Susan McCourt CLA 2009-10-21 11:18:57 EDT
cc'ing Henrik.  I'll look into this one, but I'm wondering if you know of any
existing bugs in transport/repo where we might be swallowing an error and aborting
a repo operation vs reporting to the user.
Comment 10 Susan McCourt CLA 2009-10-21 11:20:26 EDT
(In reply to comment #8)
> Pascal asked me to post my experience here.
> 
> I'm building some functionality into the Eclipse Marketplace site that will
> allow users to access a composite repository of there favorite plugins from the
> site.
> 
> That repo is available here http://marketplace.eclipse.org/user/896/p2/repo
> 
> On my test machine i downloaded the latest copy of the PDT Package unzipped and
> ran it.  Created a new workspace and then launched P2.   I added this url but
> it is not displayed in teh Work With list, however it does show up if i click
> the "Available Software Sites" link underneath the list.

Thanks, Nathan.  If feasible, can you ensure that the repo that is consistently showing the failure will remain available for the next few weeks?  The theory is that there is some failure in one of the repo references that is not getting handled correctly, so I want to ensure we debug against the same exact repo/children.
Comment 11 Henrik Lindberg CLA 2009-10-23 11:44:53 EDT
(In reply to comment #9)
> cc'ing Henrik.  I'll look into this one, but I'm wondering if you know of any
> existing bugs in transport/repo where we might be swallowing an error and
> aborting
> a repo operation vs reporting to the user.

I am not aware of anything like that being reported. The only layer that catches errors and does something else is in the mirror logic - i.e. try a different mirror.
Comment 12 Susan McCourt CLA 2009-11-12 20:24:52 EST
(In reply to comment #8)
> Pascal asked me to post my experience here.
> 
> I'm building some functionality into the Eclipse Marketplace site that will
> allow users to access a composite repository of there favorite plugins from the
> site.
> 
> That repo is available here http://marketplace.eclipse.org/user/896/p2/repo
> 
> On my test machine i downloaded the latest copy of the PDT Package unzipped and
> ran it.  Created a new workspace and then launched P2.   I added this url but
> it is not displayed in teh Work With list, however it does show up if i click
> the "Available Software Sites" link underneath the list.

I just tried this after doing some rework in this area of the code.  I don't see the problem you mention, however there is no content showing in the repository...
Comment 13 Susan McCourt CLA 2009-11-25 19:20:34 EST
I think this is fixed via a new batching mechanism in bug 234489.  Before, we had several different hacks across the UI to ignore events and try to prevent the flicker/flash of repo references.

I've pulled all of those hacks out and introduced a centralized batching mechanism in the UI.  It is set up so that repo events can be ignored during a load, and at the end of the load, the UI can optionally update the repo list.  I tried all the cases here and they work (except that the marketplace repo is empty...but the selection remained and it shows up in the combo).

The fact that this was a timing problem could have been because of the asynchronous nature of provisioning events as well as some unnecessary async execs in the UI code.

I've ensured that all async execs in the UI occur only when necessary (when we are in a background thread) so that event calls from the UI happen synchronously. I've also ensured we have a synchronous provisioning listener so that we know our batching is timing right with theare nowasynchronous Provisioning events were sneaking in.  The batching mechanism now requires a synchronous listener so that we know our batch timing is right.

I won't mark this fixed until we have a build to test it...
Comment 14 Susan McCourt CLA 2009-12-01 13:06:43 EST
marking fixed, this no longer happens in the test build (I20091126-1807).
Moving milestone to M5 since this won't be released until the merge.
Comment 15 Susan McCourt CLA 2010-01-25 13:21:04 EST
Reopening.
While smoke testing I20100124-2000, I saw this error for the very first time!

I pasted this site in the work with combo and hit enter:
http://www.oaklandsoftware.com/updates/mulesoft/mdi

The tree below showed "no site selected" and the combo was not refreshed to show the new site, although it was known.  Hitting enter again said "duplicate location" and closing/reopening the wizard fixed it.

I think I can reproduce if 
- the repo is removed
- the p2 repo caches must be cleared

The fact that the cache makes a difference suggests a timing issue or else event flow is slightly different the first time.
Comment 16 Susan McCourt CLA 2010-01-26 12:34:17 EST
I can reproduce this at will in I20100126-0100, WinXP.
Steps:
- pasted this site in the work with combo
   http://www.oaklandsoftware.com/updates/mulesoft/mdi
- hit enter
- you can see that the repo is being loaded (in the progress status area) but the combo does not contain it and therefore the content area still says "no site selected"

I can actually reproduce this in the same session by removing the repository and repeating the steps.  (ie, no clearing of cache).  In one case, the repository never even showed up in the pref page after the fact, indicating that it hadn't been truly added to the manager (or perhaps that it was removed)...

I have not been able to reproduce the bug in a self hosted workbench, regardless of whether caches are cleared or not.  However this workbench also has the UI admin loaded which may be handling/responding to events and changing the event timing.
Comment 17 Susan McCourt CLA 2010-01-26 12:35:47 EST
marking M5 for investigation, the problem seems to occur more frequently than in the past.
Comment 18 Susan McCourt CLA 2010-01-26 13:34:51 EST
Got it.
This can be reproduced in a self-hosted workbench (happy for me because I can debug and verify a fix more easily).

It occurs when background repository loading is in progress while the repo is being added.  So, if you don't have many repos in your available software sites list, or you have a fast connection, it's harder to reproduce.

I reproduced by:
- self hosted workbench (no repos by default)
- import a list of sites from the pref page
- OK (no repositories get loaded as a result of this)
- now install new software
- very quickly paste in a repo you haven't seen before
- hit enter

This is most certainly a different code flow than when this was originally reported.  Note that bug 234489 changed the event flow significantly and attempts to reduce noise by batching events.

Problem is when the UI does something during one of these batches that needs to update the UI.  This was almost certianly introduced by the fix for bug 299586.

We are stuck in the push and pull of overresponding and underresponding to the plethora of repository events that occur in the course of adding/loading a repo (esp composite repos).  By fixing many of these "flash," "losing selection", "repos are appearing and disappearing" problem, we've introduced an "underresponding" problem.
Comment 19 Susan McCourt CLA 2010-01-26 14:13:13 EST
Created attachment 157326 [details]
slider that controls event responsiveness

I can't help myself.  Pascal and I were joking about adjusting the UI responsiveness to repo events using a slider so you could dial in which bugs you wanted...

Here's a mockup (based on windows user access control notifications)
Comment 20 Susan McCourt CLA 2010-01-26 14:21:41 EST
I've got a fix, working on verifying related bugs.
Here's a list for future reference when verifying with a real build.
(I didn't take the time to draw where each bug falls in the slider in the picture, but believe me, I know.)

bug 276192
bug 282176
bug 298806
bug 299586
bug 300112
Comment 21 Susan McCourt CLA 2010-01-26 18:15:16 EST
Created attachment 157354 [details]
patch to RepositorySelectionGroup

This patch ensures that the reository combo will update the combo whenever it adds a repository (vs. relying on events that may or may not be ignored depending on what else is happening).

This is a safe fix.  It adds one line of code to update the combo.  Note that the same call is used 20 lines above in the case where the dialog was opened to add the repo.  That code path does not suffer from this problem.  This patch causes the "typed and hit enter" case to be treated the same as the "opened a dialog" case.
Comment 22 Susan McCourt CLA 2010-01-26 18:42:07 EST
committed to head >20100126 and tagged for the next I-build.
I verified that the related bugs are still fixed (using a self-hosted workbench).  I will verify them all again on the new build on Wednesday.
Comment 23 Susan McCourt CLA 2010-01-27 10:39:57 EST
verified on I20100127-0100, WinXP
Comment 24 Susan McCourt CLA 2010-01-27 11:05:54 EST
verified all associated bugs.