Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 54961

Summary: Updating through http proxy (almost) locks PC with 100% CPU usage
Product: [Eclipse Project] Platform Reporter: Constantine Dokolas <cdokolas>
Component: Update (deprecated - use Eclipse>Equinox>p2)Assignee: Konrad Kolosowski <konradk>
Status: RESOLVED FIXED QA Contact:
Severity: major    
Priority: P3 CC: david_williams
Version: 3.0   
Target Milestone: 3.0 M9   
Hardware: PC   
OS: Windows 2000   
Whiteboard:
Bug Depends on:    
Bug Blocks: 123108    
Attachments:
Description Flags
console dump results none

Description Constantine Dokolas CLA 2004-03-16 09:14:31 EST
I've correctly specified the http proxy for updates, which requires
authentication (NTLM by default, although has also worked with basic for other
applications, Java and non-Java) on a different domain than the client or proxy.
Non-NTLM authentication requires the "AUTH-DOMAIN\username" format for the username.
As soon as the updater requires some http communication, the java process hits
100% cpu usage, severely crippling other processes (i.e. they're "as slow as
molasses in January"). This continues indefinitely unless Cancel is pressed.
Comment 1 Dorian Birsan CLA 2004-03-16 09:53:29 EST
Can you provide a test update site that I can try to reproduce the problem?
Comment 2 Constantine Dokolas CLA 2004-03-16 10:28:12 EST
I first experienced the problem when I tried to install the Spindle plug-in
through their update site http://spindle.sourceforge.net/updates through the
"new components" updater but I had the same results when trying the "installed
components" updater.
I'm using M7 and the Sun 1.4.2 J2RE. I tried switching eclipse to the beta J2RE
1.5 but I got the same.
One thing to note is that I had never tried to update Eclipse before, as only
recently I gained access to the web, and only through the proxy. My case is
unusual and rare, requiring an ISA server (the web proxy), two domains, the
client on one domain together with the server, which in turn allows access only
through a specific account on the second domain. There may be a less complicated
combination that still experiences the problem, but I don't have any access to
try anything.
My first question would be "why don't I get an authorization dialog?"
Comment 3 Konrad Kolosowski CLA 2004-03-16 10:46:27 EST
Can you try starting Eclipse with -console option so you get a command window 
and press Ctrl-Break in this window, when you get to 100%CPU point?  This 
should produce thread dump which can help determine what happens.
Comment 4 Dorian Birsan CLA 2004-03-16 15:39:05 EST
See bug 37205, as they've had success with M6 and Sun JDK 1.4.2_03.
Perhaps your unusual setup is not handled correctly.
Comment 5 Konrad Kolosowski CLA 2004-03-17 10:38:13 EST
Constantine,
-console does not seem to work when starting Eclipse from a shortcut.  Open a 
dos prompt, switch to directory
	C:\Dev\eclipse
and type
	eclipse.exe -console -data C:\Dev\workspace -vm c:\j2sdk1.4.2
\bin\javaw -vmargs -Xmx256M
there.  It should work from command line.
Comment 6 Constantine Dokolas CLA 2004-03-18 03:21:15 EST
Created attachment 8648 [details]
console dump results

This is a copy of the Ctrl-Break dump on the console. It is reformated to
un-wrap line wraps (not sure if I got it right, though).
Comment 7 Constantine Dokolas CLA 2004-03-18 03:25:55 EST
BTW,

The command that worked was:

eclipse.exe -console -data C:\Dev\workspace -vm c:\j2sdk1.4.2\bin\java -vmargs
-Xmx256M

(notice the java.exe instead of javaw.exe)
Comment 8 Konrad Kolosowski CLA 2004-03-18 15:10:57 EST
The console show a NPE, that leads to a loop.  I have release a fix, but was 
not able to reproduce the original problem.  What was the proxy host you 
specified?
Please verify in builds nightly >20040318 or the next week integration build.
Comment 9 Constantine Dokolas CLA 2004-03-19 02:41:05 EST
The proxy is not visible outside our LAN as it is part of our firewall. I will
try to get build and test it on a separate setup. Would that satisfy you?
Comment 10 Konrad Kolosowski CLA 2004-03-19 13:19:54 EST
I would still like to know how you specified the host.  Have you just typed a 
host name in the Update preferences, or have you specified IP, or use any 
other format that uncovered this problem.
Comment 11 Constantine Dokolas CLA 2004-03-26 03:07:29 EST
Sorry for the delay. Here it is:

On the "Install/Update" page in the Preferences, I selected "Enable HTTP proxy 
connection" and specified the host by name (also tried by IP address, but no 
change) and the port (happens to be 8080).
Comment 12 Konrad Kolosowski CLA 2004-03-26 10:42:13 EST
Thanks.
Comment 13 Constantine Dokolas CLA 2004-03-30 07:07:44 EST
- Update for M8

I'm testing M8 and under the same settings I now get the following lines for
"Search for updates of the currently installed features":

Unable to access "http://update.eclipse.org/updates"
  Unable to access Site stream: "http://update.eclipse.org/updates" [Server
returned HTTP response code: 407 for URL: http://update.eclipse.org/updates]
  Unable to access Site stream: "http://update.eclipse.org/updates/site.xml"
[Server returned HTTP response code: 407 for URL:
http://update.eclipse.org/updates/site.xml]

These appear in a dialog box.
Comment 14 Dorian Birsan CLA 2004-03-30 09:45:05 EST
There were times when eclipe.org was not available.
Have you tried pointing your browser to those sites and check the connection?
Comment 15 Constantine Dokolas CLA 2004-03-30 10:12:28 EST
No relation, this is a 407 "Proxy Authentication Required" as per RFC 2616,
para. 10.4.8

I have no idea why this would be unexpected by the installer/updater.

BTW, sorry for "reopening" the bug ppl, if improper.
Comment 16 Dorian Birsan CLA 2004-04-12 23:37:48 EDT
Konrad, can you re-investigate this when you have a chance? thx
Comment 17 Konrad Kolosowski CLA 2004-04-19 16:30:06 EDT
My fixes in that code are shots in the dark, as I do not have access to a 
proxy, and cannot debug it.
It would help to understand what is going on, if Constantine could you provide 
more guiding details:

Has a fix talked in comment #8 fixed a 100% CPU original problem?

The problem in comment #13, I assume it is the same proxy configuration, were 
you prompted for the password, typed a correct or incorrect one to get the 
error message?
Do you know if the error prevents update from fetching updates from other 
sites, for example a local site (on your disk), or are you just concerned that 
error dialog is too imprecise and intrusive?
Comment 18 Constantine Dokolas CLA 2004-04-20 03:24:30 EDT
1. The fix mentioned in comment #8 fixed the 100% CPU problem.

2. The proxy configuration is the same. I never get a chance to enter a
username/password. The error happens for other sites too. I don't know how to
test with an on-disk update site.

I'll be happy to fill in on any other details.
Comment 19 Dorian Birsan CLA 2004-05-03 11:26:41 EDT
Got this link from someone, it may help with settings:

Link to Microsoft technote KB295667
http://support.microsoft.com/default.aspx?scid=kb;en-us;295667
Comment 20 Konrad Kolosowski CLA 2004-05-09 02:53:40 EDT
I have installed proxy at home and looked at the code around 
UpdateManagerAuthenticator:
1.  If user cancels dialog once they can naver entered a password again for 
the same proxy - they needed to restart Eclipse.  Fixed this.
2.  Most values (except the port number) obtained from Java by 
Authenticator.getReqesting...() can be null.  Only few are documented in JDK 
docs.  This can result in NPEs in UpdateManagerAuthenticator.  I have fixed 
one as in comment #8.  Now removed rest assumptions of not null.
3.  UpdateManagerAuthenticator was attempting to always create URL from values 
obtained from Authenticator.  This was likely to fail.  Event if values were 
not null, a protocol could have been "http/1.1" which new URL will not accept.
Upon MalformedURL UpdateManagerAuthenticator would return blank user and one 
space password.  If it could not handle authentication, it should return null.
4.  UpdateManagerAuthenticator contained a code for retrieving, but never 
saving user/passwords in the Platform.  Platform stores user passwords indexed 
by multiple keys, one of them URL, which makes it inapropriate for keeping 
even proxy passwords.
5.  UpdateManagerAuthenticator had a code to cache last authentication request 
and corresponding user/pass combination.

After adding fixes and refactoring UpdateManagerAuthenticator is reduced to 
few lines of logic plus the dialog class.  There is really nothing left that 
can fail there.

UpdateManagerAuthenticator had a check for authentication scheme and was only 
promting for password if it was "basic".  Recently "ntlm" was added.  Now the 
check is removed and user will be prompt for all types of autentication that 
JRE can handle.  I have tested "basic", "digest", "ntlm".
There can be only one Authenticator registered in Eclipse (1 per VM).  I have 
made sure that UserValidationDialog is passed not null string (possibly 
empty).  Since UpdateManagerAuthenticator no longer tries to create URLs. it 
should handle authentication for password protectes sites or SOCKS proxy 
(there is no preference page for SOCKS proxy, but it can be configured on 
system properties).


As for Constantine problem, it should be fixed.  If it was canceling problem 
it is gone.  It is also not clear what authentication scheme the proxy is 
using.  In original report you say: Non-NTLM authentication requires the "AUTH-
DOMAIN\username".  I know the domain can be passed with user name for NTLM, 
but have not seen this format for other schemes like basic.  If your proxy was 
not basic and not ntlm, it could have not worked as there was a specific check 
for these two.

Code in the builds 20040409 or newer.
Comment 21 Constantine Dokolas CLA 2004-05-10 03:13:59 EDT
> Code in the builds 20040409 or newer.

Shouldn't that have been 20040509?

Will test ASAP. Which is better, nightly or integration?

BTW, good job!
Comment 22 Konrad Kolosowski CLA 2004-05-10 09:21:40 EDT
Available in >=20040509.  I cannot say which will be a good build. 20040509 
and 10 have (second) startup problems.  I suggest waiting for a good 
integration build this week.
Comment 23 Constantine Dokolas CLA 2004-05-13 04:40:09 EDT
Tested with I20040512. Same results as in comment #13.

Perhaps this integration build doesn't have the changes? How will I know when to
get an integration build?
Comment 24 Konrad Kolosowski CLA 2004-05-13 09:11:47 EDT
The first integration build with the changes is I20040512 20:00.
Comment 25 Constantine Dokolas CLA 2004-05-14 08:24:37 EDT
Congratulations!!! I tried I20040513 and it works!!!

One thing to note is that it asks me for the username/password 2-3 times. That
may be intentional though...

Details:
Tried the Eclipse update site, it worked (no updates).
Tried the Splindle update site, it didn't work (there must be a problem with
thir update info, there is a parsing error).
Tried the AJDT update site, it worked (download started).

Comments:
The Eclipse update asked 3 times for username/password. Otherwise, there were no
problems
The AJDT update asked 2 times for username/password. At least, I think it was
twice only...

Problem:
After I tried the Spindle update I couldn't get the "Find and install..."
dialog. I restarted Eclipse, but it still wouldn't come up. I figured the failed
update had caused a problem, so I wiped the workspace. The new workspace was ok,
so I tried the AJDT update. I canceled the AJDT update because I couldn't wait
for it to finish, and the "Find and install..." dialog wouldn't come up again.
The strangest thing is, when I open Eclipse and I click "Find and install...", I
get a couple of seconds of hourglass cursor, and then nothing. If I try again
(not closing Eclipse in between), I don't get any hourglass cursor (i.e. I get
nowhere fast! :)
Comment 26 Dorian Birsan CLA 2004-05-14 08:48:48 EDT
The authentication prompt comes up 2 times because the code tries to access 
the site first, then the site/site.xml. 
If you add the site as http://path/to/site/site.xml, I think you'll only get 
one prompt.

There was a problem caused by a UI bug that will be fixed in today's 8AM 
build, that caused our wizard to now come up when there is a workspace from a 
previous run.
Comment 27 Constantine Dokolas CLA 2004-05-14 09:14:46 EDT
Shouldn't there be some caching of the username/password then?
Comment 28 Konrad Kolosowski CLA 2004-05-14 10:57:33 EDT
It would be nice, but new code needs to be written for that.  Adding layer of 
caching may lead to problems.  Consider logging into a ntlm proxy that with 
recently changed password.  VM may attempt to:
1 log-in with a password that it cached by JRE,
2 then it will send some generic password,
3 then it will send password it gets from Windows
4 then it will ask update for a password, and update will provide a cached one,
5 it will ask update again and update will pop-up a dialog that user will 
finally have a chance to fill in the correct password.

Every password caching adds another unsuccessful login attempt.  A proxy may 
have a limit on number of retries and block the user out.

This require more testing, beyond ensuring correctness of the code.  This bug 
was about getting it to work and I am satisfied if it works.  Caching is an 
improvement with a potential to have negative effect and we should not 
experiment with this for 3.0.  You may open a enhancement request that we will 
consider later not near the end of the release.