Bugzilla – Full Text Bug Listing
| Summary: | Many duplicate HTTP GETs during update operations | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | [Eclipse Project] Platform | Reporter: | Gordon Hirsch <gordon.hirsch> | ||||||
| Component: | Update (deprecated - use Eclipse>Equinox>p2) | Assignee: | Platform-Update-Inbox <platform-update-inbox> | ||||||
| Status: | RESOLVED FIXED | QA Contact: | |||||||
| Severity: | major | ||||||||
| Priority: | P2 | CC: | francois, Joel.Kamentz, pombredanne | ||||||
| Version: | 3.1.1 | ||||||||
| Target Milestone: | 3.2 RC7 | ||||||||
| Hardware: | PC | ||||||||
| OS: | Windows XP | ||||||||
| Whiteboard: | |||||||||
| Bug Depends on: | 144876 | ||||||||
| Bug Blocks: | |||||||||
| Attachments: |
|
||||||||
|
Description
Gordon Hirsch
yes, there has been some discussion on a similar issue in another bug. I think this should be looked at. Created attachment 17612 [details]
Proposed Patch for 84103
Uses existing connection to avoid duplicated GETs
Was 63932 the similar bug you were referring to? It would be great if some of the redundancies described there could be reduced. I'm not sure I described this problem well above, but it is a lower-level problem that causes almost every GET to be issued twice, consecutively. I've attached a very simple patch that seems to fix the problem. I think it is worth looking at independently of the more complicated issues. Created attachment 17613 [details]
modified patch
Thanks Gordon. I have modified the patch to set the connection to null after a
disconnect, and also do it for OtherResponse type.
No, not that bug (unless there is typo in the bug number you entered :-) There are a couple of bugs opened regarding networking problem, and one of them in particular talks about closing connections when opening fails (but this can't be done, at least using java API's). I'll dig the bug later... fixed as per my modified patch. Thanks Gordon. Ok, the problem closing connections sounds like https://bugs.eclipse.org/bugs/show_bug.cgi?id=81967. Thanks for the quick resolution. I am having a problem with repeated gets for a jar which has already been downloaded. I am re-opening this bug, which seems the best match, but it is potentially also related to https://bugs.eclipse.org/bugs/show_bug.cgi?id=81967. To some extent, there are two problems. The overriding problem is that the update manager downloads a plugin and then hits the web server _again_ just to check the last modified timestamp. Using if-modified-since (or whatever that http header is) and generally remembering the timestamp the first time around is the way to go. A sub-problem is that the timestamp request uses GET (rather than HEAD) and doesn't read the full response or otherwise close the connection. The _http_ response isn't using the fact that the connection is an _http_ connection! As for the solution, the best fix would be to get rid of the redundant request. However, in general, HttpURLConnection.disconnect() is probably too heavy-handed, as it may prematurely disconnect an otherwise re-usable connection to the server? Better would be to use the HEAD method for mere timestamp checks. This may sound somewhat trivial, but it really bogs down an update site we have running on Apache. The Eclipse update manager fetches a bunch of plugins to temp files. Then, as part of jar validation, it hits the server again for each jar file using the GET method. However, it doesn't actually read the file contents which are sent back. Because HTTP 1.1 connection re-use is involved, this causes everything (the update process within Eclipse) to hang until the server times out on that request. Once that happens, the server responds to the next request, tries to send the file contents which the Eclipse Update Manager isn't paying attention to, and it all happens again. The update site in question has around 175 plugins totalling over 115 meg. On our lan, the update manager performs the actual download in maybe 2 minutes. Then, because of this problem, verifying the downloaded jars takes HOUR(S)!!! This is with Eclipse 3.1.1. I've tried it with Java 1.5.0_06 and Java 1.4.2_10. Besides setting the request method to HEAD, another way to avoid this particular problem is to call getInputStream on the connection and then immediately call close() on the stream. But, using HEAD would be better, and getting rid of the extra network traffic better still. [I apparently cannot re-open this bug (or change it to Eclipse 3.1.1), as I'm not the original submitter. Am I supposed to create a dup bug? I'll try asking Gordon to re-open this.] Re-opening per Joel's request. fixed as part of the fix for bug 144876 |