Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 340298

Summary:

[transport] Downloading repositories is taking a long time

Product:

[Eclipse Project] Equinox

Reporter:

Simon Rigg <Simon.Rigg>

Component:

Assignee:

P2 Inbox <equinox.p2-inbox>

Status:

CLOSED WONTFIX

QA Contact:

Severity:

major

Priority:

CC:

andreas.hoehmann, groar, jaason2004, mauromol, mn, pascal, pdolega, rael.gc, robert, scott.wl.furry, thomas, tshepang, woskimi

Version:

3.6.2

Target Milestone:

---

Hardware:

OS:

Windows XP

Whiteboard:

stalebug

Attachments:

Description	Flags
workspace log file	none
The log file with all debug and timeout options specified	none
My eclipse.ini file, for reference	none

Description Simon Rigg

2011-03-17 06:18:51 EDT

Build Identifier: M20110210-1200

The first time Eclipse is installed and an additional feature is installed using Help->Install New Software... the Install wizard runs a job "Calculating requirements and dependencies" after you select the feature to install and this takes a very long, unreasonable amount of time (from 15 - 90 mins). It doesn't seem to matter what feature or update site you use.

I have tried installing features from http://download.eclipse.org/releases/helios and http://e-p-i-c.sf.net/updates and a local update site (via a web server and a zip file to test an archived update site)

I believe p2 is downloading site metadata to determine what is available for plugin dependency resolution and this is very slow - possibly timing out for a particular update site location.

This slowdown has increased from Eclipse 3.4 to 3.5 to 3.6.

My internet connection is ADSL (typical download rate of about 500kbps) so I believe it is not a bandwidth issue. I have also tried performing a feature installation at various times during the day with the same slow performance. My workspace and Eclipse installation is on my local drive - not a network location. I'm a Windows admin user so I have full permissions.

In addition to this problem, if you attempt to cancel the job using the red stop button then sometimes it can take a very long time to stop and you have to kill Eclipse. Other times, after it cancels and you close Eclipse, the eclipse.exe process is left running and you need to kill it manually (I waited 30 mins to see if it would die itself).

Also, I notice in the workbench trim area there is a progress bar and truncated text for this job - but I can't access it for further information when a modal dialog is in the way (i.e. the Install wizard dialog) and it seems strange to have this in the background although it is reporting progress for a job in the foreground in a different shell.

My PC is:
Windows XP Pro SP3 32-bit
Intel E2200 @ 2.2GHz
3.25GB RAM (4GB installed, but 32-bit Windows so not all available)

Reproducible: Always

Steps to Reproduce:
1. Install Eclipse Classic SDK
2. Goto Help -> Install New Software...
3. Select an update site location in the "Work with:" text box (http://e-p-i-c.sf.net/updates/testing or Helios or any one - website, local or archive - doesn't seem to matter)
4. Select a feature to install then click "Next >" and wait...and wait...

Comment 1 Simon Rigg

2011-03-17 07:01:22 EDT

Created attachment 191400 [details]
workspace log file

Any help on this issue would be greatly appreciated.

I noticed a couple of exceptions in my workspace metadata log file, which I have attached to this bug report.

It seems to be a problem with the Helios update site, accessing the http://download.eclipse.org/releases/helios/201009240900/content.jar file

Comment 2 Joe Namath

2011-03-17 14:26:45 EDT

I have same issue.. I'm also having an issue with 3.5 now as well.

Comment 3 Pascal Rapicault

2011-03-17 16:26:07 EDT

Repositories need to be contacted and downloaded. However I have never seen it to be that long even from slow networks. I suspect that there are issues at the transport level (stalled download, bogus redirect to a slow mirror, etc). that are causing the download of the files to be slow.
For example I have never have any issue download the content.jar file you mention.

Could you run with the options specified in http://wiki.eclipse.org/Equinox/p2/TransportDebugging and attach the output. Thx.

Comment 4 Rael

2011-03-17 23:18:19 EDT

New builds (mainly on Eclipse Helios) are using this option on Install New Software dialog:

- Contact all update sites during install to find required software

Uncheck this, and the "Calculating requirements and dependencies" will run fine and quickly.

I don't know why this checkbox comes checked. This is the real bug.

Comment 5 Simon Rigg

2011-03-18 05:52:40 EDT

Guys, thanks for the quick responses!

Jason - yes it was slow in 3.5 too, but seems even slower in 3.6.

Pascal - no problem, I have run with all the options on this page and will attach the log file.

Rael - the problem with unchecking that option is that it doesn't work for all update sites. It works if you're installing EPIC, for example, but we have a feature which is dependant on CDT, GEF and EMF and it doesn't work then as the update sites containing these features cannot be found, as they cannot be contacted. I assume this option is checked by default to ensure that, no matter what update site is specified in "Work with:", the dependencies can be found. Otherwise it would be fast when it works, but it wouldn't work all the time. Perhaps the best solution is to remove this option completely and only contact all update sites if dependencies cannot be found the first time round (the "quick" way). It's not clear from the wizard what the implications of checking or unchecking this option are. If it's unreasonably slow and throwing exceptions in the background (see attached logs), then it sounds like something is broken.

Comment 6 Simon Rigg

2011-03-18 05:53:44 EDT

Created attachment 191493 [details]
The log file with all debug and timeout options specified

Comment 7 Simon Rigg

2011-03-18 05:54:26 EDT

Created attachment 191494 [details]
My eclipse.ini file, for reference

Comment 8 Simon Rigg

2011-05-23 06:08:03 EDT

Hi guys, is there any status update for this issue? It still appears to be a problem in Eclipse 3.7RC2 unfortunately.

Comment 9 Thomas Hallgren

2011-05-23 06:24:52 EDT

One thing that I've discovered on 80% or so of the occasions when I've had very slow downloads is that the repository in question has bad mirrorsURL somewhere. A common mistake is to point to an existing update site that doesn't contain the right artifacts. When that happens, p2 starts searching for files on every conceivable mirror, downloading all meta-data it can find. All in vain.

Comment 10 Pascal Rapicault

2011-06-09 23:59:53 EDT

*** Bug 348851 has been marked as a duplicate of this bug. ***

Comment 11 Pascal Rapicault

2011-06-10 00:04:38 EDT

*** Bug 318014 has been marked as a duplicate of this bug. ***

Comment 12 Pascal Rapicault

2011-06-12 00:30:21 EDT

*** Bug 313702 has been marked as a duplicate of this bug. ***

Comment 13 Pascal Rapicault

2011-06-12 21:10:09 EDT

*** Bug 271790 has been marked as a duplicate of this bug. ***

Comment 14 Andreas Höhmann

2016-08-21 05:07:27 EDT

I guess there is some "loop" in (current) available update-sites ... since two (maybe more) eclipse versions (mars, neon) I got always the same "effect" sometimes ... check for updates are running until 50% then there is a kind of "endless" checking between "some" pages. Its hard to say which one ... I guess its also depends on the concrete installation ... But I believe there is a kind of loop internally :-)

If I activate fiddler to see the request pages I see there is a cycle between:

tools/gef/update-site/releases/
/tools/ajdt/34/update/
...
tools/gef/update-site/releases/
/tools/ajdt/34/update/
...
tools/gef/update-site/releases/
/tools/ajdt/34/update/

Comment 15 Andreas Höhmann

2016-08-21 05:10:30 EDT

Here is a list of requested site during such "50%" state:

http://download.scala-ide.org/update-current/p2.index
http://download.scala-ide.org/update-current/content.jar
http://download.scala-ide.org/update-current/content.xml
http://download.scala-ide.org/update-current/content.xml.xz
http://download.scala-ide.org/update-current/compositeContent.jar
http://download.scala-ide.org/update-current/compositeContent.xml
http://download.scala-ide.org/update-current/site.xml
http://marketplace.yatta.de/eclipse-mirror/tools/ajdt/34/update/p2.index
http://download.eclipse.org/tools/ajdt/34/update/p2.index
http://marketplace.yatta.de/eclipse-mirror/tools/ajdt/34/update/content.jar
http://download.eclipse.org/tools/ajdt/34/update/content.jar
http://marketplace.yatta.de/eclipse-mirror/tools/ajdt/34/update/content.xml
http://download.eclipse.org/tools/ajdt/34/update/content.xml
http://marketplace.yatta.de/eclipse-mirror/tools/ajdt/34/update/content.xml.xz
http://download.eclipse.org/tools/ajdt/34/update/content.xml.xz
http://marketplace.yatta.de/eclipse-mirror/tools/ajdt/34/update/compositeContent.jar
http://download.eclipse.org/tools/ajdt/34/update/compositeContent.jar
http://marketplace.yatta.de/eclipse-mirror/tools/ajdt/34/update/compositeContent.xml
http://download.eclipse.org/tools/ajdt/34/update/compositeContent.xml
http://marketplace.yatta.de/eclipse-mirror/tools/ajdt/34/update/site.xml
http://download.eclipse.org/tools/ajdt/34/update/site.xml
http://dev.eclipse.org:443
http://www.eclipse.org:443
http://marketplace.yatta.de/eclipse-mirror/tools/gef/updates/releases/site.xml/p2.index
http://download.eclipse.org/tools/gef/updates/releases/site.xml/p2.index
http://marketplace.yatta.de/eclipse-mirror/tools/gef/updates/releases/site.xml/content.jar
http://download.eclipse.org/tools/gef/updates/releases/site.xml/content.jar
http://marketplace.yatta.de/eclipse-mirror/tools/gef/updates/releases/site.xml/content.xml
http://download.eclipse.org/tools/gef/updates/releases/site.xml/content.xml
http://marketplace.yatta.de/eclipse-mirror/tools/gef/updates/releases/site.xml/content.xml.xz
http://download.eclipse.org/tools/gef/updates/releases/site.xml/content.xml.xz
http://marketplace.yatta.de/eclipse-mirror/tools/gef/updates/releases/site.xml/compositeContent.jar
http://download.eclipse.org/tools/gef/updates/releases/site.xml/compositeContent.jar
http://marketplace.yatta.de/eclipse-mirror/tools/gef/updates/releases/site.xml/compositeContent.xml
http://download.eclipse.org/tools/gef/updates/releases/site.xml/compositeContent.xml
http://marketplace.yatta.de/eclipse-mirror/tools/gef/updates/releases/site.xml
http://download.eclipse.org/tools/gef/updates/releases/site.xml
http://download.scala-ide.org/update-current-35/p2.index
http://download.scala-ide.org/update-current-35/content.jar
http://download.scala-ide.org/update-current-35/content.xml
http://download.scala-ide.org/update-current-35/content.xml.xz
http://download.scala-ide.org/update-current-35/compositeContent.jar
http://download.scala-ide.org/update-current-35/compositeContent.xml
http://download.scala-ide.org/update-current-35/site.xml
http://marketplace.yatta.de/eclipse-mirror/tools/gef/update-site/releases/site.xml/p2.index
http://download.eclipse.org/tools/gef/update-site/releases/site.xml/p2.index
http://marketplace.yatta.de/eclipse-mirror/tools/gef/update-site/releases/site.xml/content.jar
http://download.eclipse.org/tools/gef/update-site/releases/site.xml/content.jar
http://marketplace.yatta.de/eclipse-mirror/tools/gef/update-site/releases/site.xml/content.xml
http://download.eclipse.org/tools/gef/update-site/releases/site.xml/content.xml
http://marketplace.yatta.de/eclipse-mirror/tools/gef/update-site/releases/site.xml/content.xml.xz
http://download.eclipse.org/tools/gef/update-site/releases/site.xml/content.xml.xz
http://marketplace.yatta.de/eclipse-mirror/tools/gef/update-site/releases/site.xml/compositeContent.jar
http://download.eclipse.org/tools/gef/update-site/releases/site.xml/compositeContent.jar
http://marketplace.yatta.de/eclipse-mirror/tools/gef/update-site/releases/site.xml/compositeContent.xml
http://download.eclipse.org/tools/gef/update-site/releases/site.xml/compositeContent.xml
http://marketplace.yatta.de/eclipse-mirror/tools/gef/update-site/releases/site.xml
http://download.eclipse.org/tools/gef/update-site/releases/site.xml
http://example.org/update/site/p2.index
http://example.org/update/site/content.jar
http://example.org/update/site/content.xml
http://example.org/update/site/content.xml.xz
http://example.org/update/site/compositeContent.jar
http://example.org/update/site/compositeContent.xml
http://example.org/update/site/site.xml
http://nodj.github.io/AutoDeriv/p2.index
http://nodj.github.io/AutoDeriv/content.jar
http://nodj.github.io/AutoDeriv/content.xml
http://nodj.github.io/AutoDeriv/content.xml.xz
http://nodj.github.io/AutoDeriv/compositeContent.jar
http://nodj.github.io/AutoDeriv/compositeContent.xml
http://nodj.github.io/AutoDeriv/site.xml
http://pmd.sourceforge.net/eclipse/p2.index
http://pmd.sourceforge.net/eclipse/content.jar
http://pmd.sourceforge.net/eclipse/content.xml
http://pmd.sourceforge.net/eclipse/content.xml.xz
http://pmd.sourceforge.net/eclipse/compositeContent.jar
http://pmd.sourceforge.net/eclipse/compositeContent.xml
http://theit.github.com/m2e-lifecycle-axistools/site/p2.index
http://theit.github.io/m2e-lifecycle-axistools/site/p2.index
http://theit.github.com/m2e-lifecycle-axistools/site/content.jar
http://theit.github.io/m2e-lifecycle-axistools/site/content.jar
http://theit.github.com/m2e-lifecycle-axistools/site/content.xml
http://theit.github.io/m2e-lifecycle-axistools/site/content.xml
http://theit.github.com/m2e-lifecycle-axistools/site/content.xml.xz
http://theit.github.io/m2e-lifecycle-axistools/site/content.xml.xz
http://theit.github.com/m2e-lifecycle-axistools/site/compositeContent.jar
http://theit.github.io/m2e-lifecycle-axistools/site/compositeContent.jar
http://theit.github.com/m2e-lifecycle-axistools/site/compositeContent.xml
http://theit.github.io/m2e-lifecycle-axistools/site/compositeContent.xml
http://theit.github.com/m2e-lifecycle-axistools/site/site.xml
http://theit.github.io/m2e-lifecycle-axistools/site/site.xml
http://scala-ide.dreamhosters.com/nightly-update-worksheet-scalaide21-210/site/p2.index
http://scala-ide.dreamhosters.com/nightly-update-worksheet-scalaide21-210/site/content.jar
http://scala-ide.dreamhosters.com/nightly-update-worksheet-scalaide21-210/site/content.xml
http://github.com:443
http://github.com:443
http://github.com:443
http://github.com:443
http://github.com:443
http://github.com:443
http://github.com:443


I have this p2 settings in my eclipse.ini:

-Dorg.eclipse.ecf.provider.filetransfer.retrieve.connectTimeout=120000
-Dorg.eclipse.ecf.provider.filetransfer.retrieve.readTimeout=120000
-Dorg.eclipse.ecf.provider.filetransfer.retrieve.retryAttempts=20
-Dorg.eclipse.ecf.provider.filetransfer.retrieve.closeTimeout=120000
-Dorg.eclipse.ecf.provider.filetransfer.browse.connectTimeout=120000
-Dorg.eclipse.ecf.provider.filetransfer.browse.readTimeout=120000
-Dorg.eclipse.ecf.provider.filetransfer.httpclient.browse.connectTimeout=120000
-Dorg.eclipse.ecf.provider.filetransfer.httpclient.retrieve.connectTimeout=120000
-Dorg.eclipse.ecf.provider.filetransfer.httpclient.retrieve.readTimeout=120000
-Dorg.eclipse.ecf.provider.filetransfer.httpclient4.browse.connectTimeout=120000
-Dorg.eclipse.ecf.provider.filetransfer.httpclient4.retrieve.connectTimeout=120000
-Dorg.eclipse.ecf.provider.filetransfer.httpclient4.retrieve.connectTimeout=120000
-Dorg.eclipse.ecf.provider.filetransfer.httpclient4.retrieve.readTimeout=120000
-Dorg.eclipse.equinox.p2.transport.ecf.retry=5
-Declipse.p2.max.threads=10
-Declipse.p2.force.threading=true
-Declipse.p2.mirrors=true


Maybe the "transport.ecf.retry=5" cause "my" problems?

Comment 16 Eclipse Genie

2020-03-06 16:21:10 EST

This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. As such, we're closing this bug.

If you have further information on the current state of the bug, please add it and reopen this bug. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.