Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 335153

Summary:

Regression: p2 downloads are much slower due to picking remote artifacts even when a local file: URL is available

Product:

[Eclipse Project] Equinox

Reporter:

Meng Xin Zhu <kane.zhu>

Component:

Assignee:

DJ Houghton <dj.houghton>

Status:

RESOLVED FIXED

QA Contact:

Severity:

critical

Priority:

CC:

helmut.haigermoser, jeffmcaffer, kane.mx, mober.at+eclipse, pascal

Version:

3.6.1

Keywords:

performance

Target Milestone:

3.7 M6

Hardware:

OS:

All

Whiteboard:

Bug Depends on:

Bug Blocks:

335610, 408208

Attachments:

Description	Flags
test case and proposed fix	pascal: iplog+
test data	none

Description Meng Xin Zhu

2011-01-24 03:02:27 EST

ProvisioningContext itself has a private method named 'getLoadedArtifactRepositories' that is sorted all the available repositories, the local repository has higher priority.

There is no public API to get all artifact repositories from ProvisioningContext, so clients use below query to query out all the artifact repositories from ProvisioningContext instance like DownloadManager. The query result is stored by a set container that never keeps the results in a certain sequence.

If some artifact keys with same id and version in both local repository and remote repository, DownloadManager probably downloads them from remote repository. It's very annoying to cost much more time and bandwidth to download them.

The code snippet of DownloadManager.start():

IQueryable<IArtifactRepository> repoQueryable = provContext.getArtifactRepositories(subMonitor.newChild(250));
IQuery<IArtifactRepository> all = new ExpressionMatchQuery<IArtifactRepository>(IArtifactRepository.class, ExpressionUtil.TRUE_EXPRESSION);
IArtifactRepository[] repositories = repoQueryable.query(all, subMonitor.newChild(250)).toArray(IArtifactRepository.class);

The fix for DownloadManager is sorting the array of query result again. Maybe it's better and common to add public interface to get sorted repositories on ProvisioningContext.

Comment 1 Helmut J. Haigermoser

2011-01-24 05:05:29 EST

Ouch, if this is true than 3.6 is severely broken and must be fixed right away,
favoring "http" over "file" repos will kill performance!

Can we get a reply to this ASAP, and bump it to highest priority if your analysis confirms the issue?

Helmut

Comment 2 Pascal Rapicault

2011-01-24 14:21:24 EST

Please provide a test case showing the problem and a patch.

Comment 3 Meng Xin Zhu

2011-01-25 04:27:57 EST

Created attachment 187497 [details]
test case and proposed fix

Let bundle p2.test require the packages related to osgi http service, and add http.port vm argument in the launch.

Let me know what you think. Thanks.

Comment 4 Meng Xin Zhu

2011-01-25 04:30:40 EST

Created attachment 187498 [details]
test data

unzip it, then put the folder under 'testData'

Comment 5 Martin Oberhuber

2011-01-25 06:14:42 EST

CQ:WIND00252176

From what I understand, this is a severe regression in Eclipse 3.6, compared to Eclipse 3.5. The regression affects products that install from both a local repo (like a shipped DVD) and an online repo (like an ESD / update pack site).

Effect of the regression is that the install performance can be severely reduced by downloading artifacts that already exist locally. Customers likely wouldn't accept such a regression, so I'm setting "critical" severity.

I've also updated the summary to show the end-user effect, previous summary was:
DownloadManager.start() doesn't sort the repositories based on the predefined priorities when trying to download the artifacts from the repositories one by one

Please consider the testcase and fix that Meng Xin has provided.

Comment 6 DJ Houghton

2011-01-27 14:25:10 EST

Released a modified patch to HEAD.
Thanks for the investigation.
Closing.

Comment 7 Martin Oberhuber

2011-01-27 15:30:12 EST

Thanks so much, DJ!

Could a backport to 3.6.2 be considered?

We are very close to releasing a product (based on 3.6.x), and this issue is a showstopper for us. For sure we'll have to patch the Platform of our RCP installer ourselves if this doesn't get addressed in mainstream.

Comment 8 DJ Houghton

2011-01-27 15:39:42 EST

Sounds good to me. I've created bug 335610 for this. 
I will attach a patch and then ask a PMC member for a +1.