Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 358340

Summary:

Serve up p2 metadata at high (or, normal) priority

Product:

Community

Reporter:

David Williams <david_williams>

Component:

Servers

Assignee:

Eclipse Webmaster <webmaster>

Status:

RESOLVED WONTFIX

QA Contact:

Severity:

normal

Priority:

CC:

david_williams, denis.roy, igor, jan.sievers, john.arthorne, mknauer, mober.at+eclipse, nicolas.bros, pascal, sbouchet, steffen.pingel, t-oberlies, thomas, webmaster

Version:

unspecified

Target Milestone:

---

Hardware:

OS:

Windows XP

Whiteboard:

Attachments:

Description	Flags
List of downloads for Sept. 22	none

Description David Williams

2011-09-21 01:59:23 EDT

+++ This bug was initially created as a clone of Bug #347403 +++

This is a follow-up on bug 347403. 

There is several issue discussed in that bug, one of which is that p2 metadata files come from "eclipse.org" (not mirrors) and that metadata is "requested" in several situations that may or may not lead to "downloads". If it does lead to downloading artifacts, then it is "ok" for them to take a while since something was found to be downloaded. But, in other cases, it is desired to make a quick decision that "nothing's changed" so ... all in all, best to serve-up the metadata at normal (or high) priority. There is (also) a similarity to presenting a "download page" from which the user picks a package to download ... the expectation is the page itself displays quickly, even if the (larger) package download is served with a lower priority. 

And, in the cloned bug, it was said "... I [webmaster] _could_ do some packet mangling to force higher priority for metadata files
out of download.eclipse.org "

It is my understanding the following 9 files (served from anywhere from eclipse.org) should be high on the QoS list (if there is such a think as QoS list for files :) 

p2.index
content.xml
content.jar
artifacts.xml
artifacts.jar
compositeContent.xml
compositeContent.jar
compositeArtifacts.xml
compositeArtifacts.jar

Comment 1 Denis Roy

2011-09-23 14:15:19 EDT

Created attachment 203926 [details]
List of downloads for Sept. 22

I've compiled the top-250 files that were downloaded from download.eclipse.org for the 24-hour period of Sept. 22.

At first glance, it appears the vast majority of files we're serving are the ones you've listed.

Because that file list represents lots of data traffic, if I place it into high priority, I fear it will have an impact on current high-priority traffic, such as CVS/SVN/Git, Bugzilla and www.eclipse.org.

Comment 2 David Williams

2011-09-23 15:23:19 EDT

(In reply to comment #1)
> Created attachment 203926 [details]
> List of downloads for Sept. 22
> 

a day in the life of ... eh? Very interesting numbers. On several levels. 

If it helps, some of the "meta data files" are relatively small in size, and would always be expected to be, since they just contain content that points to other files (essentially). Maybe they could be "high"?  

p2.index
compositeContent.xml
compositeContent.jar
compositeArtifacts.xml
compositeArtifacts.jar

These other 4, though, while sometimes small, are sometimes large, as you can see in the data ... think they could be a "medium" priority instead of low? 

content.xml
content.jar
artifacts.xml
artifacts.jar

It may take some "experimentation" to see how high we change things without overly impacting other services ... unfortunately, I don't really know how to (easily) measure p2 improvement (much less, impact on other services).

Comment 3 Denis Roy

2013-10-24 10:37:30 EDT

Since bumping up the bandwidth substantially, there's been no need to do this.