Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 358340 - Serve up p2 metadata at high (or, normal) priority
Summary: Serve up p2 metadata at high (or, normal) priority
Status: RESOLVED WONTFIX
Alias: None
Product: Community
Classification: Eclipse Foundation
Component: Servers (show other bugs)
Version: unspecified   Edit
Hardware: PC Windows XP
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Eclipse Webmaster CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-09-21 01:59 EDT by David Williams CLA
Modified: 2013-10-24 10:37 EDT (History)
14 users (show)

See Also:


Attachments
List of downloads for Sept. 22 (18.24 KB, text/plain)
2011-09-23 14:15 EDT, Denis Roy CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description David Williams CLA 2011-09-21 01:59:23 EDT
+++ This bug was initially created as a clone of Bug #347403 +++

This is a follow-up on bug 347403. 

There is several issue discussed in that bug, one of which is that p2 metadata files come from "eclipse.org" (not mirrors) and that metadata is "requested" in several situations that may or may not lead to "downloads". If it does lead to downloading artifacts, then it is "ok" for them to take a while since something was found to be downloaded. But, in other cases, it is desired to make a quick decision that "nothing's changed" so ... all in all, best to serve-up the metadata at normal (or high) priority. There is (also) a similarity to presenting a "download page" from which the user picks a package to download ... the expectation is the page itself displays quickly, even if the (larger) package download is served with a lower priority. 

And, in the cloned bug, it was said "... I [webmaster] _could_ do some packet mangling to force higher priority for metadata files
out of download.eclipse.org "

It is my understanding the following 9 files (served from anywhere from eclipse.org) should be high on the QoS list (if there is such a think as QoS list for files :) 

p2.index
content.xml
content.jar
artifacts.xml
artifacts.jar
compositeContent.xml
compositeContent.jar
compositeArtifacts.xml
compositeArtifacts.jar
Comment 1 Denis Roy CLA 2011-09-23 14:15:19 EDT
Created attachment 203926 [details]
List of downloads for Sept. 22

I've compiled the top-250 files that were downloaded from download.eclipse.org for the 24-hour period of Sept. 22.

At first glance, it appears the vast majority of files we're serving are the ones you've listed.

Because that file list represents lots of data traffic, if I place it into high priority, I fear it will have an impact on current high-priority traffic, such as CVS/SVN/Git, Bugzilla and www.eclipse.org.
Comment 2 David Williams CLA 2011-09-23 15:23:19 EDT
(In reply to comment #1)
> Created attachment 203926 [details]
> List of downloads for Sept. 22
> 

a day in the life of ... eh? Very interesting numbers. On several levels. 

If it helps, some of the "meta data files" are relatively small in size, and would always be expected to be, since they just contain content that points to other files (essentially). Maybe they could be "high"?  

p2.index
compositeContent.xml
compositeContent.jar
compositeArtifacts.xml
compositeArtifacts.jar

These other 4, though, while sometimes small, are sometimes large, as you can see in the data ... think they could be a "medium" priority instead of low? 

content.xml
content.jar
artifacts.xml
artifacts.jar

It may take some "experimentation" to see how high we change things without overly impacting other services ... unfortunately, I don't really know how to (easily) measure p2 improvement (much less, impact on other services).
Comment 3 Denis Roy CLA 2013-10-24 10:37:30 EDT
Since bumping up the bandwidth substantially, there's been no need to do this.