Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 520905

Summary: Target Platform Resolution often fails for update sites at eclipse.org
Product: Community Reporter: Maximilian Koegel <mkoegel>
Component: ServersAssignee: Eclipse Webmaster <webmaster>
Status: RESOLVED FIXED QA Contact:
Severity: major    
Priority: P3 CC: cletavernier, david_williams, denis.roy, jfaltermeier, mkoegel, stepper, torkildr
Version: unspecified   
Target Milestone: ---   
Hardware: All   
OS: All   
Whiteboard:

Description Maximilian Koegel CLA 2017-08-11 12:02:09 EDT
We have frequently (about once a week) the problem that target platform resolutions fail repeatedly because p2 repositories hosted at eclipse.org are not reachable since eclipse.org servers time out or are not available temporarily. All known affected developers of us are based in Europe. I can rule out a problem with our internet connection since we have multiple offices at different locations  with different connections and updates sites in the target platforms hosted elsewhere work just fine.

This problem usually has these implications:
- Developer cannot resolve TP in local Eclipse environment
- Customer cannot access a bug fix release pushed to the official update site
- Eclipse.org hosted build fails 

Apart from raising awareness to this problem, I am also interested in potential workarounds other developers employ to fix this.

There is a similar issue that is far more general that was raised some time ago in bug 499399.
Comment 1 David Williams CLA 2017-08-11 13:52:57 EDT
In addition to Nick's good advice to Arch. Council list, here are some other comments: 

1. when a problem is seen, be sure to try something similar to
 wget http://download.eclipse.org/releases/oxygen/201706281000/content.xml.xz
which should tell you if the main initial request is being re-directed.
Even here in North Carolina as of right now (08/10/2017, 20:50 Eastern), that request replies with "302 Moved Temporarily" and then is being redirected to
http://mirror.math.princeton.edu/pub/eclipse//releases/oxygen/201706281000/content.xml.xz
[Actually, each time the request is sent, I have just now seen that is redirects to a different mirror.]

I've never seen that initial redirect strategy work well. I *think* it *might* be because then "mirror.math.princeton.edu" (or whatever) becomes the "fallback URL" which assumes it is as perfect as "download.eclipse.org". (Half guessing the reason why it does not work well, but 100% sure I have never seen it work well. Perhaps a p2 committer knows more if it should or what it assumes?)

2. for more p2 options to enable more debug and trace output to see which mirrors are being used during a p2 operation and which, if any, are causing problems see
https://wiki.eclipse.org/Equinox/p2/p2.mirrorsURL#How_to_test.3F

2A. There is a possibility of a "bad mirror" that throws everything off. (For example, one that has partial, but not full content -- I don't think p2's heuristics cover that case well -- and other cases such as sending a bundle named "abc.php.bundle.jar" as "php" content, instead of as binary content).
2B. I didn't see mention of which repositories were causing problems, but, in the past, if the p2.mirrorsURL was set incorrectly in a repository, it causes a lot of churning and everything then tries the default fallback URL.

3. The "Foundation" (SimRel engineer) really should clean up the old directories from .../releases/oxygen. Currently 11 directories there are taking approx. 28G of space. Only one is needed (201706281000) taking about 2.4G. Some mirrors might be running low on space. Just a wild guess. (I suspect in 99% of cases a measly 20G or so would not matter to a mirror-class server -- but, I do not know for sure).

4. I'm sure most readers know about setting -Declipse.p2.mirrors=false which I mention not so much as a long-term solution, but those having issues should be mentioned in this bug report if that setting solves the problem or if the problem is the same.

HTH, Good Luck,
Comment 2 David Williams CLA 2017-08-11 13:54:33 EDT
For future reference, Nick's advice to Arch. Council list is at the following URL: 

https://dev.eclipse.org/mhonarc/lists/eclipse.org-architecture-council/msg03482.html
Comment 3 Camille Letavernier CLA 2017-08-16 05:34:56 EDT
For target platforms, Mickaƫl's tool for Target Platform Definition [1] includes a "Set as target" context menu which is much more efficient and reliable than Eclipse's mechanism. So, whenever I have the opportunity, I use *.tpd files instead of *.target (*.target will be generated), and use the context menu to set/switch TPs.

One of the differences is that the TPD Tool uses a local cache for bundles, but that's not the only difference. I suspect the algorithm to actually resolve and set the TP is also simpler (Thus faster and almost 100% reliable), although I've not looked into it.

I don't know how much of this could be ported directly to the Eclipse Target Platform framework, but it might be worth investigating further

[1] https://github.com/mbarbero/fr.obeo.releng.targetplatform
Comment 4 Denis Roy CLA 2017-09-28 20:45:41 EDT
As bug 517294 is resolved, you will not see this anymore.