Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 347103

Summary: 502 response code from "download" from "build" machine
Product: Community Reporter: David Williams <david_williams>
Component: ServersAssignee: Eclipse Webmaster <webmaster>
Status: RESOLVED WONTFIX QA Contact:
Severity: normal    
Priority: P3    
Version: unspecified   
Target Milestone: ---   
Hardware: PC   
OS: Linux   
Whiteboard:

Description David Williams CLA 2011-05-25 00:19:04 EDT
I got this error recently (just now) from the Indigo aggregation build. 

Seems kind of unique ... bad gateway? ... so, thought I should report. 

If it was a one time glitch, no big deal ... but, obviously, if happens all the time for everyone it'd be "blocking". 

Where there any recent tweaks to "redirect rules"? I ask since this bundle does contain ".dtd" in its name ... so ... was just wondering if dtd requests were redirected elsewhere? 

= = = = = = = 


The following error occured when building Indigo:

org.eclipse.core.runtime.CoreException: Unable to mirror artifact osgi.bundle,org.eclipse.wst.dtd.core,1.1.500.v201103300335 from repository http://download.eclipse.org/webtools/downloads/drops/R3.3.0/S-3.3.0RC2-20110520234832/repository: HttpClient connection error response code 502.

Check the log file for more information: https://hudson.eclipse.org/hudson/view/Repository%20Aggregation/job/indigo.runAggregator/460/console
Comment 1 David Williams CLA 2011-05-25 00:22:32 EDT
If it helps ... the full URL ... for those that do not know how to translate p2 error message :) would be 

http://download.eclipse.org/webtools/downloads/drops/R3.3.0/S-3.3.0RC2-20110520234832/repository/plugins/org.eclipse.wst.dtd.core_1.1.500.v201103300335.jar
Comment 2 David Williams CLA 2011-05-25 00:33:24 EDT
Just to document it, there were other jars, with ".dtd" in the name, that were retrieved ok, such as 

    [exec] - mirroring artifact osgi.bundle,org.eclipse.wst.dtd.ui,1.0.600.v201103171359
     [exec]     doing copy of optimized artifact

So ... don't think related to having ".dtd" in the name. 

Probably just a glitch ... during this build run, has not happened to any other jars.
Comment 3 David Williams CLA 2011-05-25 03:32:16 EDT
I canceled the one build, that had the one error, re-ran that build, and then there were no errors ... I guess a glitch. I'll leave open, and let you close as you see fit, just in case you want to investigate, even though the error is no longer occurring. 

Thanks,
Comment 4 Denis Roy CLA 2011-05-25 09:15:41 EDT
I looked up the error on the proxy server.  Looks like this happened at 00:01 (just after midnight).  Our servers (including the download server) rotate their Apache logs at midnight, and afterwards Apache needs to be gracefully restarted.  This is likely what caused the glitch, as the proxy server gave up after a few seconds.

I wonder why Hudson is going through the proxy for download.eclipse.org when it can get it directly...
Comment 5 David Williams CLA 2011-06-01 00:18:53 EDT
This happened again, just now ... 

https://hudson.eclipse.org/hudson/view/Repository%20Aggregation/job/indigo.runAggregator/492/console

> I wonder why Hudson is going through the proxy for download.eclipse.org when it
> can get it directly...

Was this a question for me? A rhetorical question? :) 

If you are wondering why we use "http://" instead of file system ... well, that's so "the build" can be ran from other places. 

But, hopefully you just meant there's some config item you can tweak :) 

Maybe it'll speed up our aggregation builds!
Comment 6 Denis Roy CLA 2011-06-01 10:15:17 EDT
(In reply to comment #5)
> This happened again, just now ... 

And the reason is the same as my previous comment.

> > I wonder why Hudson is going through the proxy for download.eclipse.org when it
> > can get it directly...
> 
> Was this a question for me? A rhetorical question? :) 

It was a rhetoric... We've told Hudson to use the proxy, except for *.eclipse.org hosts... But at this point, I'm done wondering why Hudson does not do what I tell it to do.


> But, hopefully you just meant there's some config item you can tweak :) 

It seems that the more I config and tweak Hudson, the less it listens.  It is exhausting, really.


> If you are wondering why we use "http://" instead of file system ... well,
> that's so "the build" can be ran from other places. 

So we have some choices:
- switch to using the file system, which will be infinitely more robust
- put up with the occasional 502
- change the time at which you run your build so that it is not running at 00:15 local time, when the logs are done rotating.

Sorry David, there's not much more I can do.
Comment 7 David Williams CLA 2011-06-03 00:41:55 EDT
> 
> ... We've told Hudson to use the proxy, except for
> *.eclipse.org hosts... But at this point, I'm done wondering why Hudson does
> not do what I tell it to do.
> 

How do you tell Hudson not to use the proxy? We invoke eclipse (and indirectly a Java VM) independently of Hudson ... so, I'm thinking of trying to add "noproxyhosts" as a java vm parameter ... unless you think it might hurt something. Something like: 

-Dhttp.noProxyHosts="localhost|download.eclipse.org"

(though, I guess if I never tell it to use proxies, there's no need to tell it not to use it for some ... ?) 

Oh well, just wondering since it happened again. Guess we'll live with it. 

Thanks,
Comment 8 Denis Roy CLA 2011-06-03 08:21:37 EDT
(In reply to comment #7)
> We invoke eclipse (and indirectly
> a Java VM) independently of Hudson

Heh.  A Java VM running a Java app inkoves another Java VM to run a Java app.  640 MB of RAM should be enough for everyone  :/


> ... so, I'm thinking of trying to add
> "noproxyhosts" as a java vm parameter ... unless you think it might hurt
> something. Something like: 
> 
> -Dhttp.noProxyHosts="localhost|download.eclipse.org"

or "localhost|*.eclipse.org"


But then again -- wouldn't that setting make your build "not work from other places" ?  I mean, if you have to configure an eclipse.org-specific setting, why not use the filesystem and avoid the problem instead of implementing a workaround?  Isn't that what variables and conditionals are for?
Comment 9 David Williams CLA 2011-06-03 08:41:14 EDT
> 
> But then again -- wouldn't that setting make your build "not work from other
> places" ?  I mean, if you have to configure an eclipse.org-specific setting,
> why not use the filesystem and avoid the problem instead of implementing a
> workaround?  Isn't that what variables and conditionals are for?

Well, if I hard coded it it would. :) ... but, we always have some variable special just for the production build machine. 

And ... I have been looking at "using the file system". If you like the detail ... the data files, from all the project teams, would still use repository=http://download.eclipse.org/... but, that's a pretty standard, unique pattern. So, when running on the build server, I should be able to do a 'global replace" to repository=file:///home/data/httpd/download.eclipse.org/...

Exciting stuff!