Bug 338097

Comment 1 David Williams 2012-08-11 02:07:13 EDT

This test seems to fail consistently on Windows and Linux. And yet other "proxy bugs" have been fixed on Windows and Linux.  I'm wondering ... perhaps it should also fail on the Mac, but the Mac isn't set up right? :) 

I say that since ... I wonder ... on the IBM System where these used to run, were the "system" or "native" values for proxies set (before Eclipse runs)? In other words, I'm wondering if the settings used to be "empty" and completely in control of the test, but now, on Hudson, there are "real system settings" to contend with that are "filled in" already as Eclipse starts (and, as far as I know, the "native" settings can not be changed?). 

See bug 372880 for a lot of wrestling we've done with proxies on Hudson server (and, overcoming the system settings there) in particular, 
see attachment 219031 [details]
https://bugs.eclipse.org/bugs/attachment.cgi?id=219031
for the starting value of settings, on windows, as Eclipse starts. I wonder ... did these used be be "blank" for the IBM machines? And would that make a difference? 

I've read bug 257503 and appears things have changed .... there no longer is an "Eclipse Provider"? ... but I couldn't really tell what the unit tests were trying to demonstrate. So, wanted to be sure to point you to bug 372880 and maybe seeing what the proxy values are there on Hudson might give a clue to those that understand this unit test ... is there anyone? :/

Comment 2 David Williams 2012-08-20 19:33:54 EDT

New theory. 

I noticed the mac test DOES have all the proxy variables set correctly when these tests run (that is, set correctly as they are set on Hudson and Eclipse.org infrastructure) but that Windows and Linux did not. Specifically the Mac test had these properties during the (successful) test run: 

ftp.nonProxyHosts   127.0.0.1|localhost|*.localhost|local|*.local|169.254/16|*.169.254/16|eclipse.org|*.eclipse.org|hudson.eclipse.org|*.hudson.eclipse.org|dev.eclipse.org|*.dev.eclipse.org
gopherProxySet  false
http.nonProxyHosts  127.0.0.1|localhost|*.localhost|local|*.local|169.254/16|*.169.254/16|eclipse.org|*.eclipse.org|hudson.eclipse.org|*.hudson.eclipse.org|dev.eclipse.org|*.dev.eclipse.org
http.proxyHost  proxy.eclipse.org
http.proxyPort  9898
https.proxyHost proxy.eclipse.org
https.proxyPort 9898
socksNonProxyHosts  127.0.0.1|localhost|*.localhost|local|*.local|169.254/16|*.169.254/16|eclipse.org|*.eclipse.org|hudson.eclipse.org|*.hudson.eclipse.org|dev.eclipse.org|*.dev.eclipse.org


So why does the Mac get these values, and the Windows and Linux tests do not? No idea. 

But, all platforms, via Hudson settings, show these system variables are defined with "proxy info" all courtesy of webmasters set up of the Hudson slaves. 

ANT_OPTS
JVM_OPTS
ANT_ARGS
JAVA_ARGS

For example, 
ANT_OPTS=-Dhttp.proxyHost=proxy.eclipse.org -Dhttp.proxyPort=9898 -Dhttps.proxyHost=proxy.eclipse.org -Dhttps.proxyPort=9898 -Dhttp.nonProxyHosts="localhost|build.eclipse.org|dev.eclipse.org|206.191.52.58|172.30.206.*|172.30.206.0|172.25.25.*|172.25.25.0|hudson.eclipse.org|127.0.0.1|*.eclipse.org|eclipse.org|git.eclipse.org|download.eclipse.org"  -Dhttps.nonProxyHosts="localhost|build.eclipse.org|dev.eclipse.org|206.191.52.58|172.30.206.*|172.30.206.0|172.25.25.*|172.25.25.0|hudson.eclipse.org|127.0.0.1|*.eclipse.org|eclipse.org|git.eclipse.org|download.eclipse.org" -Dftp.proxyHost=proxy.eclipse.org -Dftp.proxyPort=9898 -Dftp.nonProxyHosts="*.eclipse.org" 

Not positive they are all completely accurate, but should be close.

We do not use any of these variables in _our_ scripts, but often, if you use "ant wrapper scripts" those scripts witll use ANT_OPTS and ANT_ARGS. (I know less about the JVM and JAVA args, but suspect they are not so standard or explainable). 

In any case, I've taken the action of making use of the ANT_OPTS environment variable where we specify other "system properties" for our Ant Runner call. 

I've done it for Mac, Windows and Linux, to be consistent, so we should be able to tell if any effect in tonight's nightly build. 

http://git.eclipse.org/c/platform/eclipse.platform.releng.eclipsebuilder.git/commit/?id=237d4c252457c23d8bdb5daaadbd494b8d9194c0

And I just hope it doesn't break anything else like reintroduce the bugs from 372880  :) 

[Normally, I'd mark this fixed at this point, but still feel in the area of "trial and error", so we'll see.

Comment 3 David Williams 2012-08-21 23:45:01 EDT

My attempt didn't work, but I'm not totally deterred. 

There is a property, now, that looks like 

todelsocksNonProxyHosts=localhost|build.eclipse.org|dev.eclipse.org|206.191.52.58|172.30.206.*|172.30.206.0|172.25.25.*|172.25.25.0|hudson.eclipse.org|127.0.0.1|*.eclipse.org|eclipse.org|git.eclipse.org|download.eclipse.org

To me, this implies something is going wrong with the quoting of the whole thing ... which I always find confusing! But, I think the right answer is to use single quotes to around '${ANT_OPTS}' while I hope will cause the "'d and *'s to not be interpret/expaneded by bash. (I'm not exactly sure how to do this in Windows, but I'll this a blind try).

Comment 4 David Williams 2012-08-26 15:01:07 EDT

I never could get things working trying to "pass in" ANT_OPTS "directly" from eclipseBuilder calls to invoke tests. But instead I did implement 
bug 388056, which causes test framework to automatically honor "ANT_OPTS". 

That did not fix this bug, however. 

The windows and mac tests are still running, so before jumping to complete conclusions, will way for those. But, on linux, in the "properties", I can now clearly see defined 

ftp.nonProxyHosts	*.eclipse.org
ftp.proxyHost	proxy.eclipse.org
ftp.proxyPort	9898

http.nonProxyHosts	*.eclipse.org|172.30.206.*
http.proxyHost	proxy.eclipse.org
http.proxyPort	9898

https.nonProxyHosts	*.eclipse.org
https.proxyHost	proxy.eclipse.org
https.proxyPort	9898

But, I also see "odd" changes in that test log


!ENTRY org.eclipse.core.net 1 0 2012-08-26 12:09:43.722
!MESSAGE System property http.proxyHost has been set to proxy.eclipse.org by an external source. This value will be overwritten using the values from the preferences

!ENTRY org.eclipse.core.net 1 0 2012-08-26 12:09:43.809
!MESSAGE System property https.proxyPort is set to 9898 but should not be set.

!ENTRY org.eclipse.core.net 1 0 2012-08-26 12:09:43.813
!MESSAGE System property http.proxyHost is set to proxy.eclipse.org but should be www.eclipse.org.


To cross reference, I have also opened bug 388065 to allow framework to set "manual" network preferences.

Comment 5 David Williams 2012-09-26 00:37:43 EDT

I've looked at this a little more, and pretty sure the test is in pretty good control of its own "preferences" and during "setup" tries to remember how set, and sets to known state, and then during "teardown", tries to restore to what ever state it found to begin with. But, something isn't quite right with how it does this, I think, so it might lead to "order effects"? (the order the tests are ran in). 

I could not reproduce bug, but noticed following: 

If I run testBug257503 all by itself the 
org.eclipse.core.net.prefs file ends up being

eclipse.preferences.version=1
org.eclipse.core.net.hasMigrated=true
systemProxiesEnabled=true


If I run the whole test suite, NetTest (11 tests) the 
org.eclipse.core.net.prefs file ends up being

eclipse.preferences.version=1
nonProxiedHosts=localhost|127.0.0.1
org.eclipse.core.net.hasMigrated=true
proxiesEnabled=true
systemProxiesEnabled=true


Plus, if I start off with my own "custom" preference file, such as 
proxiesEnabled=true
eclipse.preferences.version=1
org.eclipse.core.net.hasMigrated=true
proxyData/HTTP/hasAuth=false
proxyData/HTTP/host=192.168.1.10
proxyData/HTTP/port=8888
proxyData/HTTPS/hasAuth=false
proxyData/HTTPS/host=192.168.1.10
proxyData/HTTPS/port=8888
proxyData/SOCKS/hasAuth=false
proxyData/SOCKS/host=192.168.1.10
proxyData/SOCKS/port=8888
systemProxiesEnabled=false

and then run NetTest, it still ends up with the "small" set of preferences 
as above: 
eclipse.preferences.version=1
nonProxiedHosts=localhost|127.0.0.1
org.eclipse.core.net.hasMigrated=true
proxiesEnabled=true
systemProxiesEnabled=true

So, that tells me the "remember" and "restore" functions are not working quite right ... but, I can not see what's wrong with them ... but thought I'd mention it since perhaps there's some issues with ordering or writing all preferences to disk. (Does it need a "flush" or something?) 

Hope all this data is helpful.

Comment 6 David Williams 2012-09-26 01:59:46 EDT

Created attachment 221494 [details]
patch to save and restore o.e.core.net.prefs

This patch changes the "setup" and "teardown" a little so it does a more literally job of "saving initial state" and "restoring initial state". 

I think there is only a 10% chance this change would fix this build machine failure but 1) seems a  better test -- to leave things as they are found and 2) who knows ... it might help? 

The 11 tests still pass for me. And that's on Linux, with System wide proxies enabled or not. Only one funny thing. 

If I start with systemProxiesEnabled=false

such as 
$ cat org.eclipse.core.net.prefs 
eclipse.preferences.version=1
nonProxiedHosts=127.0.0.1|localhost|*.localhost|local|*.local|169.254/16|*.169.254/16|eclipse.org|*.eclipse.org|hudson.eclipse.org|*.hudson.eclipse.org|dev.eclipse.org|*.dev.eclipse.org
org.eclipse.core.net.hasMigrated=true
proxyData/HTTP/hasAuth=false
proxyData/HTTP/host=proxy.eclipse.org
proxyData/HTTP/port=9898
proxyData/HTTPS/hasAuth=false
proxyData/HTTPS/host=proxy.eclipse.org
proxyData/HTTPS/port=9898
proxyData/SOCKS/hasAuth=false
proxyData/SOCKS/host=proxy.eclipse.org
proxyData/SOCKS/port=9898
systemProxiesEnabled=false

it ends up always setting it back to "true", such as 

eclipse.preferences.version=1
nonProxiedHosts=127.0.0.1|localhost|*.localhost|local|*.local|169.254/16|*.169.254/16|eclipse.org|*.eclipse.org|hudson.eclipse.org|*.hudson.eclipse.org|dev.eclipse.org|*.dev.eclipse.org
org.eclipse.core.net.hasMigrated=true
proxiesEnabled=true
proxyData/HTTP/hasAuth=false
proxyData/HTTP/host=proxy.eclipse.org
proxyData/HTTP/port=9898
proxyData/HTTPS/hasAuth=false
proxyData/HTTPS/host=proxy.eclipse.org
proxyData/HTTPS/port=9898
proxyData/SOCKS/hasAuth=false
proxyData/SOCKS/host=proxy.eclipse.org
proxyData/SOCKS/port=9898
systemProxiesEnabled=true

Perhaps this is is a quirk of the way I'm running the tests (from workbench, having unchecked "clear config area" and then looking under 
.metadata/.plugins/org.eclipse.pde.core/pde-junit/.settings

Or, perhaps this is a symptom of the original bug this was meant to regression test?

Comment 7 David Williams 2012-10-23 09:56:16 EDT

Created attachment 222679 [details]
patch to disable test

I think this test should be disabled until/if someone is able to look at it and fix it. I've spent several hours on it, and it doesn't appear to be anything simple. 

It doesn't do us any good as a regression test, if it fails each time.

Comment 8 Dani Megert 2012-10-23 12:05:05 EDT

(In reply to comment #7)
> Created attachment 222679 [details] [diff]
> patch to disable test
> 
> I think this test should be disabled until/if someone is able to look at it
> and fix it. I've spent several hours on it, and it doesn't appear to be
> anything simple. 
> 
> It doesn't do us any good as a regression test, if it fails each time.

I disabled it without using the patch.

Szymon, please have someone investigate this, thanks.

Comment 9 Eclipse Genie 2020-01-24 08:39:49 EST

This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. As such, we're closing this bug.

If you have further information on the current state of the bug, please add it and reopen this bug. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.