Bugzilla – Full Text Bug Listing
| Summary: | One new Mac Mini to run tests on eclipse.org | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Community | Reporter: | Kim Moir <kim.moir> | ||||||
| Component: | FoE Disbursements | Assignee: | Eclipse FOE Disbursements <foe-disbursements-inbox> | ||||||
| Status: | VERIFIED FIXED | QA Contact: | |||||||
| Severity: | normal | ||||||||
| Priority: | P3 | CC: | bokowski, caniszczyk, david_williams, dj.houghton, Ed.Merks, john.arthorne, wayne.beaton, webmaster | ||||||
| Version: | unspecified | ||||||||
| Target Milestone: | --- | ||||||||
| Hardware: | PC | ||||||||
| OS: | Windows XP | ||||||||
| Whiteboard: | |||||||||
| Bug Depends on: | |||||||||
| Bug Blocks: | 325997, 354774 | ||||||||
| Attachments: |
|
||||||||
|
Description
Kim Moir
Created attachment 191930 [details]
Helios SR2 download numbers
Created attachment 191931 [details]
Mac download trends from Dec 2010 Eclipse Members meeting
I think this is a good idea. Would one mac mini be enough for now, or would two be better? How many projects at eclipse.org explicitly want to build on mac? The Mac Mini isn't being used to build, it's being used to test. One mac mini would be good to start with, but two would be optimal to run our JUnit tests in parallel. Currently, the 64,000 JUnit tests take 6+ hours on a single Mac. I'm not sure if other projects run JUnit tests on Mac, I notice the EPP project produces Mac zips, not sure if they run JUnit tests. Every committer, contributor or user who cracks open a Mac every morning and starts Eclipse would benefit :-) Everyone other project uses the bundles that the Eclipse project produces! (In reply to comment #4) > The Mac Mini isn't being used to build, it's being used to test. Sorry, my misunderstanding. > would be good to start with, but two would be optimal to run our JUnit tests in > parallel. Currently, the 64,000 JUnit tests take 6+ hours on a single Mac. I'm happy with bumping up the proposal for two. A bit of redundancy is always good and these machines aren't super expensive. I'll talk to Wayne and the other committer reps this week so we can finalize on this. I updated the bug subject and the new disbursement amount should be $1498 Since the intent is to host these at eclipse.org, we need to have Webmaster's buy-in. How do you feel about taking on the maintenance responsibility for two Macs, Denis? Do we have room in the rack? FWIW, we're investigating upgrading the OS on our existing Mac to support Java 6. Does the existing Mac have an Intel CPU, or a PowerPC? With an Intel processor, you should be able to upgrade it to Mac OS X 10.6. It has an Intel Core Duo. -M. > you should be able to upgrade it to Mac OS X 10.6. Yes, that's what we're doing in bug 340602. (In reply to comment #8) > FWIW, we're investigating upgrading the OS on our existing Mac to support Java > 6. Assuming that the upgrade is successful, will additional hardware be required? I wouldn't mind having an extra mac just in terms of redundancy and speedier tests. So if that's the case, we can downgrade this request to one mac. Any updates on this? I finished installing OS X 10.6. Just need to verify that Java 6 is happy. -M. (In reply to comment #15) > I finished installing OS X 10.6. Just need to verify that Java 6 is happy. Any updates on this? I would like to close on this request this week, and figure out whether to get one or two mac minis. So far I haven't heard Kim cursing, so I think we're ok on the java front. -M. Are you OK with getting one extra mac mini then Kim? The Java issue is fixed. However, I'm still having problems running about 4000 of our tests on that machine. They are timing out for an unknown reason. Very frustrating :-( I've been given the go-ahead by IBM to donate our existing and old iMac to the foundation after 3.7 ships. However, it takes a lot of rack space compared to a Mac mini. Not sure if this is an issue. Kim, I'll leave it up for you to decide whether one or two is more appropriate now. I'm fine with ordering one right now to take care of the immediate need. If it's still an issue, you can file a new disbursement request for a new one. Does that sound fair? Chris, one mac mini sounds fair. Thanks. (In reply to comment #21) > Chris, one mac mini sounds fair. Thanks. Ok, sweet. Can you contact emo@eclipse.org to start the order? Thanks Kim! If you think you need another one in the future, please file another request. Especially if we can help cut test time down in the future. > I've been given the go-ahead by IBM to donate our existing and old iMac to the
> foundation after 3.7 ships. However, it takes a lot of rack space compared to
> a Mac mini. Not sure if this is an issue.
Since we're OK with purchasing a new Mac Mini, please don't donate the very large iMac. The two Mac Minis we'll have will take up 2U of space at most, so that is more efficient for us than a big hulking machine which will take up 5U.
> However, I'm still having problems running about 4000
> of our tests on that machine. They are timing out for an unknown reason.
Is there a bug open for this? If not, feel free to open one up ... and tell us what it's trying to do so that we can pinpoint the reason of the timeout.
Thanks for the mac mini. It will be well loved :-) Regarding the timeout, it's described here https://bugs.eclipse.org/bugs/show_bug.cgi?id=295393#c17 Not sure if it's a network issue an Eclipse issue. Is there a way to look at the proxy log on that current Mac and see how traffic is being rerouted through the proxy? We have a number of tests that try to connect to different ports on localhost or bogus ports on different servers and it's difficult to troubleshoot them without knowing how the proxy is handling them. > We have a number of tests that try to connect to different ports on > localhost or bogus ports on different servers On the proxy server, the only entry that looks "off" is this one, since the IP address is the IP address of the Mac host: GET http://172.xx.xxx.xxx:60832/foo/info To me that is not a request to localhost -- but I've added the local 172 subnet to the exclusion list on Eclipse install that was opened. I think we should be discussing this issue in a separate bug, however. So is the plan to order a new mac mini to run tests in parallel? Just wondering because I haven't heard any news about one being order. Yes, I know I haven't been working on getting the remaining tests working on Hudson lately, have been busy with 3.7 release and git migration planning. > So is the plan to order a new mac mini to run tests in parallel?
There has been some talk at the board about which Eclipse.org services could perhaps be externalized. It would appear to me that builds and tests are likely candidates, so before ordering new hardware I'd like to see what our options are.
I may be wrong, but hosting tests (and test hardware) likely does not need to happen under the watchful eye of the Eclipse WebMasters.
Is there a timeframe for this decision? One of the advantages of running builds and tests at the foundation is that the eclipse.org filesystem is local for checking out code and copying builds to download.eclipse.org. I've tested some cloud based services in the past and the time to transfer files back and forth to eclipse.org were prohibitive, but perhaps some persistent storage at the hosting company could alleviate this bottleneck. Can someone clarify where this disbursement ended up. The bug is marked as fixed but the last few comments suggest nothing ever happened. Nothing ever happened. The mac machine currently being used is an old one from Wayne's basement. Reopening then. I think the request is still valid. Our tests now run on eclipse.org and take nearly 24 hours to run on Mac. We must also have some timeout problems, but even the tests that do run to completion are much slower than the Linux slaves. Yep, this is on my list of things to buy. Ordered .. thanks for the reminder.
Mac mini
Ships: 1 - 3 business days Receive it 10 Jul - 13 Jul by Standard Shipping
Part number: Z0M9
Configuration
2.5GHz Dual-Core Intel Core i5
8GB 1333MHz DDR3 SDRAM - 2x4GB
500GB Serial ATA Drive
User's Guide (English)
Accessory Kit
Recycling Fee $3.50
Mini DisplayPort to VGA Adapter $34.00
Ok, I've installed the new OSX 10.7 Mac Mini in the rack and have hooked it into Hudson as 'mac-tests2'. -M. I noticed there was only one executor defined for this machine. Which is normally only true for "performance test machines". I think our original intent was to use this machine for regular unit tests ... and we just wanted one a little more beefy? Or is this intended for performance tests? (We will need one for that eventually). Also, ... first build! https://hudson.eclipse.org/hudson/view/Eclipse%20and%20Equinox/job/ep4-unit-mac64/1/console Failed right away missing "pkill" ... which is nothing from us, just machine or hudson setup. Thanks!
>
> Failed right away missing "pkill" ... which is nothing from us, just machine
> or hudson setup.
>
I've seen this message again once "pkill command not found", but I just tried to run the Hudson job again and it found it that time, apparently.
But, now a larger problem [tell me if you'd prefer in a separate bug, but I think has to do with details of provisioning a mac to work with Hudson ... mostly guessing].
The problem now is some of our unit tests suites run flawlessly, but some of them fail completely. I think the difference is the "flawless" ones are all non-UI tests, and the "fail completely" ones are ones that require a "UI".
One error message that recurs often, in the logs, is
_RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL.
I searched a little on the web and found references such as
_RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL.
I should mention, at beginning of log, is does appear to be starting Xvnc and getting a DISPLAY:
Starting xvnc
[ep4-unit-mac64] $ Xvnc :11 -geometry 1024x768 -depth 24 -ac
But ... sounds like there's something else going on with Mac's own "WindowServer"?
I am just starting to investigate and in theory I could be doing something wrong :) ... but, thought I'd post what I know, in case this sounds familiar and the answer is obvious to others.
> > I searched a little on the web and found references such as > _RegisterApplication(), FAILED TO establish the default connection to the > WindowServer, _CGSDefaultConnection() is NULL. > I meant to paste http://superuser.com/questions/425125/avoid-x11-forwarding-from-mac-to-linux (But, not sure its all that helpful, now that I read it closely. But, there's more out there :). I think part of the issue may have been that the VNC server wasn't actually starting(bad path in the symlink). I've fixed that and I'll start a build here in a minute. Is there a way to break out the ui tests so we can just run those? -M. (In reply to comment #39) > I think part of the issue may have been that the VNC server wasn't actually > starting(bad path in the symlink). I've fixed that and I'll start a build > here in a minute. > > Is there a way to break out the ui tests so we can just run those? > > -M. We could pick one UI test. I've changed config to say "-Dargs=antui" which should run just the "ant ui" tests. Its pretty quick. If that runs, I'd think they all would. It looked like you'd already started one, that had the same msg in log, though ... so if need to kill that one and restart, it should run just that one UI test. Ok I think this issue is very like the one on the Windows slave, namely that the Hudson user has to be logged in for the UI tests to run. I ran 2 builds in that state after Davids change and they both passed. As soon as I logged our and re-ran the test it fell over. I've logged the Hudson user in so that will hopefully resolve the issue with the remaining UI tests. -M. (In reply to comment #41) > Ok I think this issue is very like the one on the Windows slave, namely that > the Hudson user has to be logged in for the UI tests to run. I ran 2 builds > in that state after Davids change and they both passed. As soon as I logged > our and re-ran the test it fell over. > > I've logged the Hudson user in so that will hopefully resolve the issue with > the remaining UI tests. > > -M. very cool. I'd just gotten back to looking at this, saw the failed build, and thought "darn, that didn't fix it" ... glad to hear that was your "sanity check" run. and it really is fixed. I'll run the complete suite this evening (found a little problem, on our end for "macosx x86_64" combination so checking that now. I would not know what to say ... but ... if you do, and enhancement request to "hudson project" might be in order ... I don't know what the solution would be, but seems kind of a pain to have to have a "user logged in" for the Xvnc UI to work. Thanks for your help! We'll use this for our "production runs" soon, if full suite runs ok. Just wanted to confirm this machine is running well for us now. Our Eclipse Project unit test suite now takes only 5 or 6 hours to complete on this box ... the fastest we have! (Well, it is missing an hours worth of tests for reasons unrelated to this box (more related to p2 not handling proxies well)). But it is definitely running well. Thanks! |