Bugzilla – Full Text Bug Listing
| Summary: | Something is "mixed up" about Linux test results | ||
|---|---|---|---|
| Product: | [Eclipse Project] Platform | Reporter: | David Williams <david_williams> |
| Component: | Releng | Assignee: | Platform-Releng-Inbox <platform-releng-inbox> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | normal | ||
| Priority: | P3 | CC: | akurtakov, sravankumarl |
| Version: | 4.7 | ||
| Target Milestone: | --- | ||
| Hardware: | PC | ||
| OS: | Linux | ||
| Whiteboard: | |||
|
Description
David Williams
The "regular" Linux machine (SUSE with GTK 2.18) will no longer be usable due to bug 476644 which will move to GTK 2.24 as min requirement. Thus in bug 498358 infra added second centos (GTK 3.14 and GTK 2.24) builder which can be used to run the tests against GTK2 (via SWT_GTK3=0 env variable). That would effectively means same "host" labels, is this such a big problem? Regarding the test errors - can we get this second centos + SWT_GTK3=0 job running and publishing resources (additional job?) ? This would allow me working with infra until we get the tests running stable there (missing webkitgtk is what causes the errors I think). (In reply to Alexander Kurtakov from comment #1) > The "regular" Linux machine (SUSE with GTK 2.18) will no longer be usable > due to bug 476644 which will move to GTK 2.24 as min requirement. Thus in > bug 498358 infra added second centos (GTK 3.14 and GTK 2.24) builder which > can be used to run the tests against GTK2 (via SWT_GTK3=0 env variable). > That would effectively means same "host" labels, is this such a big problem? > Regarding the test errors - can we get this second centos + SWT_GTK3=0 job > running and publishing resources (additional job?) ? This would allow me > working with infra until we get the tests running stable there (missing > webkitgtk is what causes the errors I think). I thought they were going to update the SUSE level? Oh well. Doesn't matter to me as long as it solves your problem. The "host" labels inside the test results do not matter (as long as they are accurate :) Care must be taken, though, in picking the name of the job. That is used in lots of places to "mean" different things. For example, they could not both be named "ep47I-unit-cen64", So far, all three segments of the job name is used in one way or another to "sort" or "direct" the results so they "go to" the right place, the right columns in the table, what sort of analysis is done, (unit test summary vs. performance test analysis) etc. If "windowing system" was also desired, I'd suggest adding a fourth segment, to all the jobs, even though it would only matter for Linux (at the moment). Unfortunately, "windowing system" is already part of the "Tested Platform" value, but there it means in the "OSGi sense". Not sure if that can be contorted to be gtk2 vs gtk3 and then drop the final integer when used in OSGi context? But will drop this back down to "normal" bug until we know. It seems to me, though, that http://download.eclipse.org/eclipse/downloads/drops4/N20160725-2000/testResults.php the "labels" are not "mixed up"? Not sure if something was intentionally changed, or if the 7/24 build was an experiment of some sort? (In reply to Alexander Kurtakov from comment #1) > Regarding the test errors - can we get this second centos + SWT_GTK3=0 job > running and publishing resources (additional job?) ? This would allow me > working with infra until we get the tests running stable there (missing > webkitgtk is what causes the errors I think). An easy way is just to create your own job to do what you need (from "execute shell" build step -- or get webmasters help if something needs to be installed by 'root'. And then "re-run" the exact same job to see if "cured". If you want quicker turnaround, you will notice deep in the bowels of the script that executes the tests is a parameter that says -DtestSuite=all You should be able to change that to be -DtestSuite=swt to run only the org.eclipse.swt.tests (I say "should", just because I have not tried it for a very very long time, so not 100% sure it is "pass through" as it should be. Just 90% sure). (In reply to David Williams from comment #3) > (In reply to Alexander Kurtakov from comment #1) > > > Regarding the test errors - can we get this second centos + SWT_GTK3=0 job > > running and publishing resources (additional job?) ? This would allow me > > working with infra until we get the tests running stable there (missing > > webkitgtk is what causes the errors I think). > > An easy way is just to create your own job to do what you need (from > "execute shell" build step -- or get webmasters help if something needs to > be installed by 'root'. > > And then "re-run" the exact same job to see if "cured". > > If you want quicker turnaround, you will notice deep in the bowels of the > script that executes the tests is a parameter that says > > -DtestSuite=all > > You should be able to change that to be > > -DtestSuite=swt > > to run only the org.eclipse.swt.tests So creating a new job which is a clone of e.g. ep47N-unit-cen64 and playing with it to add env variable, reduce testsuite and etc. is what you propose? Btw, I can't make my own jobb at platform HIPP instance, any idea why? I thought that every committer should be able to create jobs. > > (I say "should", just because I have not tried it for a very very long time, > so not 100% sure it is "pass through" as it should be. Just 90% sure). (In reply to David Williams from comment #2) > (In reply to Alexander Kurtakov from comment #1) > > The "regular" Linux machine (SUSE with GTK 2.18) will no longer be usable > > due to bug 476644 which will move to GTK 2.24 as min requirement. Thus in > > bug 498358 infra added second centos (GTK 3.14 and GTK 2.24) builder which > > can be used to run the tests against GTK2 (via SWT_GTK3=0 env variable). > > That would effectively means same "host" labels, is this such a big problem? > > Regarding the test errors - can we get this second centos + SWT_GTK3=0 job > > running and publishing resources (additional job?) ? This would allow me > > working with infra until we get the tests running stable there (missing > > webkitgtk is what causes the errors I think). > > I thought they were going to update the SUSE level? Oh well. Doesn't matter > to me as long as it solves your problem. It would have to be SLES 11 to SLES 12 which might happen in distant future thus we looked for quicker way like using already existing machine/builder. > > The "host" labels inside the test results do not matter (as long as they are > accurate :) > > Care must be taken, though, in picking the name of the job. That is used in > lots of places to "mean" different things. For example, they could not both > be named "ep47I-unit-cen64", So far, all three segments of the job name is > used in one way or another to "sort" or "direct" the results so they "go to" > the right place, the right columns in the table, what sort of analysis is > done, (unit test summary vs. performance test analysis) etc. smth. like ep47I-unit-gtk2 should do the trick I think. > > If "windowing system" was also desired, I'd suggest adding a fourth segment, > to all the jobs, even though it would only matter for Linux (at the moment). > > Unfortunately, "windowing system" is already part of the "Tested Platform" > value, but there it means in the "OSGi sense". Not sure if that can be > contorted to be gtk2 vs gtk3 and then drop the final integer when used in > OSGi context? But will drop this back down to "normal" bug until we know. > > > > It seems to me, though, that > http://download.eclipse.org/eclipse/downloads/drops4/N20160725-2000/ > testResults.php > > the "labels" are not "mixed up"? Not sure if something was intentionally > changed, or if the 7/24 build was an experiment of some sort? I think Sravan flipped the builder used. (In reply to Alexander Kurtakov from comment #5) > > smth. like ep47I-unit-gtk2 should do the trick I think. > Sravan may know better than I, but my memory is that "lin64" is used in a few places. Perhaps he changed them all? (In reply to Alexander Kurtakov from comment #4) > So creating a new job which is a clone of e.g. ep47N-unit-cen64 and playing > with it to add env variable, reduce testsuite and etc. is what you propose? > I think you can just use the same job (especially if "N-build), making temporary changes. And when happy with that, change it back to "standard" by clicking the little green-box-arrow at the top of "build steps". (That causes it to "inherit" the build steps from ep-unit-lin64 which if set-up correctly has the standard script that all build-tests use. (I have not looked to see if that is true on the centOS machine ... it is on the shared instance, though). > Btw, I can't make my own jobb at platform HIPP instance, any idea why? I > thought that every committer should be able to create jobs. No. Well, for "Platform" maybe, but not "Releng" ... that would give too many permissions to too many people. (For example, releng group has special permissions even for equinox, etc., so ... it is more restrictive). If you need a job created or permissions on one job, just ask and we can add you to that one job until your work is done). (In reply to David Williams from comment #7) > (In reply to Alexander Kurtakov from comment #4) > > > So creating a new job which is a clone of e.g. ep47N-unit-cen64 and playing > > with it to add env variable, reduce testsuite and etc. is what you propose? > > > > I think you can just use the same job (especially if "N-build), making > temporary changes. And when happy with that, change it back to "standard" by > clicking the little green-box-arrow at the top of "build steps". (That > causes it to "inherit" the build steps from ep-unit-lin64 which if set-up > correctly has the standard script that all build-tests use. (I have not > looked to see if that is true on the centOS machine ... it is on the shared > instance, though). > > > > Btw, I can't make my own jobb at platform HIPP instance, any idea why? I > > thought that every committer should be able to create jobs. > > No. Well, for "Platform" maybe, but not "Releng" ... that would give too > many permissions to too many people. (For example, releng group has special > permissions even for equinox, etc., so ... it is more restrictive). If you > need a job created or permissions on one job, just ask and we can add you to > that one job until your work is done). So can I get my own copy of the job running the tests on the centos machine with which I can play freely? (In reply to David Williams from comment #0) > Looking at the details of Linux tests, from both the SUSE machine on shared > instance and the CentOS results they BOTH say they are from the "centos" > machine. > > This can be seen by looking at the HTML results in the summary tables such > as from the 7/24 build: > > http://download.eclipse.org/eclipse/downloads/drops4/N20160724-2000/ > > For example, > > what should be the SUSE machine says there were 111 errors in SWT: > > http://download.eclipse.org/eclipse/downloads/drops4/N20160724-2000/ > testresults/html/org.eclipse.swt.tests_ep47N-unit-lin64_linux.gtk.x86_64_8.0. > html > > But, notice that the "host" field (on the right side, os package tables, > says "centos". > > And what should be the CentOS machine says SWT had 0 errors, and its detail > summary at > http://download.eclipse.org/eclipse/downloads/drops4/N20160724-2000/ > testresults/html/org.eclipse.swt.tests_ep47N-unit-cen64_linux.gtk.x86_64_8.0. > html > > does indeed say "centos". > > = = = = = = > > I'm not sure this is a "blocker" but have marked that way since we are at > least not capturing the "regular" Linux machine test results. That is, would > at least be "critical" for lost data, And, to me, we could not deliver a > proper milestone without the right test restuls (or, the right labels). > > = = = = = = > > Probably related to bug 498358? Sorry David, I ran the GTK2 testcases on Centos for 24th build and cnaged back to SLES (In reply to Sravan Kumar Lakkimsetti from comment #9) > Sorry David, > > I ran the GTK2 testcases on Centos for 24th build and cnaged back to SLES Ok, I'll close this as "fixed" then. When possible, in the future, if you know the test results will be "off" of what they normally are, it is best to announce to platform-releng-dev so that everyone knows what is going on. = = = = = = = = = = = = = (In reply to Alexander Kurtakov from comment #8) > > So can I get my own copy of the job running the tests on the centos machine > with which I can play freely? Well, we'd prefer "experiment carefully" rather than "play freely" :) But I have created a copy of ep47N-unit-cen64 named alex_ep47N-unit-cen64 and listed you with "configuration" permission so you can modify it however you would like. I did remove the "trigger remotely" and removed "subsequent actions" and removed email notifications being sent to Sravan (poor guy gets enough as it is :) but otherwise when ran "manually" if the same parameters are input that used in a normal nightly build it should run the normal nightly tests based on that build that you entered as a parameter (you also need to have the "version" and Hash etc., match the 'nightly'). I even created another view just for it (and perhaps similar jobs, in future) named "Other"). So a good URL to access it with is https://hudson.eclipse.org/platform/view/Other/job/alex_ep47N-unit-cent64/ BTW, as your ID, I used what was here in bugzilla. If your committer ID is something different, let me know and I will change it so you can log in. (In reply to David Williams from comment #10) > (In reply to Sravan Kumar Lakkimsetti from comment #9) > > > Sorry David, > > > > I ran the GTK2 testcases on Centos for 24th build and cnaged back to SLES > > Ok, I'll close this as "fixed" then. > > When possible, in the future, if you know the test results will be "off" of > what they normally are, it is best to announce to platform-releng-dev so > that everyone knows what is going on. > > = = = = = = = = = = = = = > > (In reply to Alexander Kurtakov from comment #8) > > > > So can I get my own copy of the job running the tests on the centos machine > > with which I can play freely? > > Well, we'd prefer "experiment carefully" rather than "play freely" :) Sure :) > > But I have created a copy of ep47N-unit-cen64 named alex_ep47N-unit-cen64 > and listed you with "configuration" permission so you can modify it however > you would like. > > I did remove the "trigger remotely" and removed "subsequent actions" and > removed email notifications being sent to Sravan (poor guy gets enough as it > is :) but otherwise when ran "manually" if the same parameters are input > that used in a normal nightly build it should run the normal nightly tests > based on that build that you entered as a parameter (you also need to have > the "version" and Hash etc., match the 'nightly'). > > I even created another view just for it (and perhaps similar jobs, in > future) named "Other"). > > So a good URL to access it with is > https://hudson.eclipse.org/platform/view/Other/job/alex_ep47N-unit-cent64/ > > BTW, as your ID, I used what was here in bugzilla. If your committer ID is > something different, let me know and I will change it so you can log in. akurtako@redhat.com please, vpns and etc. force me to use different accounts. (In reply to Alexander Kurtakov from comment #11) > akurtako@redhat.com please Done |