Bug 431047

Comment 1 David Williams 2014-03-24 13:27:21 EDT

I've checked the logs and there doesn't appear any error or warnings ... 

but I think I know basically what the problem is. 

And, the only "problem with the problem" is that it implies our ability to run multiple tests on a given build and platform (such as one with Java 8 and again with Java 7) is impossible ... at least until the "convert" program is improved. 

I think this because I've done some preliminary work to fix the "labeling" of the test output (bug 390986) so right at this moment, we end up with two "output directories ... one of them basically empty, just testing my "label fix", ant the other the old, traditional one. 

By coincidence of naming, in the linux and windows cases, the "new" (empty) directory comes before the "old" (full) directory ... so, my current theory is that's related. 

As one "step" to investigating, I've recompiled the "indexing tools" so it does not swallow exceptions -- part of the disconcerting part is there's now "warning" or anything thing, in log. And once that's in place will rerun the "fetch and summarize tests" jobs. 

The second step is that I will try putting "z" prefix on the new, empty directories, and see if that magically fixes things in tonight's N-build.

Comment 2 David Williams 2014-03-24 17:28:27 EDT

adding the improved logging didn't help provide any information. 
For tonight's N-build, I've "commented out" the new directories that are created, to see if that along fixes things. I have a feeling some other seemingly insignificant change is causing the problem. May have to do some "local debugging" to better understand how/what that "indexing" code is looking for.

Comment 3 David Williams 2014-03-25 13:02:18 EDT

Status: I've found the "culprit" ... the list of "platforms tested" is hard coded in "TestResultsGenerator" in the org.eclipse.build.tools project. 

Previously had: 


    // The four configurations, add new configurations to test results here +
    // update
    // testResults.php.template for changes
    private final String[]  testsConfig                  = { "linux.gtk.x86_6.0.xml", //
            "macosx.cocoa.x86_5.0.xml", //
            "win32.win32.x86_7.0.xml"                   };


For now, I've changed now to 

    // Configuration of test machines.  
    // Add or change new configurations here 
    // and update titles in testResults.php.template.
    // These are the suffixes used for JUnit's XML output files. 
    // On each invocation, all files in results directory are 
    // scanned, to see if they end with suffixes, and if so, 
    // are processed for summary row. The column order is determined by 
    // the order listed here. 
    // This suffix is determined, at test time, when the files junit files are 
    // generated, by the setting of a variable named "platform" in test.xml 
    // and associated property files. 
    private final String[]  testsConfig  = { 
            "linux.gtk.x86_64_8.0.xml", 
            "macosx.cocoa.x86_64_7.0.xml", 
            "win32.win32.x86_7.0.xml" };            

This should allow I-build test results to work. 

Proof of concept: I re-ran the Linux Tests for last night's N-build, 
and they show up correctly in the right places. 
http://download.eclipse.org/eclipse/downloads/drops4/N20140324-2000/testResults.php 

Only problem is we "lost" the Mac tests since they were generated with "old" suffix. 

Not sure what's up with Windows ... the "testConfig" value seems like it's correct and has been for a while, but for several N-builds, were generating suffixes with different values ... so might have been something else I (accidentally) messed up, trying to clean up some code. 

From looking at current I-build tests on Hudson, everything looks right, with a small exception that the overall "console log" for windows (only) is at the root of results, instead of under "results/onsolelogs" directory. At least that can be fixed after the fact. 

I've opened bug 431143 for an improved fix to this "hard coded" issue.

Comment 4 David Williams 2014-03-25 16:28:40 EDT

I see its still not quite right. For the individual console logs from each tests, I see simply 

Individual linux.gtk.x86_6.0 test logs
Individual win32.win32.x86_7.0 test logs
Individual macosx.cocoa.x86_5.0 test logs

That is, just the headings, not the individual logs, even though the ones for Linux and Mac are there ... Windows just finishing. 

I'm assuming I can fix current page "manually", but then need to fix template, to operate via variables, instead of hard coding. Well, I'll hard code for now, but made note on bug 431143 that's another place that needs a variable "platform".

Comment 5 David Williams 2014-03-25 17:05:44 EDT

I fixed logs.php (on downloads machine ... see if it gets overwritten when Windows tests complete) ... to 

listDegailedLogs("linux.gtk.x86_64_8.0");
listDegailedLogs("win32.win32.x86_7.0");
listDegailedLogs("macosx.cocoa.x86_64_7.0");



= = = = = 

And, about the overall log, for windows, going to wrong place, it was a clear "typo" I made while converting to use a variable, and using "full test platform name" ... previous to today, we used simply "win7console.txt" for it, though others had full platform info as part of the name. 

In fact, this is in "test.bat" (similar files for other platforms) and is actually one of the earliest places we need to know "platform" string. 
Its a little too early, actually, since it's before any of the main framework has started. May need to consider changes/alternatives when thinking of long term solution. Or, at least well document all the places that have to be changed "at once". Or, which/when are "manual settings" (for our production builds) and when  it is computed by framework.