Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 484429 - Improve mail notification of tests: included table of failures
Summary: Improve mail notification of tests: included table of failures
Status: CLOSED WONTFIX
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Releng (show other bugs)
Version: 4.6   Edit
Hardware: PC Linux
: P3 enhancement (vote)
Target Milestone: ---   Edit
Assignee: Platform-Releng-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on: 401312 485085
Blocks:
  Show dependency tree
 
Reported: 2015-12-15 15:08 EST by David Williams CLA
Modified: 2021-12-17 04:28 EST (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description David Williams CLA 2015-12-15 15:08:37 EST
Current mail gives "total failures" if any, but an improvement would be to give the detail about which components failed, so readers know more specifically if they need to look at the details.

One good way to implement this would be to use XSL to "prune down" the standard test results summary, to just include rows that have errors. Not only is this then consistent with the full table, but then, also, when "the second test finished" it would include a summary of the first test and the second test so that already someone could easily see if there were similar errors "across platforms".
Comment 1 David Williams CLA 2015-12-20 21:04:00 EST
As this is a "continuation" of bug 401312, but sure to see bug 401312 comment 0 for a URL of "what they used to look like". Given by this URL: 

http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg18533.html

While that is one way to "show more detail", I personally do not like it much, and would prefer the "mini table" method since I think they are redundant, and better to be consistent on presentation. 

= = = = = = = =

Will also use this bug to track if there should be "only one" message, sent after all tests or done, or when last one times out (current time-out time is 15 hours!) This will be more error prone than sending each "as they complete" and more work since so much can happen between them (so that they do not all complete) or fail to fetch results and the way things are now, we do not get notified of those types of problems. 

And, who knows, maybe some advantage to doing both ... final one to dev list, individual ones to those who "sign up" for them. 

As yet another option, most systems that do this sort of notification of failure, send out the notification of failure immediately, not to dev list, but to the committers responsible for the component. That may be another option to consider if someone is going to do new work. It would require a way to "signup" for notification, presumably in the test.xml or similar and adding a "notification logger" to the current ant loggers. We *might* be able to take advantage of Hudson and extend the "extended mail notification" plugin to do this? 
http://wiki.hudson-ci.org/display/HUDSON/Email-ext+plugin

Others have mentioned (on stackOverFlow, and similar) the "right way" to do this is to separate unit tests and integration tests and treat them differently. As well as break up a long-running "single test run" into multiple, smaller test runs. (But doubt that would ever get buy in?)
Comment 2 David Williams CLA 2016-08-11 15:24:47 EDT
Doing a mass "reset to default assignee" of 52 bugs to help make clear it will (very likely) not be me working on things I had previously planned to work on. I hope this will help prevent the bugs from "getting lost" in other people's queries. Feel free to "take" a bug if appropriate.
Comment 3 Alexander Kurtakov CLA 2021-12-17 04:28:28 EST
There is too much content in tests results to it's better to point to the web page direclty.