Bugzilla – Full Text Bug Listing
| Summary: | Dangerous use of GLOBAL on Hudson | ||
|---|---|---|---|
| Product: | [Modeling] MoDisco | Reporter: | Stephane Rousseau <srousseau> |
| Component: | Main | Assignee: | Nicolas Bros <nicolas.bros> |
| Status: | CLOSED FIXED | QA Contact: | |
| Severity: | normal | ||
| Priority: | P3 | CC: | fabien.giquel, gdupe, hugo.bruneliere, modisco.web-inbox, nicolas.bros |
| Version: | 0.9.0 | Flags: | gdupe:
indigo+
|
| Target Milestone: | 0.9.0 M6 | ||
| Hardware: | PC | ||
| OS: | Windows XP | ||
| Whiteboard: | |||
I do not understand what happened with nightly builds yesterday : build #346 was launched automatically (change in SVN) with "skip.long.junit.tests" to "false" -> so the java.discoverer.benchmark.test was really executed (5mn) and succeeded, whereas in older builds it was not (see #345 exec time to 0). Did someone change something on tests config for hudson ? (In reply to comment #1) > Did someone change something on tests config for hudson ? Yes, I am currently setting up the releng for EMF Facet, and I fixed a few things in the MoDisco hudson jobs as well. The rationale for the "skip.long.junit.tests" option was to be able to do a quick build when necessary. But I am not sure whether it should be enabled or disabled by default. On the one hand, continuous integration should be fast enough that developers get quick feedback after committing. For example, Martin Fowler recommends 10 minute builds [1]. But on the other hand, we would like each build to run all tests so that it is checked thoroughly. The build itself can be fast (the EMF Facet build currently takes about 2 minutes without tests). But many tests in the current MoDisco test suite are slow because they interact with the workbench (tests for query, facet, custom), refresh the workspace and wait for Jobs to finish before continuing. The CDO tests take a long time too, because they wait a full minute before doing anything (to wait for the CDO server to start, or some socket to close?) and then complete in a couple of seconds. The solution is probably to do a staged build (cf section Keep the Build Fast in [1]), with a first quick build started after each commit, and then a second one that runs all tests. [1] http://www.martinfowler.com/articles/continuousIntegration.html Yes, the idea of two build levels seems good for having quick continuous integration. So I think the level with "SKIP_LONG_TESTS_UNIT" may be set to "true" for nightly builds, and "false" for integration builds once a week. I have re-enabled SKIP_LONG_JUNIT_TESTS for nightly builds by default, and used Assume.assumeTrue to skip the test instead of just returning. The doc states that the test runner is supposed to mark the test as skipped in this case. But the test results from the last job don't reflect this. So, I have to continue investigating this problem. I now throw an exception with message "skipped", to make it clear the test did not pass. Upgrade done. |
Benchmark tests are launched on Hudson but they do nothing because a GLOBAL (SKIP_LONG_JUNIT_TESTS) is checked. So the test succeed but nothing is done. Here is the code : public class JavaDiscovererBenchmarkTest { private static final String PROJECT_NAME = "benchmark_discovery"; //$NON-NLS-1$ @Test public void test001() throws CoreException, IOException { if (Boolean.valueOf(System.getenv().get("skip.long.junit.tests"))) { return; } So be careful and may be, we have to think about a trace or a warning to be aware some tests are disabled.