Description Scott E. Schneider 2005-08-12 11:04:36 EDT

Not all use cases of test execution will experience concurrency issues if it
were allowed but we chose reliability over intermittent parallel test execution
success (given the time constraint we were under) in the TPTP 3.3 / 4.0 releases.

That being said (and not to open up another can of worms) there is a workbench
Java VM property I added in TPTP 3.3 / 4.0 that can be set that will bypass this
test execution serialization behavior and allow manual tests to run in parallel
with multiple JUnit tests, etc, and if all the tests are different there should
be no problems (to be used by the end-user of base TPTP or downstream products):

-DALLOW_CONCURRENT_TEST_EXECUTION=TRUE

In addition there is also an Eclipse jobs property I added in TPTP 3.3 / 4.0
that can be set that will allow parallel execution as well (for use by
downstream products internally):

new QualifiedName("TPTP", "ALLOW_CONCURRENT_TEST_EXECUTION_ONCE")

Assuming that concurrent test execution is turned on by one of the methods
described above here is an example of what should work reliably:

JUnit Test A is launched, then JUnit Test B is launched and then JUnit Test C is
launched
While those are being launched and executed the user can then launch Manual Test
D and Manual Test E and while those UIs come up,
A user can trigger URL test F and URL test G to launch at this point
The user can then can sit back and watch the progress view along with the
execution results UI to wait for their results while they enter results in their
manual test UIs

Here is an example of what I believe will bring out intermittent failures due to
race conditions and unknown causes of hung sessions (zombie java.exes) [assuming
once again that concurrent execution is enabled]:

JUnit Test A is launched 4 times in a row (sometimes works, sometimes doesn't
work due to hung sessions)
A downstream product (another test type has code that) tries to re-enter the
test execution harness and collect execution results using the same execution
context while another test is already executing (the execution results will be
either separated into various execution history files or merged undesirably into
the same results file)