|
Lines 1-10
Link Here
|
| 1 |
<?xml version="1.0" encoding="ISO-8859-1"?> |
1 |
<?xml version="1.0" encoding="ISO-8859-1"?> |
| 2 |
|
2 |
|
| 3 |
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"> |
3 |
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"> |
| 4 |
<html> |
4 |
<html lang="en-us" xml:lang="en-us"> |
| 5 |
<!-- |
5 |
<!-- |
| 6 |
/********************************************************************** |
6 |
/********************************************************************** |
| 7 |
* Copyright (c) 2005, 2006 IBM Corporation. |
7 |
* Copyright (c) 2005, 2007 IBM Corporation. |
| 8 |
* All rights reserved. This content |
8 |
* All rights reserved. This content |
| 9 |
* is made available under the terms of the Eclipse Public License v1.0 |
9 |
* is made available under the terms of the Eclipse Public License v1.0 |
| 10 |
* which accompanies this distribution, and is available at |
10 |
* which accompanies this distribution, and is available at |
|
Lines 35-44
Link Here
|
| 35 |
do the following:</p> |
35 |
do the following:</p> |
| 36 |
|
36 |
|
| 37 |
<ul> |
37 |
<ul> |
| 38 |
<li> <a href="#Running_tests_from_ant_scripts">Running tests from ant scripts</a> </li> |
38 |
<li> <a href="#Running_tests_from_ant_scripts">Running tests from Ant scripts</a> </li> |
| 39 |
<li> <a href="#Running_multiple_tests_and_interrogating_their_results_from_ant_scripts">Running multiple tests and interrogating their results from ant scripts</a></li> |
39 |
<li> <a href="#Running_multiple_tests_and_interrogating_their_results_from_ant_scripts">Running multiple tests and interrogating their results from Ant scripts</a></li> |
| 40 |
<li> <a href="#Generating_test_reports_from_ant_scripts">Generating test reports from ant scripts</a></li> |
40 |
<li> <a href="#Generating_test_reports_from_ant_scripts">Generating test reports from Ant scripts</a></li> |
| 41 |
<li> <a href="#Generating_test_reports_from_ant_scripts2">Customizing test reports from ant scripts</a></li> |
41 |
<li> <a href="#Generating_test_reports_from_ant_scripts2">Customizing test reports from Ant scripts</a></li> |
| 42 |
<li> <a href="#Running_tests_from_the_command_line_and_shell_scripts">Running tests from the command line and shell scripts</a></li> |
42 |
<li> <a href="#Running_tests_from_the_command_line_and_shell_scripts">Running tests from the command line and shell scripts</a></li> |
| 43 |
<li> <a href="#Running_tests_from_Java_applications_and_plug-ins">Running tests from Java applications and plug-ins</a></li> |
43 |
<li> <a href="#Running_tests_from_Java_applications_and_plug-ins">Running tests from Java applications and plug-ins</a></li> |
| 44 |
</ul> |
44 |
</ul> |
|
Lines 49-55
Link Here
|
| 49 |
service wraps existing test execution harness functionality |
49 |
service wraps existing test execution harness functionality |
| 50 |
and obsoletes the current standalone test execution |
50 |
and obsoletes the current standalone test execution |
| 51 |
harness methods. Using the test execution service, tests can |
51 |
harness methods. Using the test execution service, tests can |
| 52 |
be launched using a headless Eclipse from ant scripts, shell scripts |
52 |
be launched using a headless Eclipse from Ant scripts, shell scripts |
| 53 |
and arbitrary programs. Multiple tests can be run at the same |
53 |
and arbitrary programs. Multiple tests can be run at the same |
| 54 |
time given enough memory on the executing machine (each service |
54 |
time given enough memory on the executing machine (each service |
| 55 |
execution currently requires a separate Eclipse host instance). Even if |
55 |
execution currently requires a separate Eclipse host instance). Even if |
|
Lines 59-65
Link Here
|
| 59 |
<h3>Service properties</h3> |
59 |
<h3>Service properties</h3> |
| 60 |
<p>Refer to the reference material on the supported test |
60 |
<p>Refer to the reference material on the supported test |
| 61 |
execution service properties -- the same properties are valid across client |
61 |
execution service properties -- the same properties are valid across client |
| 62 |
automation adapters. For ant scripts, a set of simple ant tasks are |
62 |
automation adapters. For Ant scripts, a set of simple Ant tasks are |
| 63 |
provided to set service properties and execute the service. For |
63 |
provided to set service properties and execute the service. For |
| 64 |
shell scripts, the properties are specified in the expected command line |
64 |
shell scripts, the properties are specified in the expected command line |
| 65 |
paradigm (using dashes for properties followed by space and then a value |
65 |
paradigm (using dashes for properties followed by space and then a value |
|
Lines 82-88
Link Here
|
| 82 |
<ol type=1> |
82 |
<ol type=1> |
| 83 |
<li><code>tptp-automation-client.jar</code> (the |
83 |
<li><code>tptp-automation-client.jar</code> (the |
| 84 |
lightweight bit of code needed to establish the client side of the service |
84 |
lightweight bit of code needed to establish the client side of the service |
| 85 |
bus, launch Eclipse if needed and execute services headlessly, this jar will |
85 |
bus, launch Eclipse if needed and execute headless services, this jar will |
| 86 |
be referenced when running tests from the command line and shell scripts as |
86 |
be referenced when running tests from the command line and shell scripts as |
| 87 |
well as when running tests from arbitrary Java code).</li> |
87 |
well as when running tests from arbitrary Java code).</li> |
| 88 |
<li><code>tptp-automation-server.jar</code> (the TPTP |
88 |
<li><code>tptp-automation-server.jar</code> (the TPTP |
|
Lines 91-119
Link Here
|
| 91 |
used as the broker and service locator)</li> |
91 |
used as the broker and service locator)</li> |
| 92 |
<li><code>ant-tptp.jar</code> (this jar has the same |
92 |
<li><code>ant-tptp.jar</code> (this jar has the same |
| 93 |
contents as the tptp-automation-client.jar but is renamed to be |
93 |
contents as the tptp-automation-client.jar but is renamed to be |
| 94 |
consistent with other ant task libraries)</li></ol> |
94 |
consistent with other Ant task libraries)</li></ol> |
| 95 |
|
95 |
|
| 96 |
<a name="Running_tests_from_ant_scripts"></a> |
96 |
<a name="Running_tests_from_ant_scripts"></a> |
| 97 |
<h3>Running tests from ant scripts</h3> |
97 |
<h3>Running tests from Ant scripts</h3> |
| 98 |
<p>Ensure ant is properly installed if tests are |
98 |
<p>Ensure Ant is properly installed if tests are |
| 99 |
to be run outside of a running Eclipse instance -- refer to the ant |
99 |
to be run outside of a running Eclipse instance -- refer to the Ant |
| 100 |
documentation to set this up properly. In addition to the standard ant |
100 |
documentation to set this up properly. In addition to the standard Ant |
| 101 |
installation, also copy the ant-tptp.jar into the ant lib directory alongside |
101 |
installation, also copy the ant-tptp.jar into the Ant <code>lib</code> directory alongside |
| 102 |
the other tasks. If running inside of Eclipse, ant is already installed |
102 |
the other tasks. If running inside of Eclipse, Ant is already installed |
| 103 |
although be sure to include the ant-tptp.jar in the additional tasks tab of the |
103 |
although be sure to include the ant-tptp.jar in the additional tasks tab of the |
| 104 |
ant launch configuration, or in the Global Entries section of the Ant Classpath |
104 |
Ant launch configuration, or in the Global Entries section of the Ant Classpath |
| 105 |
preference page. The embedded sample ant script below is an |
105 |
preference page. The embedded sample Ant script below is an |
| 106 |
example of an ant script named "test.xml" that executes a test named testA |
106 |
example of an Ant script named "test.xml" that executes a test named testA |
| 107 |
within the unit-tests folder of the test project. The test project is |
107 |
within the unit-tests folder of the test project. The test project is |
| 108 |
located in the specified workspace of "d:\development\workspace" and the Eclipse |
108 |
located in the specified workspace of "d:\development\workspace" and the Eclipse |
| 109 |
host instance is specified to be "d:\development\eclipse". Refer to the |
109 |
host instance is specified to be "d:\development\eclipse". Refer to the |
| 110 |
reference list of supported properties for other valid combinations of |
110 |
reference list of supported properties for other valid combinations of |
| 111 |
properties.</p><p>The example below uses the TPTP test task although it is also |
111 |
properties.</p><p>The example below uses the TPTP test task although it is also |
| 112 |
possible to execute a test using the TPTP automation task. |
112 |
possible to execute a test using the TPTP automation task. |
| 113 |
The more general automation task is an ant task for executing arbitrary |
113 |
The more general automation task is an Ant task for executing arbitrary |
| 114 |
TPTP automatable services which contrasts the more specific TPTP test task. The test |
114 |
TPTP automatable services which contrasts the more specific TPTP test task. The test |
| 115 |
task is a type-safe task specifically |
115 |
task is a type-safe task specifically |
| 116 |
created for launching the test execution service. More ant script examples are included in the |
116 |
created for launching the test execution service. More Ant script examples are included in the |
| 117 |
org.eclipse.hyades.execution plug-in's src-automation-client source folder's org.eclipse.hyades.automation.test package.</p> |
117 |
org.eclipse.hyades.execution plug-in's src-automation-client source folder's org.eclipse.hyades.automation.test package.</p> |
| 118 |
|
118 |
|
| 119 |
<pre> |
119 |
<pre> |
|
Lines 124-130
Link Here
|
| 124 |
<property name="tptp.automation.eclipse" value="D:\development\eclipse"/> |
124 |
<property name="tptp.automation.eclipse" value="D:\development\eclipse"/> |
| 125 |
|
125 |
|
| 126 |
<description> |
126 |
<description> |
| 127 |
This ant script tests the TPTP automation ant task client adapters |
127 |
This Ant script tests the TPTP automation Ant task client adapters |
| 128 |
and also can be used to test the underlying execution of TPTP automatable |
128 |
and also can be used to test the underlying execution of TPTP automatable |
| 129 |
services via the TPTP automation server |
129 |
services via the TPTP automation server |
| 130 |
</description> |
130 |
</description> |
|
Lines 148-167
Link Here
|
| 148 |
</pre> |
148 |
</pre> |
| 149 |
|
149 |
|
| 150 |
<a name="Running_multiple_tests_and_interrogating_their_results_from_ant_scripts"></a> |
150 |
<a name="Running_multiple_tests_and_interrogating_their_results_from_ant_scripts"></a> |
| 151 |
<h3>Running multiple tests and interrogating their results from ant scripts</h3> |
151 |
<h3>Running multiple tests and interrogating their results from Ant scripts</h3> |
| 152 |
<p>Ensure that you have properly configured ant to run TPTP automatable services |
152 |
<p>Ensure that you have properly configured Ant to run TPTP automatable services |
| 153 |
(refer to the <a href="#Running_tests_from_ant_scripts">above section</a> on running tests from ant.) The example below shows how |
153 |
(refer to the <a href="#Running_tests_from_ant_scripts">above section</a> on running tests from Ant.) The example below shows how |
| 154 |
to the use the tptp:execution task to execute multiple tests, and how to use the |
154 |
to the use the tptp:execution task to execute multiple tests, and how to use the |
| 155 |
tptp:interrogation task to interrogate the results of those test executions. |
155 |
tptp:interrogation task to interrogate the results of those test executions. |
| 156 |
Both of these ant tasks accept multiple ant <code>filesets</code> and/or |
156 |
Both of these Ant tasks accept multiple Ant <code>filesets</code> and/or |
| 157 |
<code>filelists</code>, allowing you to use wildcards or to directly |
157 |
<code>filelists</code>, allowing you to use wildcards or to directly |
| 158 |
specify multiple files as inputs to the tasks. In this example, we execute |
158 |
specify multiple files as inputs to the tasks. In this example, we execute |
| 159 |
all of the test suites in the TPTPJUnitTests project, and the set of |
159 |
all of the test suites in the TPTPJUnitTests project, and the set of |
| 160 |
all resulting .execution files is returned to us as a <code>filelist</code> |
160 |
all resulting .execution files is returned to us as a <code>filelist</code> |
| 161 |
in the ant variable that we supply (in this case "tptp.test.results".) We then |
161 |
in the Ant variable that we supply (in this case "tptp.test.results".) We then |
| 162 |
pass that <code>filelist</code> as input to the tptp:interrogation service, |
162 |
pass that <code>filelist</code> as input to the tptp:interrogation service, |
| 163 |
which will interrogate each of those execution files to determine an overall |
163 |
which will interrogate each of those execution files to determine an overall |
| 164 |
verdict and will assign it to the ant variable that we supply (in this case |
164 |
verdict and will assign it to the Ant variable that we supply (in this case |
| 165 |
"tptp.test.verdict".)</p> |
165 |
"tptp.test.verdict".)</p> |
| 166 |
|
166 |
|
| 167 |
<pre> |
167 |
<pre> |
|
Lines 199-215
Link Here
|
| 199 |
</pre> |
199 |
</pre> |
| 200 |
|
200 |
|
| 201 |
<a name="Generating_test_reports_from_ant_scripts"></a> |
201 |
<a name="Generating_test_reports_from_ant_scripts"></a> |
| 202 |
<h3>Generating test reports from ant scripts</h3> |
202 |
<h3>Generating test reports from Ant scripts</h3> |
| 203 |
<p>Ensure that you have properly configured ant to run TPTP automatable services |
203 |
<p>Ensure that you have properly configured Ant to run TPTP automatable services |
| 204 |
(refer to the <a href="#Running_tests_from_ant_scripts">above section</a> on running tests from ant.) The example below shows how |
204 |
(refer to the <a href="#Running_tests_from_ant_scripts">above section</a> on running tests from Ant.) The example below shows how |
| 205 |
to the use the tptp:execution task to execute multiple tests, and how to use the |
205 |
to the use the tptp:execution task to execute multiple tests, and how to use the |
| 206 |
tptp:publication task to generate BIRT based reports from the results of those |
206 |
tptp:publication task to generate BIRT based reports from the results of those |
| 207 |
test executions. The publication task also accepts multiple ant |
207 |
test executions. The publication task also accepts multiple Ant |
| 208 |
<code>filesets</code> and/or <code>filelists</code>, allowing you to use |
208 |
<code>filesets</code> and/or <code>filelists</code>, allowing you to use |
| 209 |
wildcards or to directly specify multiple files as inputs. In this example, |
209 |
wildcards or to directly specify multiple files as inputs. In this example, |
| 210 |
we execute all of the test suites in the TPTPJUnitTests project, and the set of |
210 |
we execute all of the test suites in the TPTPJUnitTests project, and the set of |
| 211 |
all resulting .execution files is returned to us as a <code>filelist</code> |
211 |
all resulting .execution files is returned to us as a <code>filelist</code> |
| 212 |
in the ant variable that we supply (in this case "tptp.test.results".) We then |
212 |
in the Ant variable that we supply (in this case "tptp.test.results".) We then |
| 213 |
pass that <code>filelist</code> as input to the tptp:publication service, |
213 |
pass that <code>filelist</code> as input to the tptp:publication service, |
| 214 |
which will generate a TPTP execution report using the default report template |
214 |
which will generate a TPTP execution report using the default report template |
| 215 |
(TestExecution.rptdesign, found in the org.eclipse.tptp.test.report.birt plugin), |
215 |
(TestExecution.rptdesign, found in the org.eclipse.tptp.test.report.birt plugin), |
|
Lines 248-257
Link Here
|
| 248 |
</pre> |
248 |
</pre> |
| 249 |
|
249 |
|
| 250 |
<a name="Generating_test_reports_from_ant_scripts2"></a> |
250 |
<a name="Generating_test_reports_from_ant_scripts2"></a> |
| 251 |
<h3>Customizing test reports from ant scripts</h3> |
251 |
<h3>Customizing test reports from Ant scripts</h3> |
| 252 |
<p>You may customize the behavior of the tptp:publication task by specifying |
252 |
<p>You may customize the behavior of the tptp:publication task by specifying |
| 253 |
a different BIRT report template, for instance TabularReport.rptdesign found in |
253 |
a different BIRT report template, for instance TabularReport.rptdesign found in |
| 254 |
the templates directory of plugin org.eclipse.tptp.test.report.birt. You may |
254 |
the <code>templates</code> directory of the org.eclipse.tptp.test.report.birt plug-in. You may |
| 255 |
also customize the behavior by using the publish-testsuites publication service |
255 |
also customize the behavior by using the publish-testsuites publication service |
| 256 |
and providing test suites as the input to the publication service instead of |
256 |
and providing test suites as the input to the publication service instead of |
| 257 |
test execution results files. If you use the publish-testsuites publication service and pass test |
257 |
test execution results files. If you use the publish-testsuites publication service and pass test |
|
Lines 279-285
Link Here
|
| 279 |
<!-- Run the report --> |
279 |
<!-- Run the report --> |
| 280 |
<target name="main" depends="test-results-publication"/> <br/> |
280 |
<target name="main" depends="test-results-publication"/> <br/> |
| 281 |
<target name="set-date-range"> |
281 |
<target name="set-date-range"> |
| 282 |
<!-- Use standard ant tstamp mechanisms to specify the desired date range --> |
282 |
<!-- Use standard Ant tstamp mechanisms to specify the desired date range --> |
| 283 |
<!-- This example uses a range between today and 60 days days prior --> |
283 |
<!-- This example uses a range between today and 60 days days prior --> |
| 284 |
<tstamp> |
284 |
<tstamp> |
| 285 |
<format property="startDateTime" pattern="MM/dd/yyyy hh:mm aa" offset="-60" unit="day"/> |
285 |
<format property="startDateTime" pattern="MM/dd/yyyy hh:mm aa" offset="-60" unit="day"/> |
|
Lines 310-331
Link Here
|
| 310 |
arguments. The TPTP automation client adapters provide a .bat and |
310 |
arguments. The TPTP automation client adapters provide a .bat and |
| 311 |
.cmd file for use within Windows and a .sh file provided for use |
311 |
.cmd file for use within Windows and a .sh file provided for use |
| 312 |
within the Linux operating system. It is possible to specify long |
312 |
within the Linux operating system. It is possible to specify long |
| 313 |
sets of arguments and/or frequently repeated arguments in a config |
313 |
sets of arguments and/or frequently repeated arguments in a configuration |
| 314 |
file to minimize the amount of typing necessary to run the test execution |
314 |
file to minimize the amount of typing necessary to run the test execution |
| 315 |
service from the command line. The shell automation client adapter |
315 |
service from the command line. The shell automation client adapter |
| 316 |
manifests as a text file (for the batch file or script) and a complementary Java |
316 |
manifests as a text file (for the batch file or script) and a complementary Java |
| 317 |
component -- both must exist to execute tests from the command line. |
317 |
component -- both must exist to execute tests from the command line. |
| 318 |
All the TPTP provided automation client adapters can be found in the |
318 |
All the TPTP provided automation client adapters can be found in the |
| 319 |
tptp-automation-client.jar although the ant-tptp.jar replicates some of the |
319 |
tptp-automation-client.jar although the ant-tptp.jar replicates some of the |
| 320 |
adapter code in the form of an ant jar (this was done in order to keep the |
320 |
adapter code in the form of an Ant jar (this was done in order to keep the |
| 321 |
TPTP task jar named consistently with the other available ant |
321 |
TPTP task jar named consistently with the other available Ant |
| 322 |
tasks).</p><p>The test execution service's properties are set by prefixing the |
322 |
tasks).</p><p>The test execution service's properties are set by prefixing the |
| 323 |
property names with a hyphen making sure to also specify the Eclipse home or |
323 |
property names with a hyphen making sure to also specify the Eclipse home or |
| 324 |
root directory.</p> |
324 |
root directory.</p> |
| 325 |
<a name="Running_tests_from_Java_applications_and_plug-ins"></a> |
325 |
<a name="Running_tests_from_Java_applications_and_plug-ins"></a> |
| 326 |
<h3>Running tests from Java applications and plug-ins</h3> |
326 |
<h3>Running tests from Java applications and plug-ins</h3> |
| 327 |
<p>Tests can be launched from any Java application and in |
327 |
<p>Tests can be launched from any Java application and in |
| 328 |
fact, the other client automation adapters such as the ant and shell adapters, |
328 |
fact, the other client automation adapters such as the Ant and shell adapters, |
| 329 |
use the Java adapter internally to hook into the service bus. The only |
329 |
use the Java adapter internally to hook into the service bus. The only |
| 330 |
requirement for executing tests from some Java code is to make sure the |
330 |
requirement for executing tests from some Java code is to make sure the |
| 331 |
tptp-automation-client.jar is properly referenced (in Eclipse this translates to |
331 |
tptp-automation-client.jar is properly referenced (in Eclipse this translates to |
|
Lines 334-340
Link Here
|
| 334 |
be running. </p><p>The Java automation client adapter provides a very |
334 |
be running. </p><p>The Java automation client adapter provides a very |
| 335 |
simple API for executing tests with all specifiable properties being contained |
335 |
simple API for executing tests with all specifiable properties being contained |
| 336 |
in a standard Java properties object. An example follows for executing the |
336 |
in a standard Java properties object. An example follows for executing the |
| 337 |
same test as introduced in the previous ant script example. Note that the |
337 |
same test as introduced in the previous Ant script example. Note that the |
| 338 |
service identifier passed in to the execute method is the service |
338 |
service identifier passed in to the execute method is the service |
| 339 |
identifier for the TPTP test execution service (this service supports all the |
339 |
identifier for the TPTP test execution service (this service supports all the |
| 340 |
base TPTP test types) -- it is also possible to execute other automatable |
340 |
base TPTP test types) -- it is also possible to execute other automatable |