Attachment 105529 Details for Bug 211777 – Updated part 1

Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Updated part 1

TPTP_Testing_Strategy_Part_1.html (text/html), 15.59 KB, created by Alan Haggarty

on 2008-06-20 14:24:02 EDT

(hide)

Description:

Filename:

MIME Type:

Creator: Alan Haggarty

Created: 2008-06-20 14:24:02 EDT

Size: 15.59 KB

patch

obsolete

><!DOCTYPE html PUBLIC "-//w3c//dtd html 4.0 transitional//en">
><html><head><title>TPTP Testing Strategy Part 1 - Creating and Executing TPTP Tests</title>
>
>
>
><meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
><link rel="stylesheet" type="text/css" href="http://www.eclipse.org/tptp/eclipse_style.css">
><link rel="stylesheet" type="text/css" href="http://www.eclipse.org/tptp/home/documents/process/TPTP_Testing_Strategy_files/strategy_doc_style.css">
><script type="text/javascript">
>		function expandOrCollapse()
>		{
>			element = document.getElementById("invisible");
>			if (element != null)
>			{
>				element.id = "dynamicAppearance";
>				document.getElementById("changeHistoryLink").lastChild.nodeValue = "Hide the Change History";
>				return;
>			}
>
>			element = document.getElementById("dynamicAppearance");
>			if (element != null)
>			{
>				element.id = "invisible";
>				document.getElementById("changeHistoryLink").lastChild.nodeValue = "Show the Change History";
>			}
>		}
>  </script></head><body>
><div id="page_header">Eclipse Test and Performance Tools Platform
>Project</div>
><div id="page_sub_header">TPTP Testing Strategy Part 1 - Creating and Executing TPTP Tests</div>
><div id="section_header">TPTP Testing Strategy Part 1 - Creating and Executing TPTP Tests</div>
>
><div class="content">
><table id="main_content">
>	<tbody>
>		<tr>
>			<td><b>Title:</b></td>
>			<td>TPTP Testing Strategy</td>
>		</tr>
>		<tr>
>			<td><b>Author:</b></td>
>
>			<td>Alan Haggarty (haggarty@ca.ibm.com)</td>
>		</tr>
>	</tbody>
></table>
><a href="javascript:expandOrCollapse()"><b><span id="changeHistoryLink">Show the Change History</span></b></a>
><table id="invisible" border="1">
>	<tbody>
>		<tr>
>			<th align="left" bgcolor="#dfdfdf" width="10%">Name</th>
>
>			<th align="left" bgcolor="#dfdfdf" width="20%">Date</th>
>			<th align="left" bgcolor="#dfdfdf" width="30%">Modified Sections
>			</th>
>		</tr>
>		<tr>
>			<td>Alan Haggarty (haggarty@ca.ibm.com)</td>
>			<td>June 4, 2008</td>
>			<td>Initial Creation</td>
>
>		</tr>
>	</tbody>
></table>  	
><br>
><div id="section_hlight">Table of Contents</div>
><div id="main_content">
><a href="#1.0">1.0 Preparation</a><br>
><a href="#1.1">1.1 Install the TPTP Testing Tools</a><br>
><a href="#1.2">1.2 Extracting the Test Resources</a><br>
><a href="#1.3">1.3 Structure of the Test Resources</a><br>
><a href="#2.0">2.0 Creating New TPTP Tests</a><br>
><a href="#3.0">3.0 Executing Tests</a><br>
><a href="#3.1">3.1 Creating Test Deployments</a><br>
><a href="#3.2">3.2 Executing Manual Test Suites</a><br>
><a href="#3.3">3.3 Executing JUnit and JUnit Plug-in Test Suites</a><br>
><a href="#3.4">3.4 Executing AGR Test Suites</a><br>
><a href="#3.5">3.5 Viewing Test Logs</a><br>
><a href="#3.6">3.6 Submitting Execution Histories</a><br>
><a href="#4.0">4.0 Help</a><br>
></div>
><br>
><div id="section_hlight"><a name="1.0">1.0 Preparation</a></div>
><div id="main_content">
></div>
><br>
><div id="section_hlight"><a name="1.1">1.1 Install the TPTP Testing Tools</a></div>
><div id="main_content">
>	<ol>
>		<li>Open the <a href=http://www.eclipse.org/tptp/home/downloads>TPTP download site.</a></li>
>	  <li>Select the release of TPTP to be tested and download and unzip the TPTP plugins as well as the requirements listed 
>	  	in the TPTP Plugins for Eclipse section.</li>
>	  <li>Download and install the Manual Test Tools ("As Is Components" section).</li>
>	  <li>Download and install the BIRT Test Reports ("As Is Components" section).</li>
>	  <li>Download and install Automated GUI Recording (As Is Components section).
>	         <br><i>Note: Remember to restart eclipse with the clean option after installing the AGR</i></li>
>	  <li>If remote execution is required download and unzip the Agent Controller (Agent Controller section).</li>
>	  <li>The Installation Guide and Release Notes are located in the Documentation section.</li>
>	</ol>
></div>
><br>
><div id="section_hlight"><a name="1.2">1.2 Extracting the Test Resources</a></div>
><div id="main_content">
>	<p>Test suites, cases, deployment configurations and execution histories are stored in the TPTP CVS repository 
>		(dev.eclipse.org:/cvsroot/tptp  or www.eclipse.org/tptp  &gt;&gt; CVS Repository), under the test-results  module.</p>
>	<p>The test-results module contains subfolders for each of the four subprojects - monitoring, platform, test and trace
>		 - in the TPTP project. Each subfolder contains the test projects for that subproject.</p>
>	<ol>
>		 <li>Check out the individual test projects under test-results/monitoring/, test-results/platform/, test-results/test/, or test-results/trace/ to the local workspace.
>						<br><b>For Example: </b><a href=http://dev.eclipse.org/viewcvs/index.cgi/test-results/platform/org.eclipse.hyades.use.cases/?root=TPTP_Project>test-results/platform/org.eclipse.hyades.use.cases</a></li>
>
>   	<li>To ensure that the local workspace contains the latest test projects, update at the beginning of each test pass and report generation.</li>
>	</ol>
></div>
><br>
><div id="section_hlight"><a name="1.3">1.3 Structure of the Test Resources</a></div>
><div id="main_content">
>	<p>Test projects must follow a defined structure.
>	</p>
>	<p>
>	Please see Part 2 for the required structure.<br>
>	<a href=TPTP_Testing_Strategy_Part_2.html#1.2>Test Project Detailed Structure</a><br>
>	<a href=TPTP_Testing_Strategy_Part_2.html#2.0>Test Creation Best Practices</a><br>
></p>
></div>
><br>
><div id="section_hlight"><a name="2.0">2.0 Creating New TPTP Tests</a></div>
><div id="main_content">
>	<p>To create new TPTP Manual, JUnit, JUnit Plug-in and AGR test suites and cases please follow the testing documentation
>		for each type of test. The location of the documentation can be found in the <a href="#4.0">Help section</a> of this document.
>	</p>
>	<p>The <a href=http://www.eclipse.org/tptp/home/documents/process/TPTP_Manual_Test_Case_Generator.html>TPTP Manual Test Case Generator</a>
>		 may be used to generate the structure of manual test case descriptions based on the standardized structured manual test case format.
>	</p> 
>	<p>Please see <a href=TPTP_Testing_Strategy_Part_2.html#2.0>Test Creation Best Practices</a> in Part 2 for further information on 
>	creating effective TPTP test cases.
>	</p>
></div>
><br>
><div id="section_hlight"><a name="3.0">3.0 Executing Tests</a></div>
><div id="main_content">
></div>
><br>
><div id="section_hlight"><a name="3.1">3.1 Creating Test Deployments</a></div>
><div id="main_content">
>	<p>
>	Creating Test Deployments: Detailed steps for creating test deployments can be found in Eclipse Help &gt;&gt; Help Contents &gt;&gt; Testing Applications &gt;&gt; Creating a Test Deployment
>	</p>
>	<p>Create a Test Launch Configuration</p>
>	<ol>
>   <li>Create a new launch configuration by selecting the entry Test from the list of configurations in Run &gt;&gt; Run Configurations</li>
>   <li>Use the Select test to run pane to navigate to and select the test suite you wish to run.</li>
>   <li>Select the deployment for running the test suite. By default the test will be deployed locally. Manual tests are always run in the local deployment.</li>
>   <li>In the Test Logs tab override the default log configuration to match the desired log directory (for example plugin_results). 
>   	Please refer to the <a href=TPTP_Testing_Strategy_Part_2.html#1.1>Naming Conventions</a> section of Part 2 for the names of result directories.</li>
>  </ol>
></div>
><br>
><div id="section_hlight"><a name="3.2">3.2 Executing Manual Test Suites</a></div>
><div id="main_content">
>	<ol>
>	   <li>Open the Test perspective.</li>
>   <li>In the Test Navigator view, select the manual test suite to be executed.</li>
>   <li>Launch the selected test suite by creating a Test configuration in the Run dialog
>   	<ul>
>		<li>Under the Test tab select the test suite</li>
>		<li>Choose the results folder associated with the selected test suite and deployment platform<br>
>	<b>Note:</b> The name and directory structure containing the generated execution history file 
>	<b>MUST</b> be the same as it will be in CVS in order for generated reports to work correctly.</li>
>		<li>Click Run</li>
>		</ul>
>		</li>
>   <li>Launch a candidate workbench on the testing platform</li>
>   <li>Execute each test case as specified in the TPTP Manual Test View on the local host.</li>
>   <li>Record the result of the test case execution by capturing the status of the test case execution using the following explanation for TPTP Testing Tools Status options:
>	<table border="1" width="100%">
>		<tbody>
>			<tr>
>				<th align="left">TPTP Testing Tools Status</th>
>				<th align="left">Explanation</th>
>			</tr>
>			<tr>
>				<td>Error</td>
>				<td>Do not use.</td>
>			</tr>
>			<tr>
>				<td>Fail</td>
>				<td>The test case could be executed and failed due to a
>				blocking defect. <br>
>				</td>
>			</tr>
>			<tr>
>				<td>Inconclusive</td>
>				<td>The test case was not applicable OR the test case could be executed and failed due to a non-blocking defect.
>			A non-blocking defect is defined as a nice-to-have fix that does not degrade the core functionality being tested by the test case. 
>			</td>
>			</tr>
>			<tr>
>				<td>Pass</td>
>				<td>The test case could be executed and passed.</td>
>			</tr>
>		</tbody>
>		
>	</table>
>	For verdicts of Fail or Inconclusive please provide a defect number and explanation in the test log.</li>
></ol>
></div>
><br>
><div id="section_hlight"><a name="3.3">3.3 Executing JUnit and JUnit Plug-in Test Suites</a></div>
><div id="main_content">
>	<ol>
>		<li>Open the Test perspective.</li>
>   	<li>In the Test Navigator view, select the JUnit test suite to be executed.</li>
>   	<li>Launch the selected test suite by creating a Test configuration in the Run dialog
>   		<ul>
>				<li>Under the Test tab select the test suite</li>
>				<li>Select the deployment.<br>
>				To run a test remotely you will need to create and configure a new test deployment<br>
>				For JUnit plug-in tests you also may want to specify deployment on a different workbench location.</li>
>				<li>Choose the results folder associated with the selected test suite and deployment platform<br>
>				<b>Note: </b>The name and directory structure containing the generated execution history file MUST be the same as it will be in CVS in order for generated reports to work correctly.</li>
>				<li>Click Run</li>
>			</ul>
>		</li>
>   	<li>An execution history will be generated according to the following explanation of TPTP Testing Tools Status options:
>	<table border="1" width="100%">
>		<tbody>
>			<tr>
>				<th align="left">TPTP Testing Tools Status</th>
>				<th align="left">Explanation</th>
>			</tr>
>			<tr>
>				<td>Error</td>
>				<td>The test case was or could not be executed due to a run
>				time error.</td>
>			</tr>
>			<tr>
>				<td>Fail</td>
>				<td>The test case could be executed and failed due to a defect.</td>
>			</tr>
>			<tr>
>				<td>Pass</td>
>				<td>The test case could be executed and passed.</td>
>			</tr>
>		</tbody>
>	</table>
>	The Fail and Error verdict events in the test log will contain a stack trace that can jump to source.</li>
></ol>
></div>
><br>
><div id="section_hlight"><a name="3.4">3.4 Executing AGR Test Suites</a></div>
><div id="main_content">
>	<ol>
>		<li>Ensure that the Agent Controller is installed and started.</li>
>   <li>Open the Test perspective.</li>
>   <li>In the Test Navigator view, select the Automated GUI Test to be executed.</li>
>   <li>Execute the test suite by creating a launch configuration item.<br>
>		<b>Important:</b> Using the 'Quick Run' mode will not generate an execution history file. Make sure that a proper launch configuration is used.</li>
>   <li>The runner will generate an execution history according to the following explanation of TPTP Testing Tools Status options:
>   <table border="1" width="100%">
>		<tbody>
>			<tr>
>				<th align="left">TPTP Testing Tools Status</th>
>				<th align="left">Explanation</th>
>			</tr>
>			<tr>
>				<td>Error</td>
>				<td>The test case was or could not be executed due to a run
>				time error.</td>
>			</tr>
>			<tr>
>				<td>Fail</td>
>				<td>The test case could be executed and failed due to a defect.</td>
>			</tr>
>			<tr>
>				<td>Pass</td>
>				<td>The test case could be executed and passed.</td>
>			</tr>
>		</tbody>
>	</table>	
>	The Fail and Error verdict events in the test log will contain a stack trace that can jump to source.
>   	</li>
>	</ol>
>	
>	
></div>
><br>
><div id="section_hlight"><a name="3.5">3.5 Viewing Test Logs</a></div>
><div id="main_content">
>	<p>
>		To open a test log file, use the Open File window, or double-click in the Test Navigator the file with the extension, .execution.
>	</p>
>	<ol>
>   <li>From the File menu, click Open File.</li>
>   <li>Browse to and select the test log file to open. (Test log files use the extension, .execution.)</li>
>   <li>Click OK.</li>
>	</ol>
></div>
><br>
><div id="section_hlight"><a name="3.6">3.6 Submitting Execution Histories</a></div>
><div id="main_content">
>	<ol>
>		<li>Execution histories should be checked-in to the CVS branch for the release under test.  
>			They are checked into a results folder under the associated plugin for the type of test executed 
>			(manual_results, junit_results, junit_plugin_results, gui_results). To delineate between test executions on multiple platforms, 
>			each results folder contains subfolders for the supported platforms.
>			<br>
>			<b>For Example: </b>The results of executing the manual test 
>			<i>org.eclipse.hyades.use.cases/manual/Profiling_and_Logging/Platform.Communication.Control_Channel_Remote_Profiling.Windows_IA32.testsuite</i> 
>			should be generated to and checked into <i>org.eclipse.hyades.use.cases/manual_results/Profiling_and_Logging/Windows_XP</i>
>		</li>
>   <li>Testers who are not committers should email the execution history files to the committer for the associated component.
>			<br>
>			Include the following in your email:
>			<ul>
>				<li>Name of tester if not same as sender</li>
>				<li>Deployment platform(s) including name, version and release.</li>
>				<li>Execution specific information including name, version and release.</li>
>				<li>TPTP driver build id and test pass identifier.</li>
>			</ul>
>		</li>
>	</ol>
></div>
><br>
><div id="section_hlight"><a name="4.0">4.0 Help</a></div>
><div id="main_content">
>	<p>
>	<b>General Help</b><br><br>
>	TPTP General documentation can be found in the Eclipse Help under Help &gt;&gt; Help Contents &gt;&gt; Testing Applications<br>
>	<a href=http://www.eclipse.org/tptp/test/documents/gettingstarted/Automated_GUI_Recorder_Getting_Started.html>
>		Getting Started with TPTP's Automated GUI Recorder</a><br>
>	<a href=http://www.eclipse.org/tptp/test/documents/gettingstarted/Manual_Test_Tools_Getting_Started.html>
>		Getting Started with TPTP's Manual Test Tools</a><br>
>	<a href=http://www.eclipse.org/tptp/test/documents/gettingstarted/BIRT_Test_Reports_Getting_Started.html>
>	Getting Started with TPTP's Business Intelligence and Reporting Tools (BIRT) Test Reports</a><br>
></p>
><p>
>	<b>Creating Tests</b><br><br>
>	Manual test cases and suites: Eclipse Help &gt;&gt; Help Contents &gt;&gt;  Testing Applications &gt;&gt; Testing Manually<br>
>	TPTP JUnit: Eclipse Help &gt;&gt; Help Contents &gt;&gt;  Testing Applications &gt;&gt; Testing with JUnit<br>
>	TPTP JUnit Plug-in: Eclipse Help &gt;&gt; Help Contents &gt;&gt; Testing Applications &gt;&gt; Testing plug-ins with JUnit<br>
>	AGR: See the <a href=http://www.eclipse.org/tptp/test/documents/gettingstarted/Automated_GUI_Recorder_Getting_Started.html#3.0>
>		Documentation section of the AGR Getting Started Guide</a><br>
></p>
>	<p>
>	<b>More Resources</b><br><br>
>	<a href=http://www.eclipse.org/tptp/>Eclipse Test & Performance Tools Platform Project Web Site</a><br>
>	<a href=http://www.eclipse.org/tptp/home/documents/extenders.php>Documentation for TPTP extenders</a><br>
>	<a href=http://www.eclipse.org/tptp/home/project_info/general/mailnews.php>TPTP Mailing Lists and Newsgroups</a>
>	</p>
></div>
><br>
></body>
></html>

<!DOCTYPE html PUBLIC "-//w3c//dtd html 4.0 transitional//en">
<html><head><title>TPTP Testing Strategy Part 1 - Creating and Executing TPTP Tests</title>



<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<link rel="stylesheet" type="text/css" href="http://www.eclipse.org/tptp/eclipse_style.css">
<link rel="stylesheet" type="text/css" href="http://www.eclipse.org/tptp/home/documents/process/TPTP_Testing_Strategy_files/strategy_doc_style.css">
<script type="text/javascript">
		function expandOrCollapse()
		{
			element = document.getElementById("invisible");
			if (element != null)
			{
				element.id = "dynamicAppearance";
				document.getElementById("changeHistoryLink").lastChild.nodeValue = "Hide the Change History";
				return;
			}

			element = document.getElementById("dynamicAppearance");
			if (element != null)
			{
				element.id = "invisible";
				document.getElementById("changeHistoryLink").lastChild.nodeValue = "Show the Change History";
			}
		}
  </script></head><body>
<div id="page_header">Eclipse Test and Performance Tools Platform
Project</div>
<div id="page_sub_header">TPTP Testing Strategy Part 1 - Creating and Executing TPTP Tests</div>
<div id="section_header">TPTP Testing Strategy Part 1 - Creating and Executing TPTP Tests</div>

<div class="content"><!-- Main content goes here -->
<table id="main_content">
	<tbody>
		<tr>
			<td><b>Title:</b></td>
			<td>TPTP Testing Strategy</td>
		</tr>
		<tr>
			<td><b>Author:</b></td>

			<td>Alan Haggarty (haggarty@ca.ibm.com)</td>
		</tr>
	</tbody>
</table>
<a href="javascript:expandOrCollapse()"><b><span id="changeHistoryLink">Show the Change History</span></b></a>
<table id="invisible" border="1">
	<tbody>
		<tr>
			<th align="left" bgcolor="#dfdfdf" width="10%">Name</th>

			<th align="left" bgcolor="#dfdfdf" width="20%">Date</th>
			<th align="left" bgcolor="#dfdfdf" width="30%">Modified Sections
			</th>
		</tr>
		<tr>
			<td>Alan Haggarty (haggarty@ca.ibm.com)</td>
			<td>June 4, 2008</td>
			<td>Initial Creation</td>

		</tr>
	</tbody>
</table>  	
<br>
<div id="section_hlight">Table of Contents</div>
<div id="main_content">
<a href="#1.0">1.0 Preparation</a><br>
<a href="#1.1">1.1 Install the TPTP Testing Tools</a><br>
<a href="#1.2">1.2 Extracting the Test Resources</a><br>
<a href="#1.3">1.3 Structure of the Test Resources</a><br>
<a href="#2.0">2.0 Creating New TPTP Tests</a><br>
<a href="#3.0">3.0 Executing Tests</a><br>
<a href="#3.1">3.1 Creating Test Deployments</a><br>
<a href="#3.2">3.2 Executing Manual Test Suites</a><br>
<a href="#3.3">3.3 Executing JUnit and JUnit Plug-in Test Suites</a><br>
<a href="#3.4">3.4 Executing AGR Test Suites</a><br>
<a href="#3.5">3.5 Viewing Test Logs</a><br>
<a href="#3.6">3.6 Submitting Execution Histories</a><br>
<a href="#4.0">4.0 Help</a><br>
</div>
<br>
<div id="section_hlight"><a name="1.0">1.0 Preparation</a></div>
<div id="main_content">
</div>
<br>
<div id="section_hlight"><a name="1.1">1.1 Install the TPTP Testing Tools</a></div>
<div id="main_content">
	<ol>
		<li>Open the <a href=http://www.eclipse.org/tptp/home/downloads>TPTP download site.</a></li>
	  <li>Select the release of TPTP to be tested and download and unzip the TPTP plugins as well as the requirements listed 
	  	in the TPTP Plugins for Eclipse section.</li>
	  <li>Download and install the Manual Test Tools ("As Is Components" section).</li>
	  <li>Download and install the BIRT Test Reports ("As Is Components" section).</li>
	  <li>Download and install Automated GUI Recording (As Is Components section).
	         <br><i>Note: Remember to restart eclipse with the clean option after installing the AGR</i></li>
	  <li>If remote execution is required download and unzip the Agent Controller (Agent Controller section).</li>
	  <li>The Installation Guide and Release Notes are located in the Documentation section.</li>
	</ol>
</div>
<br>
<div id="section_hlight"><a name="1.2">1.2 Extracting the Test Resources</a></div>
<div id="main_content">
	<p>Test suites, cases, deployment configurations and execution histories are stored in the TPTP CVS repository 
		(dev.eclipse.org:/cvsroot/tptp  or www.eclipse.org/tptp  &gt;&gt; CVS Repository), under the test-results  module.</p>
	<p>The test-results module contains subfolders for each of the four subprojects - monitoring, platform, test and trace
		 - in the TPTP project. Each subfolder contains the test projects for that subproject.</p>
	<ol>
		 <li>Check out the individual test projects under test-results/monitoring/, test-results/platform/, test-results/test/, or test-results/trace/ to the local workspace.
						<br><b>For Example: </b><a href=http://dev.eclipse.org/viewcvs/index.cgi/test-results/platform/org.eclipse.hyades.use.cases/?root=TPTP_Project>test-results/platform/org.eclipse.hyades.use.cases</a></li>

   	<li>To ensure that the local workspace contains the latest test projects, update at the beginning of each test pass and report generation.</li>
	</ol>
</div>
<br>
<div id="section_hlight"><a name="1.3">1.3 Structure of the Test Resources</a></div>
<div id="main_content">
	<p>Test projects must follow a defined structure.
	</p>
	<p>
	Please see Part 2 for the required structure.<br>
	<a href=TPTP_Testing_Strategy_Part_2.html#1.2>Test Project Detailed Structure</a><br>
	<a href=TPTP_Testing_Strategy_Part_2.html#2.0>Test Creation Best Practices</a><br>
</p>
</div>
<br>
<div id="section_hlight"><a name="2.0">2.0 Creating New TPTP Tests</a></div>
<div id="main_content">
	<p>To create new TPTP Manual, JUnit, JUnit Plug-in and AGR test suites and cases please follow the testing documentation
		for each type of test. The location of the documentation can be found in the <a href="#4.0">Help section</a> of this document.
	</p>
	<p>The <a href=http://www.eclipse.org/tptp/home/documents/process/TPTP_Manual_Test_Case_Generator.html>TPTP Manual Test Case Generator</a>
		 may be used to generate the structure of manual test case descriptions based on the standardized structured manual test case format.
	</p> 
	<p>Please see <a href=TPTP_Testing_Strategy_Part_2.html#2.0>Test Creation Best Practices</a> in Part 2 for further information on 
	creating effective TPTP test cases.
	</p>
</div>
<br>
<div id="section_hlight"><a name="3.0">3.0 Executing Tests</a></div>
<div id="main_content">
</div>
<br>
<div id="section_hlight"><a name="3.1">3.1 Creating Test Deployments</a></div>
<div id="main_content">
	<p>
	Creating Test Deployments: Detailed steps for creating test deployments can be found in Eclipse Help &gt;&gt; Help Contents &gt;&gt; Testing Applications &gt;&gt; Creating a Test Deployment
	</p>
	<p>Create a Test Launch Configuration</p>
	<ol>
   <li>Create a new launch configuration by selecting the entry Test from the list of configurations in Run &gt;&gt; Run Configurations</li>
   <li>Use the Select test to run pane to navigate to and select the test suite you wish to run.</li>
   <li>Select the deployment for running the test suite. By default the test will be deployed locally. Manual tests are always run in the local deployment.</li>
   <li>In the Test Logs tab override the default log configuration to match the desired log directory (for example plugin_results). 
   	Please refer to the <a href=TPTP_Testing_Strategy_Part_2.html#1.1>Naming Conventions</a> section of Part 2 for the names of result directories.</li>
  </ol>
</div>
<br>
<div id="section_hlight"><a name="3.2">3.2 Executing Manual Test Suites</a></div>
<div id="main_content">
	<ol>
	   <li>Open the Test perspective.</li>
   <li>In the Test Navigator view, select the manual test suite to be executed.</li>
   <li>Launch the selected test suite by creating a Test configuration in the Run dialog
   	<ul>
		<li>Under the Test tab select the test suite</li>
		<li>Choose the results folder associated with the selected test suite and deployment platform<br>
	<b>Note:</b> The name and directory structure containing the generated execution history file 
	<b>MUST</b> be the same as it will be in CVS in order for generated reports to work correctly.</li>
		<li>Click Run</li>
		</ul>
		</li>
   <li>Launch a candidate workbench on the testing platform</li>
   <li>Execute each test case as specified in the TPTP Manual Test View on the local host.</li>
   <li>Record the result of the test case execution by capturing the status of the test case execution using the following explanation for TPTP Testing Tools Status options:
	<table border="1" width="100%">
		<tbody>
			<tr>
				<th align="left">TPTP Testing Tools Status</th>
				<th align="left">Explanation</th>
			</tr>
			<tr>
				<td>Error</td>
				<td>Do not use.</td>
			</tr>
			<tr>
				<td>Fail</td>
				<td>The test case could be executed and failed due to a
				blocking defect. <br>
				</td>
			</tr>
			<tr>
				<td>Inconclusive</td>
				<td>The test case was not applicable OR the test case could be executed and failed due to a non-blocking defect.
			A non-blocking defect is defined as a nice-to-have fix that does not degrade the core functionality being tested by the test case. 
			</td>
			</tr>
			<tr>
				<td>Pass</td>
				<td>The test case could be executed and passed.</td>
			</tr>
		</tbody>
		
	</table>
	For verdicts of Fail or Inconclusive please provide a defect number and explanation in the test log.</li>
</ol>
</div>
<br>
<div id="section_hlight"><a name="3.3">3.3 Executing JUnit and JUnit Plug-in Test Suites</a></div>
<div id="main_content">
	<ol>
		<li>Open the Test perspective.</li>
   	<li>In the Test Navigator view, select the JUnit test suite to be executed.</li>
   	<li>Launch the selected test suite by creating a Test configuration in the Run dialog
   		<ul>
				<li>Under the Test tab select the test suite</li>
				<li>Select the deployment.<br>
				To run a test remotely you will need to create and configure a new test deployment<br>
				For JUnit plug-in tests you also may want to specify deployment on a different workbench location.</li>
				<li>Choose the results folder associated with the selected test suite and deployment platform<br>
				<b>Note: </b>The name and directory structure containing the generated execution history file MUST be the same as it will be in CVS in order for generated reports to work correctly.</li>
				<li>Click Run</li>
			</ul>
		</li>
   	<li>An execution history will be generated according to the following explanation of TPTP Testing Tools Status options:
	<table border="1" width="100%">
		<tbody>
			<tr>
				<th align="left">TPTP Testing Tools Status</th>
				<th align="left">Explanation</th>
			</tr>
			<tr>
				<td>Error</td>
				<td>The test case was or could not be executed due to a run
				time error.</td>
			</tr>
			<tr>
				<td>Fail</td>
				<td>The test case could be executed and failed due to a defect.</td>
			</tr>
			<tr>
				<td>Pass</td>
				<td>The test case could be executed and passed.</td>
			</tr>
		</tbody>
	</table>
	The Fail and Error verdict events in the test log will contain a stack trace that can jump to source.</li>
</ol>
</div>
<br>
<div id="section_hlight"><a name="3.4">3.4 Executing AGR Test Suites</a></div>
<div id="main_content">
	<ol>
		<li>Ensure that the Agent Controller is installed and started.</li>
   <li>Open the Test perspective.</li>
   <li>In the Test Navigator view, select the Automated GUI Test to be executed.</li>
   <li>Execute the test suite by creating a launch configuration item.<br>
		<b>Important:</b> Using the 'Quick Run' mode will not generate an execution history file. Make sure that a proper launch configuration is used.</li>
   <li>The runner will generate an execution history according to the following explanation of TPTP Testing Tools Status options:
   <table border="1" width="100%">
		<tbody>
			<tr>
				<th align="left">TPTP Testing Tools Status</th>
				<th align="left">Explanation</th>
			</tr>
			<tr>
				<td>Error</td>
				<td>The test case was or could not be executed due to a run
				time error.</td>
			</tr>
			<tr>
				<td>Fail</td>
				<td>The test case could be executed and failed due to a defect.</td>
			</tr>
			<tr>
				<td>Pass</td>
				<td>The test case could be executed and passed.</td>
			</tr>
		</tbody>
	</table>	
	The Fail and Error verdict events in the test log will contain a stack trace that can jump to source.
   	</li>
	</ol>
	
	
</div>
<br>
<div id="section_hlight"><a name="3.5">3.5 Viewing Test Logs</a></div>
<div id="main_content">
	<p>
		To open a test log file, use the Open File window, or double-click in the Test Navigator the file with the extension, .execution.
	</p>
	<ol>
   <li>From the File menu, click Open File.</li>
   <li>Browse to and select the test log file to open. (Test log files use the extension, .execution.)</li>
   <li>Click OK.</li>
	</ol>
</div>
<br>
<div id="section_hlight"><a name="3.6">3.6 Submitting Execution Histories</a></div>
<div id="main_content">
	<ol>
		<li>Execution histories should be checked-in to the CVS branch for the release under test.  
			They are checked into a results folder under the associated plugin for the type of test executed 
			(manual_results, junit_results, junit_plugin_results, gui_results). To delineate between test executions on multiple platforms, 
			each results folder contains subfolders for the supported platforms.
			<br>
			<b>For Example: </b>The results of executing the manual test 
			<i>org.eclipse.hyades.use.cases/manual/Profiling_and_Logging/Platform.Communication.Control_Channel_Remote_Profiling.Windows_IA32.testsuite</i> 
			should be generated to and checked into <i>org.eclipse.hyades.use.cases/manual_results/Profiling_and_Logging/Windows_XP</i>
		</li>
   <li>Testers who are not committers should email the execution history files to the committer for the associated component.
			<br>
			Include the following in your email:
			<ul>
				<li>Name of tester if not same as sender</li>
				<li>Deployment platform(s) including name, version and release.</li>
				<li>Execution specific information including name, version and release.</li>
				<li>TPTP driver build id and test pass identifier.</li>
			</ul>
		</li>
	</ol>
</div>
<br>
<div id="section_hlight"><a name="4.0">4.0 Help</a></div>
<div id="main_content">
	<p>
	<b>General Help</b><br><br>
	TPTP General documentation can be found in the Eclipse Help under Help &gt;&gt; Help Contents &gt;&gt; Testing Applications<br>
	<a href=http://www.eclipse.org/tptp/test/documents/gettingstarted/Automated_GUI_Recorder_Getting_Started.html>
		Getting Started with TPTP's Automated GUI Recorder</a><br>
	<a href=http://www.eclipse.org/tptp/test/documents/gettingstarted/Manual_Test_Tools_Getting_Started.html>
		Getting Started with TPTP's Manual Test Tools</a><br>
	<a href=http://www.eclipse.org/tptp/test/documents/gettingstarted/BIRT_Test_Reports_Getting_Started.html>
	Getting Started with TPTP's Business Intelligence and Reporting Tools (BIRT) Test Reports</a><br>
</p>
<p>
	<b>Creating Tests</b><br><br>
	Manual test cases and suites: Eclipse Help &gt;&gt; Help Contents &gt;&gt;  Testing Applications &gt;&gt; Testing Manually<br>
	TPTP JUnit: Eclipse Help &gt;&gt; Help Contents &gt;&gt;  Testing Applications &gt;&gt; Testing with JUnit<br>
	TPTP JUnit Plug-in: Eclipse Help &gt;&gt; Help Contents &gt;&gt; Testing Applications &gt;&gt; Testing plug-ins with JUnit<br>
	AGR: See the <a href=http://www.eclipse.org/tptp/test/documents/gettingstarted/Automated_GUI_Recorder_Getting_Started.html#3.0>
		Documentation section of the AGR Getting Started Guide</a><br>
</p>
	<p>
	<b>More Resources</b><br><br>
	<a href=http://www.eclipse.org/tptp/>Eclipse Test & Performance Tools Platform Project Web Site</a><br>
	<a href=http://www.eclipse.org/tptp/home/documents/extenders.php>Documentation for TPTP extenders</a><br>
	<a href=http://www.eclipse.org/tptp/home/project_info/general/mailnews.php>TPTP Mailing Lists and Newsgroups</a>
	</p>
</div>
<br>
</body>
</html>

Actions: View

Attachments on bug 211777: 102509 | 102510 | 104770 | 104771 | 105529 | 105532