Attachment 104770 Details for Bug 211777 – Part 1

Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Part 1

TPTP_Testing_Strategy_Part_1.html (text/html), 16.74 KB, created by Alan Haggarty

on 2008-06-12 17:38:01 EDT

(hide)

Description:

Filename:

MIME Type:

Creator: Alan Haggarty

Created: 2008-06-12 17:38:01 EDT

Size: 16.74 KB

patch

obsolete

><!DOCTYPE html PUBLIC "-//w3c//dtd html 4.0 transitional//en">
><html><head><title>TPTP Testing Strategy Part 1 - Creating and Executing TPTP Tests</title>
>
>
>
><meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
><link rel="stylesheet" type="text/css" href="http://www.eclipse.org/tptp/eclipse_style.css">
><link rel="stylesheet" type="text/css" href="http://www.eclipse.org/tptp/home/documents/process/TPTP_Testing_Strategy_files/strategy_doc_style.css">
><script type="text/javascript">
>		function expandOrCollapse()
>		{
>			element = document.getElementById("invisible");
>			if (element != null)
>			{
>				element.id = "dynamicAppearance";
>				document.getElementById("changeHistoryLink").lastChild.nodeValue = "Hide the Change History";
>				return;
>			}
>
>			element = document.getElementById("dynamicAppearance");
>			if (element != null)
>			{
>				element.id = "invisible";
>				document.getElementById("changeHistoryLink").lastChild.nodeValue = "Show the Change History";
>			}
>		}
>  </script></head><body>
><div id="page_header">Eclipse Test and Performance Tools Platform
>Project</div>
><div id="page_sub_header">TPTP Testing Strategy Part 1 - Creating and Executing TPTP Tests</div>
><div id="section_header">TPTP Testing Strategy Part 1 - Creating and Executing TPTP Tests</div>
>
><div class="content">
><table id="main_content">
>	<tbody>
>		<tr>
>			<td><b>Title:</b></td>
>			<td>TPTP Testing Strategy</td>
>		</tr>
>		<tr>
>			<td><b>Author:</b></td>
>
>			<td>Alan Haggarty (haggarty@ca.ibm.com)</td>
>		</tr>
>	</tbody>
></table>
><a href="javascript:expandOrCollapse()"><b><span id="changeHistoryLink">Show the Change History</span></b></a>
><table id="invisible" border="1">
>	<tbody>
>		<tr>
>			<th align="left" bgcolor="#dfdfdf" width="10%">Name</th>
>
>			<th align="left" bgcolor="#dfdfdf" width="20%">Date</th>
>			<th align="left" bgcolor="#dfdfdf" width="30%">Modified Sections
>			</th>
>		</tr>
>		<tr>
>			<td>Alan Haggarty (haggarty@ca.ibm.com)</td>
>			<td>June 4, 2008</td>
>			<td>Initial Creation</td>
>
>		</tr>
>	</tbody>
></table>  	
><br>
><div id="section_hlight">Table of Contents</div>
><div id="main_content">
><a href="#1.0">1.0 Preparation</a><br>
><a href="#1.1">1.1 Install the TPTP Testing Tools</a><br>
><a href="#1.2">1.2 Extracting the Test Resources</a><br>
><a href="#1.3">1.3 Structure of the Test Resources</a><br>
><a href="#2.0">2.0 Creating New TPTP Tests</a><br>
><a href="#3.0">3.0 Executing Tests</a><br>
><a href="#3.1">3.1 Creating Test Deployments</a><br>
><a href="#3.2">3.2 Executing Manual Test Suites</a><br>
><a href="#3.3">3.3 Executing JUnit and JUnit Plug-in Test Suites</a><br>
><a href="#3.4">3.4 Executing AGR Test Suites</a><br>
><a href="#4.0">4.0 Common Tasks</a><br>
><a href="#4.1">4.1 Creating Test Cases</a><br>
><a href="#4.2">4.2 Viewing Test Logs</a><br>
><a href="#4.3">4.3 Submitting Execution Histories</a><br>
><a href="#5.0">5.0 Help</a><br>
></div>
><br>
><div id="section_hlight"><a name="1.0">1.0 Preparation</a></div>
><div id="main_content">
></div>
><br>
><div id="section_hlight"><a name="1.1">1.1 Install the TPTP Testing Tools</a></div>
><div id="main_content">
>	<ol>
>		<li>Open the <a href=http://www.eclipse.org/tptp/home/downloads>TPTP download site.</a></li>
>	  <li>Select the release of TPTP to be tested and download and unzip the TPTP plugins as well as the requirements listed 
>	  	in the TPTP Plugins for Eclipse section.</li>
>	  <li>Download and install the Manual Test Tools ("As Is Components" section).</li>
>	  <li>Download and install the BIRT Test Reports ("As Is Components" section).</li>
>	  <li>Download and install Automated GUI Recording (As Is Components section).
>	         <br><i>Note: Remember to restart eclipse with the clean option after installing the AGR</i></li>
>	  <li>If remote execution is required download and unzip the Agent Controller (Agent Controller section).</li>
>	  <li>The Installation Guide and Release Notes are located in the Documentation section.</li>
>	</ol>
></div>
><br>
><div id="section_hlight"><a name="1.2">1.2 Extracting the Test Resources</a></div>
><div id="main_content">
>	<p>Test suites, cases, deployment configurations and execution histories are stored in the TPTP CVS repository 
>		(dev.eclipse.org:/cvsroot/tptp  or www.eclipse.org/tptp  &gt;&gt; CVS Repository), under the test-results  module.</p>
>	<p>The test-results module contains subfolders for each of the four subprojects - monitoring, platform, test and trace
>		 - in the TPTP project. Each subfolder contains the test projects for that subproject.</p>
>	<ol>
>		 <li>Check out the individual test projects under test-results/monitoring/, test-results/platform/, test-results/test/, or test-results/trace/ to the local workspace.
>						<br><b>For Example: </b><a href=http://dev.eclipse.org/viewcvs/index.cgi/test-results/platform/org.eclipse.hyades.use.cases/?root=TPTP_Project>test-results/platform/org.eclipse.hyades.use.cases</a></li>
>
>   	<li>To ensure that the local workspace contains the latest test projects, update at the beginning of each test pass and report generation.</li>
>	</ol>
></div>
><br>
><div id="section_hlight"><a name="1.3">1.3 Structure of the Test Resources</a></div>
><div id="main_content">
>	<p>Test projects must follow a defined structure.
>	</p>
>	<p>
>	Please see Part 2 for the required structure.<br>
>	<a href=TPTP_Testing_Strategy_Part_2.html#1.2>Test Project Detailed Structure</a><br>
>	<a href=TPTP_Testing_Strategy_Part_2.html#2.0>Test Creation Best Practices</a><br>
></p>
></div>
><br>
><div id="section_hlight"><a name="2.0">2.0 Creating New TPTP Tests</a></div>
><div id="main_content">
>	<p>To create new TPTP Manual, JUnit, JUnit Plug-in and AGR test suites and cases please follow the testing documentation
>		for each type of test. The location of the documentation can be found in the <a href="#5.0">Help section</a> of this document.
>	</p>
>	<p>The <a href=http://www.eclipse.org/tptp/home/documents/process/TPTP_Manual_Test_Case_Generator.html>TPTP Manual Test Case Generator</a>
>		 may be used to generate the structure of manual test case descriptions based on the standardized structured manual test case format.
>	</p> 
></div>
><br>
><div id="section_hlight"><a name="3.0">3.0 Executing Tests</a></div>
><div id="main_content">
></div>
><br>
><div id="section_hlight"><a name="3.1">3.1 Creating Test Deployments</a></div>
><div id="main_content">
>	<p>
>	Creating Test Deployments: Detailed steps for creating test deployments can be found in Eclipse Help &gt;&gt; Help Contents &gt;&gt; Testing Applications &gt;&gt; Creating a Test Deployment
>	</p>
>	<p>Create a Test Launch Configuration</p>
>	<ol>
>   <li>Create a new launch configuration by selecting the entry Test from the list of configurations in Run &gt;&gt; Run Configurations</li>
>   <li>Use the Select test to run pane to navigate to and select the test suite you wish to run.</li>
>   <li>Select the deployment for running the test suite. By default the test will be deployed locally. Manual tests are always run in the local deployment.</li>
>   <li>In the Test Logs tab override the default log configuration to match the desired log directory (for example plugin_results). 
>   	Please refer to the <a href=TPTP_Testing_Strategy_Part_2.html#1.1>Naming Conventions</a> section of Part 2 for the names of result directories.</li>
>  </ol>
></div>
><br>
><div id="section_hlight"><a name="3.2">3.2 Executing Manual Test Suites</a></div>
><div id="main_content">
>	<ol>
>	   <li>Open the Test perspective.</li>
>   <li>In the Test Navigator view, select the manual test suite to be executed.</li>
>   <li>Launch the selected test suite by creating a Test configuration in the Run dialog
>   	<ul>
>		<li>Under the Test tab select the test suite</li>
>		<li>Choose the results folder associated with the selected test suite and deployment platform<br>
>	<b>Note:</b> The name and directory structure containing the generated execution history file 
>	<b>MUST</b> be the same as it will be in CVS in order for generated reports to work correctly.</li>
>		<li>Click Run</li>
>		</ul>
>		</li>
>   <li>Launch a candidate workbench on the testing platform</li>
>   <li>Execute each test case as specified in the TPTP Manual Test View on the local host.</li>
>   <li>Record the result of the test case execution by capturing the status of the test case execution using the following explanation for TPTP Testing Tools Status options:
>	<table border="1" width="100%">
>		<tbody>
>			<tr>
>				<th align="left">TPTP Testing Tools Status</th>
>				<th align="left">Explanation</th>
>			</tr>
>			<tr>
>				<td>Error</td>
>				<td>Do not use.</td>
>			</tr>
>			<tr>
>				<td>Fail</td>
>				<td>The test case could be executed and failed due to a
>				blocking defect. <br>
>				</td>
>			</tr>
>			<tr>
>				<td>Inconclusive</td>
>				<td>The test case was not applicable OR the test case could be executed and failed due to a non-blocking defect.
>			A non-blocking defect is defined as a nice-to-have fix that does not degrade the core functionality being tested by the test case. 
>			</td>
>			</tr>
>			<tr>
>				<td>Pass</td>
>				<td>The test case could be executed and passed.</td>
>			</tr>
>		</tbody>
>		
>	</table>
>	For verdicts of Fail or Inconclusive please provide a defect number and explanation in the test log.</li>
></ol>
></div>
><br>
><div id="section_hlight"><a name="3.3">3.3 Executing JUnit and JUnit Plug-in Test Suites</a></div>
><div id="main_content">
>	<ol>
>		<li>Open the Test perspective.</li>
>   	<li>In the Test Navigator view, select the JUnit test suite to be executed.</li>
>   	<li>Launch the selected test suite by creating a Test configuration in the Run dialog
>   		<ul>
>				<li>Under the Test tab select the test suite</li>
>				<li>Select the deployment.<br>
>				To run a test remotely you will need to create and configure a new test deployment<br>
>				For JUnit plug-in tests you also may want to specify deployment on a different workbench location.</li>
>				<li>Choose the results folder associated with the selected test suite and deployment platform<br>
>				<b>Note: </b>The name and directory structure containing the generated execution history file MUST be the same as it will be in CVS in order for generated reports to work correctly.</li>
>				<li>Click Run</li>
>			</ul>
>		</li>
>   	<li>An execution history will be generated according to the following explanation of TPTP Testing Tools Status options:
>	<table border="1" width="100%">
>		<tbody>
>			<tr>
>				<th align="left">TPTP Testing Tools Status</th>
>				<th align="left">Explanation</th>
>			</tr>
>			<tr>
>				<td>Error</td>
>				<td>The test case was or could not be executed due to a run
>				time error.</td>
>			</tr>
>			<tr>
>				<td>Fail</td>
>				<td>The test case could be executed and failed due to a defect.</td>
>			</tr>
>			<tr>
>				<td>Pass</td>
>				<td>The test case could be executed and passed.</td>
>			</tr>
>		</tbody>
>	</table>
>	The Fail and Error verdict events in the test log will contain a stack trace that can jump to source.</li>
></ol>
></div>
><br>
><div id="section_hlight"><a name="3.4">3.4 Executing AGR Test Suites</a></div>
><div id="main_content">
>	<ol>
>		<li>Ensure that the Agent Controller is installed and started.</li>
>   <li>Open the Test perspective.</li>
>   <li>In the Test Navigator view, select the Automated GUI Test to be executed.</li>
>   <li>Execute the test suite by creating a launch configuration item.<br>
>		<b>Important:</b> Using the 'Quick Run' mode will not generate an execution history file. Make sure that a proper launch configuration is used.</li>
>   <li>The runner will generate an execution history according to the following explanation of TPTP Testing Tools Status options:
>   <table border="1" width="100%">
>		<tbody>
>			<tr>
>				<th align="left">TPTP Testing Tools Status</th>
>				<th align="left">Explanation</th>
>			</tr>
>			<tr>
>				<td>Error</td>
>				<td>The test case was or could not be executed due to a run
>				time error.</td>
>			</tr>
>			<tr>
>				<td>Fail</td>
>				<td>The test case could be executed and failed due to a defect.</td>
>			</tr>
>			<tr>
>				<td>Pass</td>
>				<td>The test case could be executed and passed.</td>
>			</tr>
>		</tbody>
>	</table>	
>	The Fail and Error verdict events in the test log will contain a stack trace that can jump to source.
>   	</li>
>	</ol>
>	
>	
></div>
><br>
><div id="section_hlight"><a name="4.0">4.0 Common Tasks</a></div>
><div id="main_content">
></div>
><br>
><div id="section_hlight"><a name="4.1">4.1 Creating Test Cases</a></div>
><div id="main_content">
>	<p>
>General Guidelines for test case creation:
></p>
>	<ol>
>   <li>Create short concise cases that cover identifiable partitions of functionality. </li>
>  This increases reporting accuracy and allows for easy reproduction by testers and fixers.</li>
>   <li>Comment test cases thoroughly in the description section of the including the manual steps for what the test case is performing.</li>
>   <li>If any resource (such as a datapool) needs to be modified before the test suite is executed, 
>   	then include that as part of the description of the test suite. 
>  Any other important information that the tester should be aware of needs to be included in the description of the test suite.</li>
>   <li>If an existing manual test case is being automated, then add '(DEPRECATED - Use <automated test suite name>)' 
>  to the beginning of the description of the manual test case.</li>
>   <li>For more guidance creating effective test cases please refer to the <a href=TPTP_Testing_Strategy_Part_2.html#2.0>Best Practices</a> section of Part 2.</li>
>  </ol>
></div>
><br>
><div id="section_hlight"><a name="4.2">4.2 Viewing Test Logs</a></div>
><div id="main_content">
>	<p>
>		To open a test log file, use the Open File window, or double-click in the Test Navigator the file with the extension, .execution.
>	</p>
>	<ol>
>   <li>From the File menu, click Open File.</li>
>   <li>Browse to and select the test log file to open. (Test log files use the extension, .execution.)</li>
>   <li>Click OK.</li>
>	</ol>
></div>
><br>
><div id="section_hlight"><a name="4.3">4.3 Submitting Execution Histories</a></div>
><div id="main_content">
>	<ol>
>		<li>Execution histories should be checked-in to the CVS branch for the release under test.  
>			They are checked into a results folder under the associated plugin for the type of test executed 
>			(manual_results, junit_results, junit_plugin_results, gui_results). To delineate between test executions on multiple platforms, 
>			each results folder contains subfolders for the supported platforms.
>			<br>
>			<b>For Example: </b>The results of executing the manual test 
>			<i>org.eclipse.hyades.use.cases/manual/Profiling_and_Logging/Platform.Communication.Control_Channel_Remote_Profiling.Windows_IA32.testsuite</i> 
>			should be generated to and checked into <i>org.eclipse.hyades.use.cases/manual_results/Profiling_and_Logging/Windows_XP</i>
>		</li>
>   <li>Testers who are not committers should email the execution history files to the committer for the associated component.
>			<br>
>			Include the following in your email:
>			<ul>
>				<li>Name of tester if not same as sender</li>
>				<li>Deployment platform(s) including name, version and release.</li>
>				<li>Execution specific information including name, version and release.</li>
>				<li>TPTP driver build id and test pass identifier.</li>
>			</ul>
>		</li>
>	</ol>
></div>
><br>
><div id="section_hlight"><a name="5.0">5.0 Help</a></div>
><div id="main_content">
>	<p>
>	<b>General Help</b><br><br>
>	TPTP General documentation can be found in the Eclipse Help under Help &gt;&gt; Help Contents &gt;&gt; Testing Applications<br>
>	<a href=http://www.eclipse.org/tptp/test/documents/gettingstarted/Automated_GUI_Recorder_Getting_Started.html>
>		Getting Started with TPTP's Automated GUI Recorder</a><br>
>	<a href=http://www.eclipse.org/tptp/test/documents/gettingstarted/Manual_Test_Tools_Getting_Started.html>
>		Getting Started with TPTP's Manual Test Tools</a><br>
>	<a href=http://www.eclipse.org/tptp/test/documents/gettingstarted/BIRT_Test_Reports_Getting_Started.html>
>	Getting Started with TPTP's Business Intelligence and Reporting Tools (BIRT) Test Reports</a><br>
></p>
><p>
>	<b>Creating Tests</b><br><br>
>	Manual test cases and suites: Eclipse Help &gt;&gt; Help Contents &gt;&gt;  Testing Applications &gt;&gt; Testing Manually<br>
>	TPTP JUnit: Eclipse Help &gt;&gt; Help Contents &gt;&gt;  Testing Applications &gt;&gt; Testing with JUnit<br>
>	TPTP JUnit Plug-in: Eclipse Help &gt;&gt; Help Contents &gt;&gt; Testing Applications &gt;&gt; Testing plug-ins with JUnit<br>
>	AGR: See the <a href=http://www.eclipse.org/tptp/test/documents/gettingstarted/Automated_GUI_Recorder_Getting_Started.html#3.0>
>		Documentation section of the AGR Getting Started Guide</a><br>
></p>
>	<p>
>	<b>More Resources</b><br><br>
>	<a href=http://www.eclipse.org/tptp/>Eclipse Test & Performance Tools Platform Project Web Site</a><br>
>	<a href=http://www.eclipse.org/tptp/home/documents/extenders.php>Documentation for TPTP extenders</a><br>
>	<a href=http://www.eclipse.org/tptp/home/project_info/general/mailnews.php>TPTP Mailing Lists and Newsgroups</a>
>	</p>
></div>
><br>
></body>
></html>

<!DOCTYPE html PUBLIC "-//w3c//dtd html 4.0 transitional//en">
<html><head><title>TPTP Testing Strategy Part 1 - Creating and Executing TPTP Tests</title>



<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<link rel="stylesheet" type="text/css" href="http://www.eclipse.org/tptp/eclipse_style.css">
<link rel="stylesheet" type="text/css" href="http://www.eclipse.org/tptp/home/documents/process/TPTP_Testing_Strategy_files/strategy_doc_style.css">
<script type="text/javascript">
		function expandOrCollapse()
		{
			element = document.getElementById("invisible");
			if (element != null)
			{
				element.id = "dynamicAppearance";
				document.getElementById("changeHistoryLink").lastChild.nodeValue = "Hide the Change History";
				return;
			}

			element = document.getElementById("dynamicAppearance");
			if (element != null)
			{
				element.id = "invisible";
				document.getElementById("changeHistoryLink").lastChild.nodeValue = "Show the Change History";
			}
		}
  </script></head><body>
<div id="page_header">Eclipse Test and Performance Tools Platform
Project</div>
<div id="page_sub_header">TPTP Testing Strategy Part 1 - Creating and Executing TPTP Tests</div>
<div id="section_header">TPTP Testing Strategy Part 1 - Creating and Executing TPTP Tests</div>

<div class="content"><!-- Main content goes here -->
<table id="main_content">
	<tbody>
		<tr>
			<td><b>Title:</b></td>
			<td>TPTP Testing Strategy</td>
		</tr>
		<tr>
			<td><b>Author:</b></td>

			<td>Alan Haggarty (haggarty@ca.ibm.com)</td>
		</tr>
	</tbody>
</table>
<a href="javascript:expandOrCollapse()"><b><span id="changeHistoryLink">Show the Change History</span></b></a>
<table id="invisible" border="1">
	<tbody>
		<tr>
			<th align="left" bgcolor="#dfdfdf" width="10%">Name</th>

			<th align="left" bgcolor="#dfdfdf" width="20%">Date</th>
			<th align="left" bgcolor="#dfdfdf" width="30%">Modified Sections
			</th>
		</tr>
		<tr>
			<td>Alan Haggarty (haggarty@ca.ibm.com)</td>
			<td>June 4, 2008</td>
			<td>Initial Creation</td>

		</tr>
	</tbody>
</table>  	
<br>
<div id="section_hlight">Table of Contents</div>
<div id="main_content">
<a href="#1.0">1.0 Preparation</a><br>
<a href="#1.1">1.1 Install the TPTP Testing Tools</a><br>
<a href="#1.2">1.2 Extracting the Test Resources</a><br>
<a href="#1.3">1.3 Structure of the Test Resources</a><br>
<a href="#2.0">2.0 Creating New TPTP Tests</a><br>
<a href="#3.0">3.0 Executing Tests</a><br>
<a href="#3.1">3.1 Creating Test Deployments</a><br>
<a href="#3.2">3.2 Executing Manual Test Suites</a><br>
<a href="#3.3">3.3 Executing JUnit and JUnit Plug-in Test Suites</a><br>
<a href="#3.4">3.4 Executing AGR Test Suites</a><br>
<a href="#4.0">4.0 Common Tasks</a><br>
<a href="#4.1">4.1 Creating Test Cases</a><br>
<a href="#4.2">4.2 Viewing Test Logs</a><br>
<a href="#4.3">4.3 Submitting Execution Histories</a><br>
<a href="#5.0">5.0 Help</a><br>
</div>
<br>
<div id="section_hlight"><a name="1.0">1.0 Preparation</a></div>
<div id="main_content">
</div>
<br>
<div id="section_hlight"><a name="1.1">1.1 Install the TPTP Testing Tools</a></div>
<div id="main_content">
	<ol>
		<li>Open the <a href=http://www.eclipse.org/tptp/home/downloads>TPTP download site.</a></li>
	  <li>Select the release of TPTP to be tested and download and unzip the TPTP plugins as well as the requirements listed 
	  	in the TPTP Plugins for Eclipse section.</li>
	  <li>Download and install the Manual Test Tools ("As Is Components" section).</li>
	  <li>Download and install the BIRT Test Reports ("As Is Components" section).</li>
	  <li>Download and install Automated GUI Recording (As Is Components section).
	         <br><i>Note: Remember to restart eclipse with the clean option after installing the AGR</i></li>
	  <li>If remote execution is required download and unzip the Agent Controller (Agent Controller section).</li>
	  <li>The Installation Guide and Release Notes are located in the Documentation section.</li>
	</ol>
</div>
<br>
<div id="section_hlight"><a name="1.2">1.2 Extracting the Test Resources</a></div>
<div id="main_content">
	<p>Test suites, cases, deployment configurations and execution histories are stored in the TPTP CVS repository 
		(dev.eclipse.org:/cvsroot/tptp  or www.eclipse.org/tptp  &gt;&gt; CVS Repository), under the test-results  module.</p>
	<p>The test-results module contains subfolders for each of the four subprojects - monitoring, platform, test and trace
		 - in the TPTP project. Each subfolder contains the test projects for that subproject.</p>
	<ol>
		 <li>Check out the individual test projects under test-results/monitoring/, test-results/platform/, test-results/test/, or test-results/trace/ to the local workspace.
						<br><b>For Example: </b><a href=http://dev.eclipse.org/viewcvs/index.cgi/test-results/platform/org.eclipse.hyades.use.cases/?root=TPTP_Project>test-results/platform/org.eclipse.hyades.use.cases</a></li>

   	<li>To ensure that the local workspace contains the latest test projects, update at the beginning of each test pass and report generation.</li>
	</ol>
</div>
<br>
<div id="section_hlight"><a name="1.3">1.3 Structure of the Test Resources</a></div>
<div id="main_content">
	<p>Test projects must follow a defined structure.
	</p>
	<p>
	Please see Part 2 for the required structure.<br>
	<a href=TPTP_Testing_Strategy_Part_2.html#1.2>Test Project Detailed Structure</a><br>
	<a href=TPTP_Testing_Strategy_Part_2.html#2.0>Test Creation Best Practices</a><br>
</p>
</div>
<br>
<div id="section_hlight"><a name="2.0">2.0 Creating New TPTP Tests</a></div>
<div id="main_content">
	<p>To create new TPTP Manual, JUnit, JUnit Plug-in and AGR test suites and cases please follow the testing documentation
		for each type of test. The location of the documentation can be found in the <a href="#5.0">Help section</a> of this document.
	</p>
	<p>The <a href=http://www.eclipse.org/tptp/home/documents/process/TPTP_Manual_Test_Case_Generator.html>TPTP Manual Test Case Generator</a>
		 may be used to generate the structure of manual test case descriptions based on the standardized structured manual test case format.
	</p> 
</div>
<br>
<div id="section_hlight"><a name="3.0">3.0 Executing Tests</a></div>
<div id="main_content">
</div>
<br>
<div id="section_hlight"><a name="3.1">3.1 Creating Test Deployments</a></div>
<div id="main_content">
	<p>
	Creating Test Deployments: Detailed steps for creating test deployments can be found in Eclipse Help &gt;&gt; Help Contents &gt;&gt; Testing Applications &gt;&gt; Creating a Test Deployment
	</p>
	<p>Create a Test Launch Configuration</p>
	<ol>
   <li>Create a new launch configuration by selecting the entry Test from the list of configurations in Run &gt;&gt; Run Configurations</li>
   <li>Use the Select test to run pane to navigate to and select the test suite you wish to run.</li>
   <li>Select the deployment for running the test suite. By default the test will be deployed locally. Manual tests are always run in the local deployment.</li>
   <li>In the Test Logs tab override the default log configuration to match the desired log directory (for example plugin_results). 
   	Please refer to the <a href=TPTP_Testing_Strategy_Part_2.html#1.1>Naming Conventions</a> section of Part 2 for the names of result directories.</li>
  </ol>
</div>
<br>
<div id="section_hlight"><a name="3.2">3.2 Executing Manual Test Suites</a></div>
<div id="main_content">
	<ol>
	   <li>Open the Test perspective.</li>
   <li>In the Test Navigator view, select the manual test suite to be executed.</li>
   <li>Launch the selected test suite by creating a Test configuration in the Run dialog
   	<ul>
		<li>Under the Test tab select the test suite</li>
		<li>Choose the results folder associated with the selected test suite and deployment platform<br>
	<b>Note:</b> The name and directory structure containing the generated execution history file 
	<b>MUST</b> be the same as it will be in CVS in order for generated reports to work correctly.</li>
		<li>Click Run</li>
		</ul>
		</li>
   <li>Launch a candidate workbench on the testing platform</li>
   <li>Execute each test case as specified in the TPTP Manual Test View on the local host.</li>
   <li>Record the result of the test case execution by capturing the status of the test case execution using the following explanation for TPTP Testing Tools Status options:
	<table border="1" width="100%">
		<tbody>
			<tr>
				<th align="left">TPTP Testing Tools Status</th>
				<th align="left">Explanation</th>
			</tr>
			<tr>
				<td>Error</td>
				<td>Do not use.</td>
			</tr>
			<tr>
				<td>Fail</td>
				<td>The test case could be executed and failed due to a
				blocking defect. <br>
				</td>
			</tr>
			<tr>
				<td>Inconclusive</td>
				<td>The test case was not applicable OR the test case could be executed and failed due to a non-blocking defect.
			A non-blocking defect is defined as a nice-to-have fix that does not degrade the core functionality being tested by the test case. 
			</td>
			</tr>
			<tr>
				<td>Pass</td>
				<td>The test case could be executed and passed.</td>
			</tr>
		</tbody>
		
	</table>
	For verdicts of Fail or Inconclusive please provide a defect number and explanation in the test log.</li>
</ol>
</div>
<br>
<div id="section_hlight"><a name="3.3">3.3 Executing JUnit and JUnit Plug-in Test Suites</a></div>
<div id="main_content">
	<ol>
		<li>Open the Test perspective.</li>
   	<li>In the Test Navigator view, select the JUnit test suite to be executed.</li>
   	<li>Launch the selected test suite by creating a Test configuration in the Run dialog
   		<ul>
				<li>Under the Test tab select the test suite</li>
				<li>Select the deployment.<br>
				To run a test remotely you will need to create and configure a new test deployment<br>
				For JUnit plug-in tests you also may want to specify deployment on a different workbench location.</li>
				<li>Choose the results folder associated with the selected test suite and deployment platform<br>
				<b>Note: </b>The name and directory structure containing the generated execution history file MUST be the same as it will be in CVS in order for generated reports to work correctly.</li>
				<li>Click Run</li>
			</ul>
		</li>
   	<li>An execution history will be generated according to the following explanation of TPTP Testing Tools Status options:
	<table border="1" width="100%">
		<tbody>
			<tr>
				<th align="left">TPTP Testing Tools Status</th>
				<th align="left">Explanation</th>
			</tr>
			<tr>
				<td>Error</td>
				<td>The test case was or could not be executed due to a run
				time error.</td>
			</tr>
			<tr>
				<td>Fail</td>
				<td>The test case could be executed and failed due to a defect.</td>
			</tr>
			<tr>
				<td>Pass</td>
				<td>The test case could be executed and passed.</td>
			</tr>
		</tbody>
	</table>
	The Fail and Error verdict events in the test log will contain a stack trace that can jump to source.</li>
</ol>
</div>
<br>
<div id="section_hlight"><a name="3.4">3.4 Executing AGR Test Suites</a></div>
<div id="main_content">
	<ol>
		<li>Ensure that the Agent Controller is installed and started.</li>
   <li>Open the Test perspective.</li>
   <li>In the Test Navigator view, select the Automated GUI Test to be executed.</li>
   <li>Execute the test suite by creating a launch configuration item.<br>
		<b>Important:</b> Using the 'Quick Run' mode will not generate an execution history file. Make sure that a proper launch configuration is used.</li>
   <li>The runner will generate an execution history according to the following explanation of TPTP Testing Tools Status options:
   <table border="1" width="100%">
		<tbody>
			<tr>
				<th align="left">TPTP Testing Tools Status</th>
				<th align="left">Explanation</th>
			</tr>
			<tr>
				<td>Error</td>
				<td>The test case was or could not be executed due to a run
				time error.</td>
			</tr>
			<tr>
				<td>Fail</td>
				<td>The test case could be executed and failed due to a defect.</td>
			</tr>
			<tr>
				<td>Pass</td>
				<td>The test case could be executed and passed.</td>
			</tr>
		</tbody>
	</table>	
	The Fail and Error verdict events in the test log will contain a stack trace that can jump to source.
   	</li>
	</ol>
	
	
</div>
<br>
<div id="section_hlight"><a name="4.0">4.0 Common Tasks</a></div>
<div id="main_content">
</div>
<br>
<div id="section_hlight"><a name="4.1">4.1 Creating Test Cases</a></div>
<div id="main_content">
	<p>
General Guidelines for test case creation:
</p>
	<ol>
   <li>Create short concise cases that cover identifiable partitions of functionality. </li>
  This increases reporting accuracy and allows for easy reproduction by testers and fixers.</li>
   <li>Comment test cases thoroughly in the description section of the including the manual steps for what the test case is performing.</li>
   <li>If any resource (such as a datapool) needs to be modified before the test suite is executed, 
   	then include that as part of the description of the test suite. 
  Any other important information that the tester should be aware of needs to be included in the description of the test suite.</li>
   <li>If an existing manual test case is being automated, then add '(DEPRECATED - Use <automated test suite name>)' 
  to the beginning of the description of the manual test case.</li>
   <li>For more guidance creating effective test cases please refer to the <a href=TPTP_Testing_Strategy_Part_2.html#2.0>Best Practices</a> section of Part 2.</li>
  </ol>
</div>
<br>
<div id="section_hlight"><a name="4.2">4.2 Viewing Test Logs</a></div>
<div id="main_content">
	<p>
		To open a test log file, use the Open File window, or double-click in the Test Navigator the file with the extension, .execution.
	</p>
	<ol>
   <li>From the File menu, click Open File.</li>
   <li>Browse to and select the test log file to open. (Test log files use the extension, .execution.)</li>
   <li>Click OK.</li>
	</ol>
</div>
<br>
<div id="section_hlight"><a name="4.3">4.3 Submitting Execution Histories</a></div>
<div id="main_content">
	<ol>
		<li>Execution histories should be checked-in to the CVS branch for the release under test.  
			They are checked into a results folder under the associated plugin for the type of test executed 
			(manual_results, junit_results, junit_plugin_results, gui_results). To delineate between test executions on multiple platforms, 
			each results folder contains subfolders for the supported platforms.
			<br>
			<b>For Example: </b>The results of executing the manual test 
			<i>org.eclipse.hyades.use.cases/manual/Profiling_and_Logging/Platform.Communication.Control_Channel_Remote_Profiling.Windows_IA32.testsuite</i> 
			should be generated to and checked into <i>org.eclipse.hyades.use.cases/manual_results/Profiling_and_Logging/Windows_XP</i>
		</li>
   <li>Testers who are not committers should email the execution history files to the committer for the associated component.
			<br>
			Include the following in your email:
			<ul>
				<li>Name of tester if not same as sender</li>
				<li>Deployment platform(s) including name, version and release.</li>
				<li>Execution specific information including name, version and release.</li>
				<li>TPTP driver build id and test pass identifier.</li>
			</ul>
		</li>
	</ol>
</div>
<br>
<div id="section_hlight"><a name="5.0">5.0 Help</a></div>
<div id="main_content">
	<p>
	<b>General Help</b><br><br>
	TPTP General documentation can be found in the Eclipse Help under Help &gt;&gt; Help Contents &gt;&gt; Testing Applications<br>
	<a href=http://www.eclipse.org/tptp/test/documents/gettingstarted/Automated_GUI_Recorder_Getting_Started.html>
		Getting Started with TPTP's Automated GUI Recorder</a><br>
	<a href=http://www.eclipse.org/tptp/test/documents/gettingstarted/Manual_Test_Tools_Getting_Started.html>
		Getting Started with TPTP's Manual Test Tools</a><br>
	<a href=http://www.eclipse.org/tptp/test/documents/gettingstarted/BIRT_Test_Reports_Getting_Started.html>
	Getting Started with TPTP's Business Intelligence and Reporting Tools (BIRT) Test Reports</a><br>
</p>
<p>
	<b>Creating Tests</b><br><br>
	Manual test cases and suites: Eclipse Help &gt;&gt; Help Contents &gt;&gt;  Testing Applications &gt;&gt; Testing Manually<br>
	TPTP JUnit: Eclipse Help &gt;&gt; Help Contents &gt;&gt;  Testing Applications &gt;&gt; Testing with JUnit<br>
	TPTP JUnit Plug-in: Eclipse Help &gt;&gt; Help Contents &gt;&gt; Testing Applications &gt;&gt; Testing plug-ins with JUnit<br>
	AGR: See the <a href=http://www.eclipse.org/tptp/test/documents/gettingstarted/Automated_GUI_Recorder_Getting_Started.html#3.0>
		Documentation section of the AGR Getting Started Guide</a><br>
</p>
	<p>
	<b>More Resources</b><br><br>
	<a href=http://www.eclipse.org/tptp/>Eclipse Test & Performance Tools Platform Project Web Site</a><br>
	<a href=http://www.eclipse.org/tptp/home/documents/extenders.php>Documentation for TPTP extenders</a><br>
	<a href=http://www.eclipse.org/tptp/home/project_info/general/mailnews.php>TPTP Mailing Lists and Newsgroups</a>
	</p>
</div>
<br>
</body>
</html>

Actions: View

Attachments on bug 211777: 102509 | 102510 | 104770 | 104771 | 105529 | 105532