Bugzilla – Full Text Bug Listing
| Summary: | TPTP report generation [automation] | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | z_Archived | Reporter: | Sheldon Lee-Loy <sleeloy> | ||||||
| Component: | TPTP | Assignee: | Joel Cayne <jcayne> | ||||||
| Status: | CLOSED FIXED | QA Contact: | |||||||
| Severity: | enhancement | ||||||||
| Priority: | P1 | CC: | jkubasta, paulslau | ||||||
| Version: | unspecified | Keywords: | plan | ||||||
| Target Milestone: | --- | ||||||||
| Hardware: | All | ||||||||
| OS: | All | ||||||||
| URL: | http://www.eclipse.org/tptp/groups/Architecture/documents/features/hf_174643.html | ||||||||
| Whiteboard: | closed460 | ||||||||
| Attachments: |
|
||||||||
Created attachment 59271 [details]
ant script
*** Bug 173662 has been marked as a duplicate of this bug. *** Created attachment 80404 [details]
Feature description document
Approved by the AG for TPTP 4.5 with the following comments: -The sizing should include 1 - 2 PDs for documentation to document the mechanics for reuse by our users/consumers. -This needs to be done early in 4.5 (i2). -This enhancement should be coordinated with the test automation initiative. As part of this initiative, we are using ASF to automate our test execution and ASF should be used for automating report generation as well. -We need a way to turn off this service when we are not in a test pass. Note, we need to generate reports from a workspace containing all the test plugins (not just containing the test-results module). The scripts have been checked in for report automation under platform\org.eclipse.tptp.platform.releng.tools\org\eclipse\tptp\platform\releng\tools\testautomation The readme.html inside the project describes how to use the report automation, including how to schedule it during a test pass. The scripts in use are common with the test automation (https://bugs.eclipse.org/bugs/show_bug.cgi?id=211752). As of TPTP 4.6.0, TPTP is in maintenance mode and focusing on improving quality by resolving relevant enhancements/defects and increasing test coverage through test creation, automation, Build Verification Tests (BVTs), and expanded run-time execution. As part of the TPTP Bugzilla housecleaning process (see http://wiki.eclipse.org/Bugzilla_Housecleaning_Processes), this enhancement/defect is verified/closed by the Project Lead since this originator of this enhancement/defect has an inactive Bugzilla account and considered to be fixed. If this enhancement/defect is still unresolved and reproducible in the latest TPTP release (http://www.eclipse.org/tptp/home/downloads/), please re-open. |
As we discussed I want to automate the report generation as a cron job. Preferably scheduled after the nightly builds. As I said the report generation can run from the command line using an ant script. I will attach the antscript to this bugzilla. Here is a sample command that shows how to use the ant script; ant -f buildTabular.xml -Dtptp.driver=D:\development\TPTP-4.3.0-200611160100A\eclipse -Dworkspace=D:\TPTP31_2Reports -DoutFolder=D:/TPTP31_2Reports/report/4.4 -Dworkspace.project=test-results -Dtestsuite.list=platform\org.eclipse.hyades.tests\*.testsuite -Dstarttime="01/15/2007 1:47 PM" where: tptp.driver is the location of the tptp driver with BIRT installed workspace is the location of the workspace that has the test-result cvs module checked out outFolder is the location where the generated reports should be stored workspace.project is the project that contains the test execution history. NOTE: you don't need to change this parameter testsuite.list is the testsuite list that the reports should contain. NOTE: you don't need to change this paramter. starttime is the time that tells the report generator that it should only report on test execution histories that have been created after this time. Note this is usually the time the test pass starts. Notes: The tptp.driver should be a driver that has BIRT installed. You should only install this driver ONCE on the machine. You don't need to install the driver after each iteration. The workspace directory should contain the following CVS module CVS repository: :extssh:sleeloy@dev.eclipse.org:/cvsroot/tptp module: test-results Note: depending on the driver that we are testing you need to check our the appropriate branch. For example if we're testing 4.3.1 you need to check out the 4.3.1 branch for the above module. The outFolder contain be the location where the generated reports will be published. You should check out the following CVS module. This module contains the published reports. CVS repository: :extssh:sleeloy@dev.eclipse.org:/cvsroot/tptp module: www/tptp/test/reports Note: depending on the driver that we are testing you need to check our the appropriate branch. For example if we're testing 4.3.1 you need to check out the 4.3.1 branch for the above module. Also depending on the branch you need to create the appropriate sub folder. For example you should create a "4.4" sub folder for the 4.4 generated reports and a "4.3.1" sub folder for the 4.3.1 generate reports. Once the reports are generated they have to be checked into CVS. It usually takes an hour and half to generate the reports. For testing purposes you should make the start time close to the current time so that it does not load all the test execution histories in your workspace.