Community
Participate
Working Groups
We need to make it easier for users to run AGR test suites against plug-ins that are included under a workspace. There is a much bigger enhancement open to handle this for the local/remote scenario: https://bugs.eclipse.org/bugs/show_bug.cgi?id=109880 but because of the increasing demands from AGR users I'm opening this defect so that we can provide a robust method for the local scenario. When the user creates a launch configuration of type 'Test' and selects an AGR test suite, then we need to display a plug-in tab similar to that shown when launch configs of type "Eclipse Application" are created. Users should be able to select the plug-ins that they wish to include as part of their context workbench, thus allowing them to run functional tests against plug-ins that are under development.
I think this would help any test case that requires the workbench to run the SUT/test, for example TPTP JUnit Plugin test cases. I also think this as a feature rather than a bug. I see more than just the plugin list, you may want to have all the other tabs (take a look at "JUnit Plug-in Test" launch configuration). If required you could split the work in multiple features and do it incrementaly.
This will be deferred in 4.4.
This defect will be used to commit the work for the local scenario of defect 109880 (e.g. plug-in tab in test launch configuration page). This plug-in tab should be exposed for both AGR and JUnit plug-in tests via an extension point so users may enable this tab for their custom test type.
*** Bug 165727 has been marked as a duplicate of this bug. ***
Created attachment 55045 [details] Allow the execution of AGR test suites from a runtime workbench This patch is based off of the 4.2.1 branch of TPTP AGR. It allows you to run a runtime workbench with your test suite in one of the plugins in your workspace and have the runtime workbench immediately execute your test suite. All that are required are three VM arguments: -Dtptp.automated.gui=true -Dtptp.automated.gui.testsuite.plugin=<plugin id that has your test suite in it> -Dtptp.automated.gui.testsuite.path=/<path to the test suite within the plugin> There are some nice advantages to this patch: 1) No agent controller setup is required 2) All of the plugins in my development workbench are loaded by the PDE runtime of the runtime workbench in the standard way (allow me to use AGR test suites to verify that code that I have not yet delivered) 3) Debugging verification point code, resolvers and TPTP AGR itself is as simple as debugging any other plugin in your workspace 4) Adding a new resolver doesn't require that you deploy it into your runtime 5) Changes to the AGR code itself doesn't require that you deploy it into your runtime There are some limitations to the patch: 1) It doesn't support loops in the behavior section of the test suite 2) It doesn't produce a nice execution file with all of the results of the execution 3) It hasn't been thoroughly tested
Created attachment 55046 [details] Allow the execution of AGR test suites from a runtime workbench This patch is based off of the 4.2.1 branch of TPTP AGR. It allows you to run a runtime workbench with your test suite in one of the plugins in your workspace and have the runtime workbench immediately execute your test suite. All that are required are three VM arguments: -Dtptp.automated.gui=true -Dtptp.automated.gui.testsuite.plugin=<plugin id that has your test suite in it> -Dtptp.automated.gui.testsuite.path=/<path to the test suite within the plugin> There are some nice advantages to this patch: 1) No agent controller setup is required 2) All of the plugins in my development workbench are loaded by the PDE runtime of the runtime workbench in the standard way (allow me to use AGR test suites to verify that code that I have not yet delivered) 3) Debugging verification point code, resolvers and TPTP AGR itself is as simple as debugging any other plugin in your workspace 4) Adding a new resolver doesn't require that you deploy it into your runtime 5) Changes to the AGR code itself doesn't require that you deploy it into your runtime There are some limitations to the patch: 1) It doesn't support loops in the behavior section of the test suite 2) It doesn't produce a nice execution file with all of the results of the execution 3) It hasn't been thoroughly tested
Comment on attachment 55045 [details] Allow the execution of AGR test suites from a runtime workbench Submitted twice by accident
Setting to P2 for 4.4.
Updating original estimate based information provided by Liz.
Targeting to future since cannot be contained in 4.4.
As discussed on this week's Test Project call (January 22, 2007), the Test Project will focus on existing P1 - P2/Blocker - Major and P1/Normal and test creation/automation defects (omitting defects dependant on outstanding features) in TPTP 4.4. All other Test Project defects have been targeted to future. If this defect has been targeted to future and you/originator feel it should be completed in 4.4, please provide the necessary reason as a reply to this comment or a post to the Test Project mailing list (tptp-test-tooling-dev@eclipse.org). We will collectively triage and assess our resources to determine a case-by-case decision.
Correction: The Test Project mailing list is tptp-testing-tools-dev@eclipse.org.
Created attachment 60186 [details] Figures 1 Awesome job Govin and Kim. I've beatified figure 1 (please see the attached zip file for a .png). I've also included the document file if you need to make more changes. Here are some suggestions: 1) "overview" and "workflow" are one word. You've spelled them as "over view" and "work flow". 2) Problems 1, 2, and 3 have already been fixed. I recommend removing them from the section "Some Known Issues with AGR" Thanks, Ali M.
(In reply to comment #15) I believe this comment was intended for defect https://bugs.eclipse.org/bugs/show_bug.cgi?id=151249.
Reassigning to Project Lead until new Component Lead is assigned.
In TPTP 4.5, the AGR was moved from a Technology Preview component to an As-Is component. As-Is components are primarily provided for prior users but imply no support (for example, defects, news group, and mailing lists) or commitment to triage or resolve opened defects. For this defect to be considered, please re-open with an attached patch including code to resolve the symptom and test cases to test the fix.
Closing by default since not closed by the originator in the 7+ months since being resolved. Please reopen if the issue is still present in the latest TPTP release or the resolution is not correct.
