Bugzilla – Full Text Bug Listing
| Summary: | Contribute the automated test suites | ||
|---|---|---|---|
| Product: | [Modeling] Sirius | Reporter: | Pierre-Charles David <pierre-charles.david> |
| Component: | Releng | Assignee: | Pierre-Charles David <pierre-charles.david> |
| Status: | CLOSED FIXED | QA Contact: | |
| Severity: | major | ||
| Priority: | P3 | CC: | cedric.brun, esteban.dugueperoux |
| Version: | 1.0.0 | Keywords: | triaged |
| Target Milestone: | 2.0.0 | ||
| Hardware: | All | ||
| OS: | All | ||
| Whiteboard: | |||
| Bug Depends on: | |||
| Bug Blocks: | 423300, 452318 | ||
|
Description
Pierre-Charles David
The CQ has been created, and Modeling PMC has voted +1. We're waiting for feedback from the IP team. The CQ has been accepted. I'll commit the initial version of the code soon, but it will be a while before we can actually switch, as in addition to the code itself we will need to adjust the builds and Hudson jobs. The version of the code reviewed in the CQ has been commited as b64b2b4c0060329a68b77c0734c82b3eca24529b. Commit 5e50463c35ffb502f436c31c48e33757b91da914 updates the code to integrate the latest changes after the snapshot sent in the CQ. We will need to make some adjustements in terms of naming and packaging of features. Currently we have a single org.eclipse.sirius.tests feature, which only includes the test support API plug-ins. We probably want to keep the possibility to install just these plug-ins without the tests themselves (which are relatively big) for projects who just want to use our APIs to test their own modelers. The current situation and issues: * the org.eclipse.sirius.tests feature contains the org.eclipse.sirius.tests.support and org.eclipse.sirius.tests.swtbot.support plug-ins. * the org.eclipse.sirius.samples feature contains only org.eclipse.sirius.sample.ecore.design, but that plug-in should be "downgraded" as a sample only used by automated tests, and renamed into org.eclipse.sirius.tests.sample.ecore.design (see #441599) * the org.eclipse.sirius.sample.interactions.* plug-ins are currently part of a org.eclipse.sirius.sample.interactions.feature feature, and were planned to be contributed as a full-blown example (see #423300), but they still depend on the legacy query language (as do most of our tests), so they will need to be downgraded too. * there is currently a name clash between the org.eclipse.sirius.tests feature (which provides the APIs) and the org.eclipse.sirius.tests plug-in (which provides the main JUnit test suite) Proposal: * rename the current org.eclipse.sirius.tests feature into org.eclipse.sirius.tests.support, with the same content. This makes it clearer that it does not contain the tests themselves, but support code and APIs to help writing tests. * remove org.eclipse.sirius.sample.ecore.design from the org.eclipse.sirius.samples feature (which would become empty, at least temporarily), and rename it into org.eclipse.sirius.tests.sample.ecore.design * remove the org.eclipse.sirius.sample.interactions.feature feature, and rename the org.eclipse.sirius.sample.interactions.* plug-ins it contains into org.eclipse.sirius.tests.sample.interactions.* * rename the org.eclipse.sirius.tests plug-in into org.eclipse.sirius.tests.unit * add a new feature named org.eclipse.sirius.tests (now that that name if free again) which would contain all the test plug-ins themselves (oes.tests.unit, oes.tests.tree, oes.tests.swtbot as of now) and all the sample metamodels and modelers needed to run them (which would all be in the org.eclipse.sirius.tests.sample namespace). The feature would also depend on oes.query.legacy, and UML2 (for example), which are also needed for the tests. Looks good to me. Should we assume once this is done we'll have a hard time merging back the tests changes until we effectively migrated on the Eclipse infrastructure ? (In reply to Cedric Brun from comment #5) > Looks good to me. > Should we assume once this is done we'll have a hard time merging back the > tests changes until we effectively migrated on the Eclipse infrastructure ? I propose this ordering: 1. At Eclipse: rename the current org.eclipse.sirius.tests feature into org.eclipse.sirius.tests.support, with the same content. Build and publish a version with the new feature and adapt our internal builds to it. 2. Rename our internal version of org.eclipse.sirius.tests plug-in into org.eclipse.sirius.tests.unit, make sure everything still runs fine. 3. At Eclipse: create the new feature named org.eclipse.sirius.tests, and put it it all the sample metamodels and modelers, with their current names (this includes the interactions and ecore modeler). Once a build with this new feature is published, remove all the corresponding plug-ins which we have internally. At this point the only thing we still have internally are the 3 plug-ins with the actual test code, and a feature which aggregates them but only exists for us. 4. Rename/rebrand the org.eclipse.sirius.sample.ecore.design (now contributed by the oes.tests feature) into the correct namespace, and perform the adjustments needed in our internal version of the test code. 5. Rename/rebrand the interactions plug-ins (now contributed by the oes.tests feature) into the correct namespace, and perform the adjustments needed in our internal version of the test code. 6. Update the 3 plug-ins with the test code at Eclipse with the latest content from our internal version, add them to the org.eclipse.sirius.tests feature, and remove the version from our internal repo. After discussing with Laurent, he proposes org.eclipse.sirius.tests.junit instead of just .unit for the non-SWTbot tests. That's OK for me (the .swtbot one is already specific to a particular test framework). He also mentions one thing to be aware of: at least in old versions of Tycho, it was not possible to execute tests which were entirely built elsewhere. If this is still true, and given that we can not rely only on the Eclipse HIPP to execute our tests (they are too resource-hungry), we may need to keep shim plug-ins internally with fake test suites that delegate to the Sirius ones just to keep Tycho happy with something to compile. Our tests (and some of the sample modelers used by them) depend on oes.query.legacy, which is published separately, and itself depend on sirius. This means we can not craft a Target Platform which includes this without having Sirius itself in the target platform. The new plan is to have two separate builds: The first one, equivalent to the current one, which build Sirius itself and the test support APIs, and produces an update-site will all of these. The second one will use a different target platform, which will include the result of the first build (either from a published location or from a local folder build in a first step of a CI job) and oes.query.legacy. It will build the tests dependencies (all the sample modelers and metamodels), build the tests themselves, and execute the tests. It will also publish an update-site with all the tests and their dependencies, but at a separate location. This is needed for downstream projects who reuse the sample modelers and metamodels, or even reuse and extend the tests themselves for their own tests. At a later point, the sample modelers and metamodels used by the tests could be built and published as part of the first build, once the only dependency to oes.query.legacy is removed (it is in the interactions modeler). They could be published in a feature named org.eclipse.sirius.tests.samples. This is mostly done now. All that remains is the creation of the appropriate CI job(s) and their integration into the workflow (in particular with Gerrit). See bug #445371 for these aspects. For now the job at https://hudson.eclipse.org/sirius/job/sirius-tests-master/ works but mostly as a proof of concept. There are some "leftover" tasks which have been postponed for now: * the oes.tests plug-in has been renamed into oes.tests.junit, but the Java packages inside have not been renamed and so do not match the new namespace. * the sample Ecore modeler and Interactions modeler and metamodel, which are now published as part of the tests feature, have not been rebranding into the correct oes.tests.sample namespace. Neither of these is critical for the 2.0.0 release, especially since all the concerned plug-ins are now published into their own separate p2 repository, with little risk for normal users to ever see or install them. I'm leaving the ticket open, and if time permits we will try to do these leftovers before 2.0, but if we can not this ticket will be closed and new ones will be created for the remaining tasks. (In reply to Pierre-Charles David from comment #9) > This is mostly done now. All that remains is the creation of the appropriate > CI job(s) and their integration into the workflow (in particular with > Gerrit). See bug #445371 for these aspects. For now the job at > https://hudson.eclipse.org/sirius/job/sirius-tests-master/ works but mostly > as a proof of concept. The job has been renamed, it is now https://hudson.eclipse.org/sirius/job/sirius.tests-master/ It works fine, except that its support for the "quick" mode (where we skip tests which are too long and/or known to be unreliable) is broken. (In reply to Pierre-Charles David from comment #9) > There are some "leftover" tasks which have been postponed for now: > * the oes.tests plug-in has been renamed into oes.tests.junit, but the Java > packages inside have not been renamed and so do not match the new namespace. See bug #447026. > * the sample Ecore modeler and Interactions modeler and metamodel, which are > now published as part of the tests feature, have not been rebranding into > the correct oes.tests.sample namespace. See bug #447028. > Neither of these is critical for the 2.0.0 release, especially since all the > concerned plug-ins are now published into their own separate p2 repository, > with little risk for normal users to ever see or install them. > > I'm leaving the ticket open, and if time permits we will try to do these > leftovers before 2.0, but if we can not this ticket will be closed and new > ones will be created for the remaining tasks. Given the time remaining until the release, I'm closing this one. The two leftover tickets mentioned above can be treated at a later time. Available in Sirius 2.0.0. |