Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 458024

Summary: Introduce support for an alternative implementation of the Acceleo Query Language
Product: [Modeling] Sirius Reporter: Cedric Brun <cedric.brun>
Component: CoreAssignee: Cedric Brun <cedric.brun>
Status: CLOSED FIXED QA Contact: Pierre-Charles David <pierre-charles.david>
Severity: enhancement    
Priority: P3 CC: laurent.goubet, laurent.redor, philippe.dul, pierre-charles.david
Version: 2.0.0Keywords: triaged
Target Milestone: 3.0.0M6   
Hardware: All   
OS: All   
See Also: https://git.eclipse.org/r/40055
https://git.eclipse.org/r/41455
https://git.eclipse.org/c/sirius/org.eclipse.sirius.git/commit/?id=fbf8869240014b6cebf5c94d05eba12eda1adbe5
https://git.eclipse.org/r/43726
https://git.eclipse.org/r/44113
https://git.eclipse.org/c/sirius/org.eclipse.sirius.git/commit/?id=7b2b4a07e00a1e9d2c5dfc91c645b3949deb5091
https://git.eclipse.org/r/47046
https://git.eclipse.org/c/sirius/org.eclipse.sirius.git/commit/?id=626daee13e2eec2f8652d8b36da1df0734f40dd9
Whiteboard:
Bug Depends on: 459767, 459884    
Bug Blocks: 459472    

Description Cedric Brun CLA 2015-01-21 05:36:27 EST
The current support for '[/]' expressions has a few drawbacks notably :
- it rely on a large number of dependencies which makes it harder to package a modeling environment in slightly different contexts (RCP-like, headless ...)
- it has performance characteristics which can be seen as quite unpredictable by the end users : the evaluation times depends on the Ecore models complexity and levels of inheritance
- some queries can get quite cumbersome because of the numerous collection types which are baked into the language (Set, Bag, OrderedSet, Sequence)

The Acceleo project started to work on an alternative implementation more suited to use cases where queries are intensely executed on top of models suffering constant changes. In a nutshell an alternative implementation tailored for the Sirius use case.

This ticket is about enabling the use of this implementation, first alongside the current implementation and then once we consider the implementation mature enough for the general public as the default one (and deprecating the current support)

Both implementation will not be compatible strictly speaking, there is no guarantee a valid MTL expression is valid in the context of the AQL engine, we'll have to go through a deprecation phase of the MTL implementation then. As a first step, the alternative should be usable by "opting in" directly in the .odesign, hence introducing a specific prefix. I'm proposing 'aql:' (for Acceleo Query Language).

Regarding timing, and depending on how it goes on the AQL side, the best case scenario would be IMO:
- Sirius 3.0 brings AQL support with "opt-in". We start mentionning to the user community that the MTL support is being deprecated in Sirius 4.0
- Sirius 4.0 makes AQL support "first class citizen" (removing the prefix) and MTL 
is accessible through the mtl: prefix.
- Sirius 5.0 makes MTL support 'optional in its own unsupported update-site'
Comment 1 Pierre-Charles David CLA 2015-01-21 07:50:08 EST
Cedric, do you know if there are specific changes required to the core of Sirius to properly support AQL, or if it would be a completely separate plug-in using the existing extension point and APIs?

Also, we should probably have a "debug" mode which does not require an explicit change in the VSM to switch to AQL. This would allow us to use the existing Sirius test suites (which contain hundreds of Acceleo expressions) to find differences between MTL and AQL, and fix AQL when appropriate.

This could take two forms:
- a flag which simply uses AQL for all "[/]" expressions;
- a mode that runs all expressions through both MTL and AQL engines, compares the results and the execution time, and returns either MTL or AQL. This could probably be implemented by a "wrapping interpreter" using only the existing APIs.
Comment 2 Cedric Brun CLA 2015-01-21 09:51:57 EST
So far I have a dedicated bridge which reacts to the 'aql:' prefix. No specific change *required* in the core of Sirius that being said if Sirius would leverage and pass around infered type information from one intepreter to another that could make the specifier's life easier.

An MTL expression is not always a valid AQL expression : the design choice of not supporting an implicit "self" variable in AQL means most of the time one has to review its expressions anyway. The other way around can be more obvious, most AQL expressions should work *just fine* with the MTL engine. ->select( expression would then have to move to the common form which is the verbose a SomeType| a.somethingPredicate), but that means we should be able to update the test cases to make sure they are compatible with both.

Do we actually have many test cases which are using the MTL engine ?
Comment 3 Pierre-Charles David CLA 2015-01-21 09:58:58 EST
(In reply to Cedric Brun from comment #2)
> Do we actually have many test cases which are using the MTL engine ?


A quick grep finds 1104 expressions in 66 different VSMs:

% git grep -l '/\]"' -- '**/*.odesign' | wc -l
66
% find . -name "*.odesign" | xargs cat | sed -e 's|/\]"|/]"\n|g' | grep -c "/]"
1104
Comment 4 Cedric Brun CLA 2015-01-21 12:15:50 EST
Pushed a first review here :

https://git.eclipse.org/r/#/c/40055/

I did not had the chance to review all the name and the code so far, so please do not submit as is, but feel free to have a look and give your opinion.
Comment 5 Pierre-Charles David CLA 2015-01-22 09:58:46 EST
(In reply to Cedric Brun from comment #4)
> Pushed a first review here :
> 
> https://git.eclipse.org/r/#/c/40055/
> 
> I did not had the chance to review all the name and the code so far, so
> please do not submit as is, but feel free to have a look and give your
> opinion.

By convention the commit message should contain "DRAFT" for such patches which ones want to be visible but are not ready for an actual merge.
Comment 6 Laurent Goubet CLA 2015-01-30 11:11:43 EST
A build of Acceleo including preliminary support for AQL has been published at [1] and added to the aggregation for Mars M5. This will thus be available in the result of the aggregation starting from [2] (as long as this build passes, still running at the time of writing).

The AQL feature id is 'org.eclipse.acceleo.query'.

[1] http://download.eclipse.org/acceleo/updates/milestones/3.6/S201501301201
[2] https://hudson.eclipse.org/hudson/job/simrel.mars.runaggregator/384/
Comment 7 Pierre-Charles David CLA 2015-02-02 07:36:59 EST
(In reply to Laurent Goubet from comment #6)
> A build of Acceleo including preliminary support for AQL has been published
> at [1] and added to the aggregation for Mars M5. This will thus be available
> in the result of the aggregation starting from [2] (as long as this build
> passes, still running at the time of writing).
> 
> The AQL feature id is 'org.eclipse.acceleo.query'.
> 
> [1] http://download.eclipse.org/acceleo/updates/milestones/3.6/S201501301201
> [2] https://hudson.eclipse.org/hudson/job/simrel.mars.runaggregator/384/

Actually, AQL will not be part of M5 because of an issue with the ANTLR jar it needs in Orbit; see https://bugs.eclipse.org/bugs/show_bug.cgi?id=458925 for the details.
Comment 8 Pierre-Charles David CLA 2015-02-03 07:45:03 EST
(In reply to Pierre-Charles David from comment #7)
> (In reply to Laurent Goubet from comment #6)
> > A build of Acceleo including preliminary support for AQL has been published
> > at [1] and added to the aggregation for Mars M5. This will thus be available
> > in the result of the aggregation starting from [2] (as long as this build
> > passes, still running at the time of writing).
> > 
> > The AQL feature id is 'org.eclipse.acceleo.query'.
> > 
> > [1] http://download.eclipse.org/acceleo/updates/milestones/3.6/S201501301201
> > [2] https://hudson.eclipse.org/hudson/job/simrel.mars.runaggregator/384/
> 
> Actually, AQL will not be part of M5 because of an issue with the ANTLR jar
> it needs in Orbit; see https://bugs.eclipse.org/bugs/show_bug.cgi?id=458925
> for the details.

It looks like this is actually fixed, on time for inclusion in M5: https://bugs.eclipse.org/bugs/show_bug.cgi?id=458925#c8
Comment 9 Eclipse Genie CLA 2015-02-12 03:46:45 EST
WARNING: this patchset contains 3004 new lines of code and may require a Contribution Questionnaire (CQ) if the author is not a committer on the project. Please see:https://wiki.eclipse.org/Project_Management_Infrastructure/Creating_A_Contribution_Questionnaire
Comment 10 Cedric Brun CLA 2015-02-12 07:54:21 EST
Patchset https://git.eclipse.org/r/#/c/41455/ is failing becaose of Bug 459767
Comment 11 Pierre-Charles David CLA 2015-02-12 10:06:00 EST
The migration of interactions.odesign to use something else than the legacy query language was already tracked by bug 452318 "Replace all expressions written with legacy query langage in the Interaction Sequence sample".

Cédric, the part of your work here which concerns interactions.odesign could reference that other ticket instead (or maybe we should close the other as duplicate?).
Comment 12 Eclipse Genie CLA 2015-02-12 11:03:28 EST
WARNING: this patchset contains 3004 new lines of code and may require a Contribution Questionnaire (CQ) if the author is not a committer on the project. Please see:https://wiki.eclipse.org/Project_Management_Infrastructure/Creating_A_Contribution_Questionnaire
Comment 13 Eclipse Genie CLA 2015-02-13 10:22:53 EST
WARNING: this patchset contains 3018 new lines of code and may require a Contribution Questionnaire (CQ) if the author is not a committer on the project. Please see:https://wiki.eclipse.org/Project_Management_Infrastructure/Creating_A_Contribution_Questionnaire
Comment 14 Eclipse Genie CLA 2015-02-25 11:06:23 EST
WARNING: this patchset contains 3035 new lines of code and may require a Contribution Questionnaire (CQ) if the author is not a committer on the project. Please see:https://wiki.eclipse.org/Project_Management_Infrastructure/Creating_A_Contribution_Questionnaire
Comment 15 Eclipse Genie CLA 2015-03-02 11:31:22 EST
WARNING: this patchset contains 3035 new lines of code and may require a Contribution Questionnaire (CQ) if the author is not a committer on the project. Please see:https://wiki.eclipse.org/Project_Management_Infrastructure/Creating_A_Contribution_Questionnaire
Comment 16 Eclipse Genie CLA 2015-03-03 05:03:21 EST
WARNING: this patchset contains 3035 new lines of code and may require a Contribution Questionnaire (CQ) if the author is not a committer on the project. Please see:https://wiki.eclipse.org/Project_Management_Infrastructure/Creating_A_Contribution_Questionnaire
Comment 17 Eclipse Genie CLA 2015-03-03 09:37:09 EST
WARNING: this patchset contains 3051 new lines of code and may require a Contribution Questionnaire (CQ) if the author is not a committer on the project. Please see:https://wiki.eclipse.org/Project_Management_Infrastructure/Creating_A_Contribution_Questionnaire
Comment 18 Eclipse Genie CLA 2015-03-04 05:12:46 EST
WARNING: this patchset contains 3053 new lines of code and may require a Contribution Questionnaire (CQ) if the author is not a committer on the project. Please see:https://wiki.eclipse.org/Project_Management_Infrastructure/Creating_A_Contribution_Questionnaire
Comment 19 Eclipse Genie CLA 2015-03-04 06:00:17 EST
Gerrit change https://git.eclipse.org/r/40055 was merged to [master].
Commit: http://git.eclipse.org/c/sirius/org.eclipse.sirius.git/commit/?id=fbf8869240014b6cebf5c94d05eba12eda1adbe5

WARNING: this patchset contains 3053 new lines of code and may require a Contribution Questionnaire (CQ) if the author is not a committer on the project. Please see:https://wiki.eclipse.org/Project_Management_Infrastructure/Creating_A_Contribution_Questionnaire
Comment 20 Eclipse Genie CLA 2015-03-12 10:54:36 EDT
New Gerrit change created: https://git.eclipse.org/r/43726
Comment 21 Eclipse Genie CLA 2015-03-18 12:34:48 EDT
New Gerrit change created: https://git.eclipse.org/r/44113
Comment 23 Pierre-Charles David CLA 2015-03-23 09:44:29 EDT
The main feature is here. There are some additional imrpovements and related tasks left, but they will be handled in different tickets.
Comment 24 Eclipse Genie CLA 2015-05-04 09:38:23 EDT
New Gerrit change created: https://git.eclipse.org/r/47046
Comment 26 Pierre-Charles David CLA 2015-05-26 09:41:14 EDT
Verified on Sirius 3.0.0rc1a.

I only verified the basics of evaluation, completion and validation. For the details, I thinkg we can rely on the fact that the full SWTBot Sequence test suites uses a modeler which uses AQL extensively, and also on EcoreTools 3.0.
Comment 27 Pierre-Charles David CLA 2015-06-24 11:14:35 EDT
Available in Sirius 3.0.0. See https://wiki.eclipse.org/Sirius/3.0.0.