Bug 340306

Description Jurgen Vinju 2011-03-17 08:39:35 EDT

Currently there is a data dependency between parse controller and the language services within a single run() of parse scheduler. This dependency is necessary and natural because the services work on the data that parsers produce.

The dependency is coded like this:
  Object  IParseController.parse(...); // generates an ast, but the ast is ignored at the call site in run()
  void     IASTListener.update(IParseController pc, ....) // called after parse(...) has finished
  Object  IParseController.getCurrentAST() // called by IASTListener to get the parse tree.

A recently proposed change to have annotators in between the parser and the IASTListeners changes this to:
  Object  IParseController.parse(...); // generates an ast, which is safe in a local variable "ast" 
  Object  IAnnotatorService.annotate(ast, ...) ; // gets an ast and returns a new one
  void     IASTListener.update(IParseController pc, ....) // called after parse(...) has finished
  Object  IParseController.getCurrentAST() // called by IASTListener to get the parse tree.

The new ast's produced in the first scenario can only be accessed if and only if the implementor of IParseController has been so kind to provide the same ast. One huge problem is that this state is prone to data races.

The new ast's produced in the second scenario by the annotators, can unfortunately not be accessed at all by the IASTListeners because they cant access the internal state of any IParseController.

So, let's change it to:
  Object  IParseController.parse(...); // generates an ast, which is safe in a local variable "ast" 
  Object  IAnnotatorService.annotate(ast, ...) ; // gets an ast and returns a new one
  Object     IASTListener.update(ast, ....) // called after parse(...) has finished

* If an IASTListeners needs access to other services of a parse controller, it can always implement IEditorService, get the parse controller and ask for SyntaxProperties, or anything else
* The IASTListener now is guaranteed to get the ast that was produced by the parser and the annotators, without any data races
* The IASTListeners will have access to the annotated AST's, which allows interesting IDE features such as semantic coloring.

I'm aware that this represents a significant API change, and that all IParseControllers will have to be adapted as well as all IASTListeners... I'm willing to do my part in imp.runtime, but of course clients will also have to comply.