Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 432956

Summary: [esprima] Program beginning with invalid Variable Declaration should parse
Product: [ECD] Orion Reporter: Mark Macdonald <mamacdon>
Component: JS ToolsAssignee: Michael Rennie <Michael_Rennie>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: Michael_Rennie
Version: 5.0   
Target Milestone: 6.0 M2   
Hardware: PC   
OS: Windows 7   
Whiteboard:
Bug Depends on: 423091    
Bug Blocks: 433441    
Attachments:
Description Flags
fix none

Description Mark Macdonald CLA 2014-04-16 14:52:14 EDT
Attempt to parse this code with the tolerant esprima parser:

var g = {
	f
};

It returns an AST with errors, but no program body.
> { type: "Program", 
>   body: []
>   errors: [ /*omitted*/ ]
> } 

Was expecting to get a body containing a  VariableDeclaration, VariableDeclarator, and ObjectExpression.
Comment 1 Michael Rennie CLA 2014-04-20 10:17:13 EDT
I pushed tests to the branch that should all pass once the support is added: http://git.eclipse.org/c/orion/org.eclipse.orion.client.git/commit/?h=esprima_tolerant3&id=6170c42d00ba10243bfebe50830b6381cec199e7

The commit also adds support to the parser suite to verify the token stream and to visit the returned AST to make sure the expected node(s) appear in the expected hierarchy.

In most cases we just need to enhance lexing properties to skip bad property decls and continue on to the next expected token - so in the given snippet we would process the identifier token (add it to the stream) but not end up with a property node.

The AST would look like:

Body
  VariableDeclaration
    VariableDeclarator (lhs, init)
      ObjectExpression

and the stream should look like:

[Keyword, Identifier, Punctuator, Punctuator, Identifier, Punctuator]

We could also try patching the AST to repair the AST by inserting a dummy node to complete the property decl. In that case we would skip until the next value node, rewind and insert the missing Punctuator and init.
Comment 2 Michael Rennie CLA 2014-04-24 16:56:39 EDT
Created attachment 242300 [details]
fix

This patch provides recovery support for bad property declarations of the following forms:

1. incomplete statements - var f = {a }
2. missing commas - var f = {a:1 b:2} 

the recovery support does not handle cases where a colon is present with no following expression. The patch also contains the code to make our new esprima the default in the workspace + a tonne of tests
Comment 3 Michael Rennie CLA 2014-04-28 09:43:22 EDT
Fixed with: http://git.eclipse.org/c/orion/org.eclipse.orion.client.git/commit/?id=0abe51a10fe70ef9c82327ac6644f86f15a62e28

This commit also makes the new Esprima the default (for bug 423091)