Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 347817

Summary: Make OQL parser rebuildable from javacc grammar file
Product: [Tools] MAT Reporter: Andrew Johnson <andrew_johnson>
Component: CoreAssignee: Andrew Johnson <andrew_johnson>
Status: RESOLVED FIXED QA Contact:
Severity: enhancement    
Priority: P3    
Version: unspecified   
Target Milestone: ---   
Hardware: All   
OS: All   
Whiteboard:
Bug Depends on:    
Bug Blocks: 241154, 287268    

Description Andrew Johnson CLA 2011-05-31 11:12:31 EDT 
The OQL parser is built using the javacc compiler compiler from the OQLParser.jj file.

This has been done in the past, and the generated java files have been checked into SVN, and have then been modified.

If we ever need to change the .jj file then we need a process for doing that can generate java files with the modifications.
1.Document process - e.g. run javacc, reformat files, merge changes from existing version
2.Apply more of the changes to the .jj file to minimise fixups
3.Have automatic process for applying fixups
3.If no changes required, then run javacc as part of build
Comment 1 Andrew Johnson CLA 2011-06-29 12:55:34 EDT 
I have written an ant file which runs javacc, then fix up the generated code:
				<!-- javacc converts backslash to a unicode escape, which works but looks messy, so reverse it -->
				<replaceregex pattern="u005c" replace="" flags="g" />
				<!-- avoid problems with the Turkish locale with toUpperCase -->
				<replaceregex pattern="toUpperCase\(\)" replace="toUpperCase\(Locale.ENGLISH\)" flags="g" />
				<!-- internationalize the missing return message -->
				<replaceregex pattern='"Missing return statement in function"' replace="Messages.OQLParser_Missing_return_statement_in_function" flags="g" />
				<!-- avoid warnings by adding suppressWarnings -->
				<replaceregex pattern="public class OQLParser" replace='@SuppressWarnings({"nls", "unused"}) \0' />
				<!-- remove semicolon on its own -->
				<replaceregex pattern="^\s*;$" replace="" />

The generated code then needs to be reformatted into the project conventions and the imports organized.

The differences then are quite small.
Comment 2 Andrew Johnson CLA 2011-10-10 12:07:39 EDT 
The smallest differences are when using javacc 4.1

Changes made to generate files:
OQLParser.java
Added copyright header
Added @SuppressWarnings("serial") to LookaheadSuccess
Added @SuppressWarnings("unchecked") to some methods which don't use generics (javacc 5.0 will fix this)

OQLParserConstants.java
Added copyright header
Added @SuppressWarnings("nls") to public interface OQLParserConstants

OQLParserTokenManager.java
Added copyright header
Added some @SuppressWarnings("cast")
Added some @SuppressWarnings("unused")

ParseException.java
Added copyright header
Added @SuppressWarnings( { "serial", "nls" }) to ParseException

SimpleCharStream.java
Added copyright header
Added @SuppressWarnings("unused") to getTabSize(int i)
Changed JavaCC - OriginalChecksum=5a86e00a0b941ec037f0713e69ba91c3 (do not edit this
 line)

Token.java
Added copyright header
Added @SuppressWarnings( { "serial", "nls" }) to TokenMgrError
Added @SuppressWarnings("unused") to some methods
Changed JavaCC - OriginalChecksum=c1af33c36aa1c949195f80d1cbfbfd9a (do not edit this
 line)

We should check in versions of these files generated by a reproducible process with javacc 4.1 then move up to javacc 5.0