Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 355988

Summary: NewLazyTokenStream doesn't start on "valid" token
Product: z_Archived Reporter: Chris Cieslinski <the.maltese.duck>
Component: WTP IncubatorAssignee: Project Inbox <wtp.inc.xquery-inbox>
Status: NEW --- QA Contact: XQDT <xqdt>
Severity: normal    
Priority: P3    
Version: unspecified   
Target Milestone: ---   
Hardware: All   
OS: All   
Whiteboard:

Description Chris Cieslinski CLA 2011-08-26 17:39:07 EDT
It seems in cases where an XQuery file starts with a comment, i.e:

(: Comment :)
xquery version "1.0-ml";
fn:current-dateTime()

that the token steam doesn't start on the first valid token (like the CommonTokenStream does after filling the buffer) so it has problems parsing the file because the "pointer" isn't in a good place:

line 2:0 no viable alternative at input 'xquery'
<unexpected: [@2,14:19='xquery',<111>,2:0], resync=xquery version "1.0-ml";
fn:current-dateTime()> 

One possible solution is to update the LT() method:

    public Token LT(int k) {
        if (k == 0)
            return null;
        if (k < 0)
            return readReverseNthGoodToken(-k);

        Token t = readNthGoodToken(k);
        
        // If we are just starting out make sure the pointer to the stream is on
        // a valid token (similar to what the CommonTokenStream does when it
        // fills it's buffer)
        if (p == 0) {
            p = t.getTokenIndex();
        }
        return t;
    }