Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 355988 - NewLazyTokenStream doesn't start on "valid" token
Summary: NewLazyTokenStream doesn't start on "valid" token
Status: NEW
Alias: None
Product: z_Archived
Classification: Eclipse Foundation
Component: WTP Incubator (show other bugs)
Version: unspecified   Edit
Hardware: All All
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact: XQDT CLA
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-08-26 17:39 EDT by Chris Cieslinski CLA
Modified: 2021-11-18 16:19 EST (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Chris Cieslinski CLA 2011-08-26 17:39:07 EDT
It seems in cases where an XQuery file starts with a comment, i.e:

(: Comment :)
xquery version "1.0-ml";
fn:current-dateTime()

that the token steam doesn't start on the first valid token (like the CommonTokenStream does after filling the buffer) so it has problems parsing the file because the "pointer" isn't in a good place:

line 2:0 no viable alternative at input 'xquery'
<unexpected: [@2,14:19='xquery',<111>,2:0], resync=xquery version "1.0-ml";
fn:current-dateTime()> 

One possible solution is to update the LT() method:

    public Token LT(int k) {
        if (k == 0)
            return null;
        if (k < 0)
            return readReverseNthGoodToken(-k);

        Token t = readNthGoodToken(k);
        
        // If we are just starting out make sure the pointer to the stream is on
        // a valid token (similar to what the CommonTokenStream does when it
        // fills it's buffer)
        if (p == 0) {
            p = t.getTokenIndex();
        }
        return t;
    }