Bug 329624

Description Ralf Ebert 2010-11-06 19:13:55 EDT

PartialParsingHelper#calculatePartialParsingPointers currently merges all document errors in the replace region and looks for a node that encloses this region. Merging error nodes fails if the error is sitting on a document node that is empty (totalLength = 0), because it cannot grasp this node using an offset + length.

This can happen with an AbstractSplittingTokenSource that inserts synthetic tokens in the token stream to represent concepts that don't have a direct representation in the text document, f.e. BEGIN/END tokens for increasing/decreasing indentation in the document.

Attached patch series fixes this issue regarding partial parsing and such synthetic tokens.

0001/0002 adds a simple indentation-sensitive test-language. It doesn't work yet with partial parsing as IndentTokenSource is stateful. Still, it is useful to demonstrate this issue and as I plan to continue the work on such languages, it would be great if this could sit around as general-purpose test language so I could refer to it in later patches.

0003 adds a test that shows the problem regarding partial parsing and empty tokens (thx again to Jan who helped me to get this working at ESE)

0004+0005 extracts some general-purpose code from PartialParsingHelper into the general-purpose Range / NodeUtil classes. I added functionality to walk trees searching for specific nodes matching a predicate and narrow it up to enclose a document region. Includes docs + tests.

0006 uses this new functionality to find the node(s) that encloses the document region and includes all errors. Essentially, this changes the inclusion of errors 'by document region' to 'by node'.

AllTests are green for me.