Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 344723

Summary: [parser] StackOverFlowError in erroneous jsp file - destabilizing issue
Product: [WebTools] WTP Source Editing Reporter: Patric Rufflar <patric>
Component: jst.jspAssignee: Nick Sandonato <nsand.dev>
Status: RESOLVED FIXED QA Contact: Nick Sandonato <nsand.dev>
Severity: major    
Priority: P3 CC: nsand.dev, thatnitind
Version: unspecified   
Target Milestone: 3.4.2   
Hardware: PC   
OS: Windows XP   
Whiteboard:
Attachments:
Description Flags
file which will reproduce the bug
none
patch
none
patch for styles none

Description Patric Rufflar CLA 2011-05-04 11:55:27 EDT
Build Identifier: 3.2.3

java.lang.StackOverflowError
	at org.eclipse.wst.xml.core.internal.parser.IntStack.push(IntStack.java:88)
	at org.eclipse.jst.jsp.core.internal.parser.internal.JSPTokenizer.primGetNextToken(JSPTokenizer.java:3918)
	at org.eclipse.jst.jsp.core.internal.parser.internal.JSPTokenizer.getNextToken(JSPTokenizer.java:1637)
	at org.eclipse.wst.xml.core.internal.parser.XMLSourceParser.getNextRegion(XMLSourceParser.java:189)
	at org.eclipse.jst.jsp.core.internal.parser.JSPSourceParser.parseNodes(JSPSourceParser.java:237)
	at org.eclipse.wst.xml.core.internal.parser.XMLSourceParser.getDocumentRegions(XMLSourceParser.java:173)
	at org.eclipse.wst.sse.core.internal.text.StructuredDocumentReParser._core_reparse_text(StructuredDocumentReParser.java:310)
	at org.eclipse.wst.sse.core.internal.text.StructuredDocumentReParser.core_reparse(StructuredDocumentReParser.java:710)
	at org.eclipse.jst.jsp.core.internal.parser.JSPReParser.reparse(JSPReParser.java:143)
	at org.eclipse.wst.sse.core.internal.text.StructuredDocumentReParser.reparse(StructuredDocumentReParser.java:1291)
	at org.eclipse.wst.sse.core.internal.text.BasicStructuredDocument.updateModel(BasicStructuredDocument.java:2709)
	at org.eclipse.wst.sse.core.internal.text.BasicStructuredDocument.internalReplaceText(BasicStructuredDocument.java:1923)
	at org.eclipse.wst.sse.core.internal.text.BasicStructuredDocument.replaceText(BasicStructuredDocument.java:2419)
	at org.eclipse.wst.sse.core.internal.text.BasicStructuredDocument.set(BasicStructuredDocument.java:2931)
	at org.eclipse.core.internal.filebuffers.ResourceTextFileBuffer.setDocumentContent(ResourceTextFileBuffer.java:575)
	at org.eclipse.core.internal.filebuffers.ResourceTextFileBuffer.initializeFileBufferContent(ResourceTextFileBuffer.java:286)
	at org.eclipse.core.internal.filebuffers.ResourceFileBuffer.create(ResourceFileBuffer.java:245)
	at org.eclipse.core.internal.filebuffers.TextFileBufferManager.connect(TextFileBufferManager.java:112)
	at org.eclipse.ui.editors.text.TextFileDocumentProvider.createFileInfo(TextFileDocumentProvider.java:559)
	at org.eclipse.ui.editors.text.TextFileDocumentProvider.connect(TextFileDocumentProvider.java:478)
	at org.eclipse.ui.texteditor.AbstractTextEditor.doSetInput(AbstractTextEditor.java:4050)
	at org.eclipse.ui.texteditor.StatusTextEditor.doSetInput(StatusTextEditor.java:217)
	at org.eclipse.ui.texteditor.AbstractDecoratedTextEditor.doSetInput(AbstractDecoratedTextEditor.java:1444)
	at org.eclipse.ui.editors.text.TextEditor.doSetInput(TextEditor.java:169)
	at org.eclipse.wst.sse.ui.StructuredTextEditor.doSetInput(StructuredTextEditor.java:1849)
	at org.eclipse.ui.texteditor.AbstractTextEditor.setInputWithNotify(AbstractTextEditor.java:4114)
	at org.eclipse.ui.texteditor.AbstractTextEditor.setInput(AbstractTextEditor.java:4136)
	at org.eclipse.ui.texteditor.AbstractTextEditor$7.run(AbstractTextEditor.java:527)
	at org.eclipse.ui.texteditor.AbstractTextEditor$ElementStateListener.execute(AbstractTextEditor.java:596)
	at org.eclipse.ui.texteditor.AbstractTextEditor$ElementStateListener.elementMoved(AbstractTextEditor.java:557)
	at org.eclipse.ui.editors.text.TextFileDocumentProvider$FileBufferListener.underlyingFileMoved(TextFileDocumentProvider.java:322)
	at org.eclipse.core.internal.filebuffers.TextFileBufferManager$6.run(TextFileBufferManager.java:742)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
	at org.eclipse.core.internal.filebuffers.TextFileBufferManager.fireUnderlyingFileMoved(TextFileBufferManager.java:740)
	at org.eclipse.core.internal.filebuffers.ResourceFileBuffer.handleFileMoved(ResourceFileBuffer.java:463)
	at org.eclipse.core.internal.filebuffers.ResourceFileBuffer$3.execute(ResourceFileBuffer.java:162)
	at org.eclipse.core.internal.filebuffers.ResourceFileBuffer$SafeFileChange.run(ResourceFileBuffer.java:86)
	at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
	at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:134)
	at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4041)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3660)
	at org.eclipse.jface.operation.ModalContext$ModalContextThread.block(ModalContext.java:173)
	at org.eclipse.jface.operation.ModalContext.run(ModalContext.java:388)
	at org.eclipse.ltk.internal.ui.refactoring.RefactoringWizardDialog2.run(RefactoringWizardDialog2.java:330)
	at org.eclipse.ltk.ui.refactoring.RefactoringWizard.internalPerformFinish(RefactoringWizard.java:573)
	at org.eclipse.ltk.ui.refactoring.UserInputWizardPage.performFinish(UserInputWizardPage.java:153)
	at org.eclipse.ltk.ui.refactoring.resource.RenameResourceWizard$RenameResourceRefactoringConfigurationPage.performFinish(RenameResourceWizard.java:117)
	at org.eclipse.ltk.ui.refactoring.RefactoringWizard.performFinish(RefactoringWizard.java:646)
	at org.eclipse.ltk.internal.ui.refactoring.RefactoringWizardDialog2.okPressed(RefactoringWizardDialog2.java:454)
	at org.eclipse.jface.dialogs.Dialog.buttonPressed(Dialog.java:472)
	at org.eclipse.jface.dialogs.Dialog$2.widgetSelected(Dialog.java:624)
	at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:234)
	at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1053)
	at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4066)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3657)
	at org.eclipse.jface.window.Window.runEventLoop(Window.java:825)
	at org.eclipse.jface.window.Window.open(Window.java:801)
	at org.eclipse.ltk.ui.refactoring.RefactoringWizardOpenOperation$1.run(RefactoringWizardOpenOperation.java:180)
	at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:70)
	at org.eclipse.ltk.ui.refactoring.RefactoringWizardOpenOperation.run(RefactoringWizardOpenOperation.java:192)
	at org.eclipse.ltk.ui.refactoring.RefactoringWizardOpenOperation.run(RefactoringWizardOpenOperation.java:115)
	at org.eclipse.ltk.internal.ui.refactoring.actions.RenameResourceHandler.execute(RenameResourceHandler.java:39)
	at org.eclipse.ui.internal.handlers.HandlerProxy.execute(HandlerProxy.java:293)
	at org.eclipse.core.commands.Command.executeWithChecks(Command.java:476)
	at org.eclipse.core.commands.ParameterizedCommand.executeWithChecks(ParameterizedCommand.java:508)
	at org.eclipse.ui.internal.handlers.HandlerService.executeCommandInContext(HandlerService.java:270)
	at org.eclipse.ui.internal.ide.actions.LTKLauncher.runCommand(LTKLauncher.java:95)
	at org.eclipse.ui.internal.ide.actions.LTKLauncher.openRenameWizard(LTKLauncher.java:73)
	at org.eclipse.ui.actions.RenameResourceAction.run(RenameResourceAction.java:461)
	at org.eclipse.ui.actions.BaseSelectionListenerAction.runWithEvent(BaseSelectionListenerAction.java:168)
	at org.eclipse.jface.commands.ActionHandler.execute(ActionHandler.java:119)
	at org.eclipse.core.commands.Command.executeWithChecks(Command.java:476)
	at org.eclipse.core.commands.ParameterizedCommand.executeWithChecks(ParameterizedCommand.java:508)
	at org.eclipse.ui.internal.handlers.HandlerService.executeCommand(HandlerService.java:169)
	at org.eclipse.ui.internal.keys.WorkbenchKeyboard.executeCommand(WorkbenchKeyboard.java:468)
	at org.eclipse.ui.internal.keys.WorkbenchKeyboard.press(WorkbenchKeyboard.java:786)
	at org.eclipse.ui.internal.keys.WorkbenchKeyboard.processKeyEvent(WorkbenchKeyboard.java:885)
	at org.eclipse.ui.internal.keys.WorkbenchKeyboard.filterKeySequenceBindings(WorkbenchKeyboard.java:567)
	at org.eclipse.ui.internal.keys.WorkbenchKeyboard.access$3(WorkbenchKeyboard.java:508)
	at org.eclipse.ui.internal.keys.WorkbenchKeyboard$KeyDownFilter.handleEvent(WorkbenchKeyboard.java:123)
	at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
	at org.eclipse.swt.widgets.Display.filterEvent(Display.java:1253)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1052)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1077)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1062)
	at org.eclipse.swt.widgets.Widget.sendKeyEvent(Widget.java:1103)
	at org.eclipse.swt.widgets.Widget.sendKeyEvent(Widget.java:1099)
	at org.eclipse.swt.widgets.Widget.wmKeyDown(Widget.java:1808)
	at org.eclipse.swt.widgets.Control.WM_KEYDOWN(Control.java:4496)
	at org.eclipse.swt.widgets.Tree.WM_KEYDOWN(Tree.java:5976)
	at org.eclipse.swt.widgets.Control.windowProc(Control.java:4191)
	at org.eclipse.swt.widgets.Tree.windowProc(Tree.java:5844)
	at org.eclipse.swt.widgets.Display.windowProc(Display.java:4873)
	at org.eclipse.swt.internal.win32.OS.DispatchMessageW(Native Method)
	at org.eclipse.swt.internal.win32.OS.DispatchMessage(OS.java:2459)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3655)
	at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2629)
	at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2593)
	at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2427)
	at org.eclipse.ui.internal.Workbench$7.run(Workbench.java:670)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
	at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:663)
	at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
	at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:115)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:369)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:619)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:574)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1407)

java.lang.StackOverflowError
	at org.eclipse.wst.xml.core.internal.parser.IntStack.push(IntStack.java:88)
	at org.eclipse.jst.jsp.core.internal.parser.internal.JSPTokenizer.primGetNextToken(JSPTokenizer.java:3918)
	at org.eclipse.jst.jsp.core.internal.parser.internal.JSPTokenizer.getNextToken(JSPTokenizer.java:1637)
	at org.eclipse.wst.xml.core.internal.parser.XMLSourceParser.getNextRegion(XMLSourceParser.java:189)
	at org.eclipse.jst.jsp.core.internal.parser.JSPSourceParser.parseNodes(JSPSourceParser.java:237)
	at org.eclipse.wst.xml.core.internal.parser.XMLSourceParser.getDocumentRegions(XMLSourceParser.java:173)
	at org.eclipse.wst.sse.core.internal.text.StructuredDocumentReParser._core_reparse_text(StructuredDocumentReParser.java:310)
	at org.eclipse.wst.sse.core.internal.text.StructuredDocumentReParser.core_reparse(StructuredDocumentReParser.java:710)
	at org.eclipse.jst.jsp.core.internal.parser.JSPReParser.reparse(JSPReParser.java:143)
	at org.eclipse.wst.sse.core.internal.text.StructuredDocumentReParser.reparse(StructuredDocumentReParser.java:1291)
	at org.eclipse.wst.sse.core.internal.text.BasicStructuredDocument.updateModel(BasicStructuredDocument.java:2709)
	at org.eclipse.wst.sse.core.internal.text.BasicStructuredDocument.internalReplaceText(BasicStructuredDocument.java:1923)
	at org.eclipse.wst.sse.core.internal.text.BasicStructuredDocument.replaceText(BasicStructuredDocument.java:2419)
	at org.eclipse.wst.sse.core.internal.text.BasicStructuredDocument.set(BasicStructuredDocument.java:2931)
	at org.eclipse.core.internal.filebuffers.ResourceTextFileBuffer.setDocumentContent(ResourceTextFileBuffer.java:575)
	at org.eclipse.core.internal.filebuffers.ResourceTextFileBuffer.initializeFileBufferContent(ResourceTextFileBuffer.java:286)
	at org.eclipse.core.internal.filebuffers.ResourceFileBuffer.create(ResourceFileBuffer.java:245)
	at org.eclipse.core.internal.filebuffers.TextFileBufferManager.connect(TextFileBufferManager.java:112)
	at org.eclipse.wst.sse.core.internal.FileBufferModelManager.getModel(FileBufferModelManager.java:654)
	at org.eclipse.wst.sse.core.internal.model.ModelManagerImpl._doCommonGetModel(ModelManagerImpl.java:541)
	at org.eclipse.wst.sse.core.internal.model.ModelManagerImpl._commonGetModel(ModelManagerImpl.java:508)
	at org.eclipse.wst.sse.core.internal.model.ModelManagerImpl._commonGetModel(ModelManagerImpl.java:481)
	at org.eclipse.wst.sse.core.internal.model.ModelManagerImpl.getModelForRead(ModelManagerImpl.java:1410)
	at org.eclipse.wst.jsdt.web.core.javascript.search.JsSearchDocument.getJSTranslation(JsSearchDocument.java:117)
	at org.eclipse.wst.jsdt.web.core.javascript.search.JsSearchDocument.getPath(JsSearchDocument.java:165)
	at org.eclipse.wst.jsdt.web.core.javascript.search.JSDTSearchDocumentDelegate.<init>(JSDTSearchDocumentDelegate.java:37)
	at org.eclipse.wst.jsdt.web.core.javascript.search.JsSearchSupport.createSearchDocument(JsSearchSupport.java:425)
	at org.eclipse.wst.jsdt.web.core.javascript.search.JsSearchSupport.addJspFile(JsSearchSupport.java:317)
	at org.eclipse.wst.jsdt.web.core.javascript.search.JsIndexManager$ProcessFilesJob.run(JsIndexManager.java:277)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)



Reproducible: Always

Steps to Reproduce:
1. Try to open attached file with WTP
Comment 1 Patric Rufflar CLA 2011-05-04 11:56:27 EDT
Created attachment 194732 [details]
file which will reproduce the bug

Just open this file within WTP.
Comment 2 Nick Sandonato CLA 2011-05-04 16:42:37 EDT
As a fringe case, looking like a fuzz test similar to the others that you recently opened, we're going to take down the severity. This doesn't seem like a frequent enough scenario to warrant critical.
Comment 3 Nick Sandonato CLA 2011-05-04 16:45:01 EDT
Created attachment 194767 [details]
patch

The problem with the stack was that it wasn't resizable. It was assumed that by the time we got to a state stack of 100, the document was already beyond repair. I guess this test file fits right in line with that assumption ;)
Comment 4 Nick Sandonato CLA 2011-05-04 17:07:30 EDT
Thanks for the sample file, Patric.
Comment 5 Nick Sandonato CLA 2011-05-04 18:09:38 EDT
Created attachment 194776 [details]
patch for styles

Even with the patch to correct the IntStack, the style ranges generated were off in places. Looking into it, I realized we could definitely do with some bullet proofing to the PresentationCollector to avoid being unable to open an editor for some of these invalid styles. For example, I was seeing negative length text regions. I believe a bug you opened before, Patric, ended up with negative length text regions as well.
Comment 6 Patric Rufflar CLA 2011-05-05 09:23:48 EDT
Nick,

please note that I am using the fuzz tests only to be able to provide you a way to reproduce the bug.

Every bug report which I opened has been seen in "real life" from my colleagues or myself.
During my fuzz tests I saw even more errors which I didn't post (yet).
Comment 7 Nick Sandonato CLA 2011-05-06 17:23:34 EDT
At 214097, there's a bit <%--%> that seems to cause all the problems in terms of overlapping style regions. Cleanup the <%-- and no more syntax highlighting problems. This particular bit seems to be causing a negative text region length.
Comment 8 Nick Sandonato CLA 2011-05-09 17:05:27 EDT
Checked in changes to the IntStack as well as the PresentationCollector.Still investigating why we're getting negative lengths.

It seems like yy_markedPos is sometimes ending up behind yy_startRead when we invoke yylength() in getNextToken()
Comment 9 Nitin Dahyabhai CLA 2012-04-25 15:06:11 EDT
(In reply to comment #8)
> Checked in changes to the IntStack as well as the PresentationCollector.Still
> investigating why we're getting negative lengths.
> 
> It seems like yy_markedPos is sometimes ending up behind yy_startRead when we
> invoke yylength() in getNextToken()

It could be that the manipulation going on in #doScan() doesn't properly guard against cases where the start region contents overlap with the searchString.
Comment 10 Nick Sandonato CLA 2012-12-03 11:10:22 EST
Final bits pushed today to correct the overlapping ranges.

http://git.eclipse.org/c/sourceediting/webtools.sourceediting.git/commit/?id=19ef5ca9119489b5952b6c152aa1b0964f2448be