Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 360710

Summary: Error while parsing /Project/header.h. ArrayIndexOutOfBoundsException
Product: [Tools] CDT Reporter: Marc-André Laperle <malaperle>
Component: cdt-indexerAssignee: Markus Schorn <mschorn.eclipse>
Status: RESOLVED FIXED QA Contact: Markus Schorn <mschorn.eclipse>
Severity: normal    
Priority: P3 CC: cdtdoug
Version: 8.0   
Target Milestone: 8.0.2   
Hardware: All   
OS: All   
Whiteboard:

Description Marc-André Laperle CLA 2011-10-12 14:13:05 EDT
C/C++ Development Tools 8.0.0.201110101006

I'm not sure of the exact repro but I get this pretty frequently. I have Refresh on Access turned on and I use active build configuration for the indexer. This seems to happen when I have a lot of modified files externally and change the active configuration.

java.lang.ArrayIndexOutOfBoundsException: 15360
	at org.eclipse.cdt.internal.core.pdom.db.Database.getChunk(Database.java:274)
	at org.eclipse.cdt.internal.core.pdom.db.Database.getChar(Database.java:517)
	at org.eclipse.cdt.internal.core.pdom.db.LongString.readChars(LongString.java:309)
	at org.eclipse.cdt.internal.core.pdom.db.LongString.getString(LongString.java:373)
	at org.eclipse.cdt.internal.core.pdom.dom.PDOMFile.getLocation(PDOMFile.java:745)
	at org.eclipse.cdt.internal.core.index.IndexBasedFileContentProvider.collectFileContent(IndexBasedFileContentProvider.java:161)
	at org.eclipse.cdt.internal.core.index.IndexBasedFileContentProvider.collectFileContent(IndexBasedFileContentProvider.java:186)
	at org.eclipse.cdt.internal.core.index.IndexBasedFileContentProvider.getContentForInclusion(IndexBasedFileContentProvider.java:125)
	at org.eclipse.cdt.internal.core.parser.scanner.CPreprocessor$1.checkFile(CPreprocessor.java:103)
	at org.eclipse.cdt.internal.core.parser.scanner.CPreprocessor$1.checkFile(CPreprocessor.java:1)
	at org.eclipse.cdt.internal.core.parser.scanner.CPreprocessor.findInclusion(CPreprocessor.java:984)
	at org.eclipse.cdt.internal.core.parser.scanner.CPreprocessor.executeInclude(CPreprocessor.java:1276)
	at org.eclipse.cdt.internal.core.parser.scanner.CPreprocessor.executeDirective(CPreprocessor.java:1087)
	at org.eclipse.cdt.internal.core.parser.scanner.CPreprocessor.internalFetchToken(CPreprocessor.java:743)
	at org.eclipse.cdt.internal.core.parser.scanner.CPreprocessor.fetchToken(CPreprocessor.java:482)
	at org.eclipse.cdt.internal.core.parser.scanner.CPreprocessor.nextToken(CPreprocessor.java:578)
	at org.eclipse.cdt.internal.core.dom.parser.AbstractGNUSourceCodeParser.fetchToken(AbstractGNUSourceCodeParser.java:265)
	at org.eclipse.cdt.internal.core.dom.parser.AbstractGNUSourceCodeParser.nextToken(AbstractGNUSourceCodeParser.java:289)
	at org.eclipse.cdt.internal.core.dom.parser.AbstractGNUSourceCodeParser.acceptInactiveCodeBoundary(AbstractGNUSourceCodeParser.java:350)
	at org.eclipse.cdt.internal.core.dom.parser.AbstractGNUSourceCodeParser.declarationList(AbstractGNUSourceCodeParser.java:1313)
	at org.eclipse.cdt.internal.core.dom.parser.AbstractGNUSourceCodeParser.parseTranslationUnit(AbstractGNUSourceCodeParser.java:1283)
	at org.eclipse.cdt.internal.core.dom.parser.AbstractGNUSourceCodeParser.translationUnit(AbstractGNUSourceCodeParser.java:1278)
	at org.eclipse.cdt.internal.core.dom.parser.AbstractGNUSourceCodeParser.parse(AbstractGNUSourceCodeParser.java:650)
	at org.eclipse.cdt.core.dom.parser.AbstractCLikeLanguage.getASTTranslationUnit(AbstractCLikeLanguage.java:147)
	at org.eclipse.cdt.internal.core.pdom.AbstractIndexerTask.createAST(AbstractIndexerTask.java:308)
	at org.eclipse.cdt.internal.core.pdom.AbstractIndexerTask.createAST(AbstractIndexerTask.java:281)
	at org.eclipse.cdt.internal.core.pdom.AbstractIndexerTask.parseFile(AbstractIndexerTask.java:874)
	at org.eclipse.cdt.internal.core.pdom.AbstractIndexerTask.parseLinkage(AbstractIndexerTask.java:793)
	at org.eclipse.cdt.internal.core.pdom.AbstractIndexerTask.runTask(AbstractIndexerTask.java:371)
	at org.eclipse.cdt.internal.core.pdom.indexer.PDOMIndexerTask.run(PDOMIndexerTask.java:150)
	at org.eclipse.cdt.internal.core.pdom.indexer.PDOMRebuildTask.run(PDOMRebuildTask.java:85)
	at org.eclipse.cdt.internal.core.pdom.PDOMIndexerJob.run(PDOMIndexerJob.java:137)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)
Comment 1 Markus Schorn CLA 2011-11-02 06:00:26 EDT
Hi Marc,
the stack-trace suggests that the index gets corrupted, which is a severe issue. I need to figure out why it happens, however I have not seen the problem with my installations. 
Do you get that with CDT 8.0.1 or on the master branch? Did you have the problem before (e.g. with CDT 8.0)?
Comment 2 Marc-André Laperle CLA 2011-11-03 02:54:42 EDT
I am using the master branch but now I can't reproduce this using the latest build 8.1.0.201111021006. However, I can reproduce this using 8.0.0. I will try to find the exact commit that fixed the problem to make sure the change is appropriate.

As a test, I use the gtk+ project and I change between branches outside Eclipse, between master and gtk-2-10. Refresh on access needs to be turned on as far as I can tell. Then I rebuild the index and a large number of errors appear in the error log.
Comment 3 Markus Schorn CLA 2011-11-03 08:06:15 EDT
(In reply to comment #2)
I have reviewed the source code and I am able to reproduce the failure with a specific sequence of indexing with an urgent task.
The issue has been introduced with the implementation for bug 287907 and was fixed with bug 197989.
The issue is present since 7.0 and became more likely to happen because of refresh on demand. 

To reproduce in the debugger:
* Create 3 headers h1.h - h3.h
     h1.h includes h2.h
     h2.h includes h3.h
* Create 3 source files s1.cpp - s3.cpp 
     s1.cpp, s2.cpp, s3.cpp include h1.h
* Set CModelListener.sSuppressUpdateOfLastRecentlyUsed to true
* Breakpoint in AbstractIndexerTask.parseLinkage(...)
* Rebuild index.
* Let the indexer index s1.cpp and s2.cpp
* In the editor remove include from s2.h to s3.h and save it.
* Let the indexer run (it injects an urgent task) 
--> The exception is thrown.
Comment 4 Markus Schorn CLA 2011-11-03 08:24:39 EDT
Fixed the issue and manually tested as outlined in comment 3.

Marc-Andre, please verify whether my change actually fixes the problem in your environment.
Comment 5 CDT Genie CLA 2011-11-03 09:23:05 EDT
*** cdt git genie on behalf of Markus Schorn ***

    Bug 360710: Index corruption due to urgent task.

[*] http://git.eclipse.org/c/cdt/org.eclipse.cdt.git/commit/?id=92955735be10ef645dd8ea793a8acdbd0045f677
Comment 6 Marc-André Laperle CLA 2011-11-08 15:01:39 EST
(In reply to comment #4)
> Marc-Andre, please verify whether my change actually fixes the problem in your
> environment.

I tested before and after the change in 8.0.2 and I confirm that it fixes the problem in my environment. I also double checked with 8.1 (master) and it works there too. Thank you!
Comment 7 Markus Schorn CLA 2011-11-09 01:19:35 EST
(In reply to comment #6)
> (In reply to comment #4)
> > Marc-Andre, please verify whether my change actually fixes the problem in your
> > environment.
> I tested before and after the change in 8.0.2 and I confirm that it fixes the
> problem in my environment. I also double checked with 8.1 (master) and it works
> there too. Thank you!

Perfect.