Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 364188

Summary: Indexer crashes during build of LLVM + Clang
Product: [Tools] CDT Reporter: Jeff Sember <jeffeclipsebug>
Component: cdt-indexerAssignee: Markus Schorn <mschorn.eclipse>
Status: RESOLVED FIXED QA Contact: Markus Schorn <mschorn.eclipse>
Severity: critical    
Priority: P3 CC: cdtdoug, malaperle, yevshif
Version: 8.0.1   
Target Milestone: 8.1.0   
Hardware: All   
OS: All   
Whiteboard:
Bug Depends on: 364225, 364226    
Bug Blocks:    
Attachments:
Description Flags
Log file from index operation
none
Log file after removing clang, llvm:tests, and llvm:examples none

Description Jeff Sember CLA 2011-11-18 13:19:40 EST
Build Identifier: 20110916-0149

 
I'm attempting to build the llvm/clang C++ project, and eclipse fails while attempting to index the project.  I monitored the log file during the indexing attempt, and a number of exceptions appear while the indexer seems to be making progress.  Then the indexer appeared to hang: the files processed stopped increasing (at 3,254 of 4,710 sources).  After a few more minutes, Eclipse showed a dialog box "Problem Occurred: 'Update Monitor' has encountered a problem" with 'Java heap space' mentioned in the details.

I've attached the log file.  Note that there are several exceptions shown; the first set are ClassCastExceptions, while the last set seem to be memory problems caused by infinite recursions.

I noticed that bug numbers 253050 and 317004 also involve the indexer; they refer to a 'fast vs full' indexer option, which does not exist in my eclipse build.  I am using the default indexer options (it's a fresh install of eclipse on a Mac Mini I purchased today).  I also tried increasing the JVM memory settings (in eclipse.ini), but this seems to have no effect.


Reproducible: Always

Steps to Reproduce:
1. Select LLVM+Clang project
2. Right-click on project in Project Explorer, choose Index->Rebuild 
3.
Comment 1 Jeff Sember CLA 2011-11-18 13:20:47 EST
Created attachment 207242 [details]
Log file from index operation
Comment 2 Marc-André Laperle CLA 2011-11-18 20:35:15 EST
I'm assuming you are using 8.0.1 since you installed a "fresh" Eclipse with a build id of 20110916-0149. Is that right? (About > Installation Details)

How much memory did you try in eclipse.ini (-Xmx)? My Macbook just died but I'll test on Windows. I'm not very familiar with llvm/clang, are you trying to build both the source contained in llvm-2.9.tgz and clang-2.9.tgz in a single project? Or everything in svn trunk?
Comment 3 Marc-André Laperle CLA 2011-11-19 14:28:11 EST
I have both llvm and clang in a single project on Windows and I get the errors. The ClassCastException and the NullPointerException are definitely bugs. For the OutOfMemoryError, see bug 280196. I'm not sure about the StackOverflowError yet.

For the NPE, I isolated the code and opened bug 364225. I'll try to do the same for the ClassCastException and StackOverflowError.
Comment 4 Marc-André Laperle CLA 2011-11-19 14:55:22 EST
Opened bug 364226 for the ClassCastException.
Comment 5 Marc-André Laperle CLA 2011-11-19 15:57:05 EST
About the StackOverflowError, it occurs because of a huge number of compound statements. 

void foo()
{{{ ...... many more ...... }}}

Maybe the parser could abort when reaching a certain number of compound statements and warn instead of triggering a StackOverflowError. But I don't know what that number would be.
Comment 6 Jeff Sember CLA 2011-11-21 17:36:00 EST
(In reply to comment #2)
> I'm assuming you are using 8.0.1 since you installed a "fresh" Eclipse with a
> build id of 20110916-0149. Is that right? (About > Installation Details)

The id for both C/C++ Development Tools and C/C++ Development Tools SDK is:   8.0.0.201109151620 

 
> How much memory did you try in eclipse.ini (-Xmx)? My Macbook just died but
> I'll test on Windows.

I have the following settings in my eclipse.ini file:

-Xms500m
-Xmx500m

I believe I actually tried it with a higher value, 600m or even 800m, before setting it back to 500m; there was no improvement.

> I'm not very familiar with llvm/clang, are you trying to
> build both the source contained in llvm-2.9.tgz and clang-2.9.tgz in a single
> project? Or everything in svn trunk?

To simplify things, after posting the bug report, I tried installing LLVM without the Clang subproject.  I also told the CMake program to omit the 'tests' and 'examples' subprojects. Now I think the indexing completes without hanging, though there are still some errors reported in the log file.  I can prepare a new bug report (or at least provide more details) for this simplified scenario if you like.
Comment 7 Marc-André Laperle CLA 2011-11-21 22:32:13 EST
(In reply to comment #6)
> I can
> prepare a new bug report (or at least provide more details) for this simplified
> scenario if you like.

Can you attach a log file that contains the new errors? Thanks!
Comment 8 Jeff Sember CLA 2011-11-22 15:05:01 EST
(In reply to comment #7)
> Can you attach a log file that contains the new errors? Thanks!

Hi Marc-Andre,

Attached is the log file.  It's only showing the one problem now. As mentioned earlier, this build omits both the example and test components.
Comment 9 Jeff Sember CLA 2011-11-22 15:07:26 EST
Created attachment 207386 [details]
Log file after removing clang, llvm:tests, and llvm:examples

Log file indicates only a single crash
Comment 10 Marc-André Laperle CLA 2011-11-22 19:51:43 EST
(In reply to comment #8)
> (In reply to comment #7)
> > Can you attach a log file that contains the new errors? Thanks!
> 
> Hi Marc-Andre,
> 
> Attached is the log file.  It's only showing the one problem now. As mentioned
> earlier, this build omits both the example and test components.

I'm pretty sure the error is the same NPE as in bug 364225. I think that covers everything, thank you Jeff!
Comment 11 Marc-André Laperle CLA 2012-01-22 11:47:47 EST
(In reply to comment #5)
> About the StackOverflowError, it occurs because of a huge number of compound
> statements. 
> 
> void foo()
> {{{ ...... many more ...... }}}
> 
> Maybe the parser could abort when reaching a certain number of compound
> statements and warn instead of triggering a StackOverflowError. But I don't
> know what that number would be.

Markus, do you think it would be worthwhile to add a limit to the number of compound statements? I wasn't sure if I should create a bug for that.
Comment 12 Markus Schorn CLA 2012-01-23 03:58:47 EST
(In reply to comment #11)
> (In reply to comment #5)
> > About the StackOverflowError, it occurs because of a huge number of compound
> > statements. 
> > 
> > void foo()
> > {{{ ...... many more ...... }}}
> > 
> > Maybe the parser could abort when reaching a certain number of compound
> > statements and warn instead of triggering a StackOverflowError. But I don't
> > know what that number would be.
> Markus, do you think it would be worthwhile to add a limit to the number of
> compound statements? I wasn't sure if I should create a bug for that.

When the stack overflow occurs, the indexer skips the file causing it. So, the stack overflow already aborts the parser when the nesting becomes too deep. I don't see how a fixed limit would improve this behavior. 
With the current treatment, the user can increase the stack size (e.g: -Xss1m) to allow for deeper nestings.
Comment 13 Marc-André Laperle CLA 2013-04-23 00:26:32 EDT
(In reply to comment #12)
> (In reply to comment #11)
> > (In reply to comment #5)
> > > About the StackOverflowError, it occurs because of a huge number of compound
> > > statements. 
> > > 
> > > void foo()
> > > {{{ ...... many more ...... }}}
> > > 
> > > Maybe the parser could abort when reaching a certain number of compound
> > > statements and warn instead of triggering a StackOverflowError. But I don't
> > > know what that number would be.
> > Markus, do you think it would be worthwhile to add a limit to the number of
> > compound statements? I wasn't sure if I should create a bug for that.
> 
> When the stack overflow occurs, the indexer skips the file causing it. So,
> the stack overflow already aborts the parser when the nesting becomes too
> deep. I don't see how a fixed limit would improve this behavior. 

You're right. Resolving.