Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 269838

Summary: CDT gdb/mi fails to parse result of gdb cmd "thread info"
Product: [Tools] CDT Reporter: Tobias Hahn <tobias.hahn>
Component: cdt-debug-cdi-gdbAssignee: Nobody - feel free to take it <nobody>
Status: RESOLVED WONTFIX QA Contact: Elena Laskavaia <elaskavaia.cdt>
Severity: normal    
Priority: P3 CC: ken.ryall, malaperle, marc.khouzam, mike.jackson, nobody, pawel.1.piech
Version: 6.0   
Target Milestone: ---   
Hardware: Macintosh   
OS: Mac OS X - Carbon (unsup.)   
Whiteboard:
Attachments:
Description Flags
A patch which WFM.
none
A patch for the patch...
none
Override info threads with -thread-list-ids for Mac command set
none
mylyn/context/zip
none
Override info threads with -thread-list-ids for Mac command set
none
pthreads test code
none
gdb traces
none
DSF threads patch
none
DSF threads patch
none
Updated CDI patch, fixes threads stacks
none
Screenshot of the thread stack problem
none
Updated CDI patch, fixes threads stacks none

Description Tobias Hahn CLA 2009-03-24 11:56:11 EDT
Build ID: I20090313-0100

Steps To Reproduce:
1. Debug any multithreaded c/c++ project.
2. Interrupt inferior.

Expected results:
The debug view shows all threads of the inferior with correct thread ids.

Actual results:
The debug view shows only the current thread with incorrect thread id '0'. 

More information:
As far as I can tell, the bug is in org.eclipse.cdt.debug.mi.core.output.CLIInfoThreadsInfo.parse()

This method expects the result in 
    MIOutput out = getMIOutput(); 
    MIOOBRecord[] oobs = out.getMIOOBRecords();

Inspecting oobs, one sees that it is an array of oobs.length == 1 containing the cstring "info threads\\n" in oobs[1].

The actual thread info, however, is contained in the MIResultRecord out.getMIResultRecord(), which is never even looked at by parse.

Therefore, it defaults to assuming that there is just a single thread with id 0.

Regression:
This bug occurs on org.eclipse.cdt.debug.mi.core version 6.0.0.200903161435 on Mac OSX with gdb 6.3.50-20050815 (Apple version gdb-962), standard process launcher, gdb/mi, Default (Mac OS) command set.

It does not occur when using the DSF launcher (which has more serious issues in combination with Apple's gdb, so using the DSF launcher is not an alternative, unfortunately).
Comment 1 Marc Khouzam CLA 2009-03-24 14:25:35 EDT
(In reply to comment #0)
> It does not occur when using the DSF launcher (which has more serious issues in
> combination with Apple's gdb, so using the DSF launcher is not an alternative,
> unfortunately).

Besides the unability to interrupt the inferior, what other serious issues do you have?  It should be possible to improve the situation with DSF-GDB 

Comment 2 Tobias Hahn CLA 2009-03-24 15:06:43 EDT
Created attachment 129741 [details]
A patch which WFM.

This patch uses -thread-list-ids to obtain the currently running threads. It works in my setup (OSX, apple-gdb), but please review if this is correct for other archs, too.
Comment 3 Tobias Hahn CLA 2009-03-24 15:29:40 EDT
(In reply to comment #1)
> Besides the unability to interrupt the inferior, what other serious issues do
> you have?  It should be possible to improve the situation with DSF-GDB 

Well, actually, DSF-GDB would also benefit from the above patch, since it has the same issue when it stops. The advantage is that DSF-GDB has a refresh button, which in turn uses -thread-list-ids, and only after pressing refresh, you see all threads.

The most problematic issue with DSF-GDB is that it cannot update its variables when stepping. Initially, i.e. when hitting a breakpoint, the locals show up fine. But then when single stepping, it uses 

-var-update 1 var1
-var-update 1 var2

etc. to update them. Apple-gdb does not understand this and responds with

^error,msg="Variable object not found",changelist=[]

and the variables in the view are replaced with the error message

Failed to execute MI command:
-var-update 1 var2
Error message from debugger back end:
Variable object not found

Combined with not being able to interrupt, that's a showstopper for me. I'd be happy to try again and see what other issues might come up if you are able to fix these.

I guess most of the pain with cdt and gdb comes from the fact that apple hasn't merged mainline gdb for almost four years now. It would be so wonderful if cdt were able to fully support it...

Comment 4 Tobias Hahn CLA 2009-03-25 15:57:55 EDT
Created attachment 129886 [details]
A patch for the patch...

Sorry, the last one was a bit too fast and created an infinite recursion.
Comment 5 Elena Laskavaia CLA 2009-03-31 12:52:43 EDT
Can you post your mi-session log before the patch?
Is it mac os specific? I have not seen such problem with Linux and Windows...
Comment 6 Marc-André Laperle CLA 2009-11-10 21:54:51 EST
Created attachment 151905 [details]
Override info threads with -thread-list-ids for Mac command set

I used a different approach. I left the common code in Target.java intact and created a new command to replace info threads for the Mac OS command set. Let me know what you think!
Comment 7 Marc-André Laperle CLA 2009-11-10 21:54:55 EST
Created attachment 151906 [details]
mylyn/context/zip
Comment 8 Mike Jackson CLA 2010-01-04 22:31:59 EST
(In reply to comment #6)
> Created an attachment (id=151905) [details]
> Override info threads with -thread-list-ids for Mac command set
> 
> I used a different approach. I left the common code in Target.java intact and
> created a new command to replace info threads for the Mac OS command set. Let
> me know what you think!

I tried out the patch and I can set a break point in a secondary thread and the debugger will stop, but the list of threads only has one element, labeled "Thread-0". I know there is at least 2 threads running when the break point gets hit so something may still need to be coded up in the patch. Great so far though.
Comment 9 Marc-André Laperle CLA 2010-01-06 00:47:00 EST
Created attachment 155392 [details]
Override info threads with -thread-list-ids for Mac command set

(In reply to comment #8)
> (In reply to comment #6)
> > Created an attachment (id=151905) [details] [details]
> > Override info threads with -thread-list-ids for Mac command set
> > 
> > I used a different approach. I left the common code in Target.java intact and
> > created a new command to replace info threads for the Mac OS command set. Let
> > me know what you think!
> 
> I tried out the patch and I can set a break point in a secondary thread and the
> debugger will stop, but the list of threads only has one element, labeled
> "Thread-0". I know there is at least 2 threads running when the break point
> gets hit so something may still need to be coded up in the patch. Great so far
> though.

That's weird, this sounds like the original problem. Are you sure you used patch 151905 and used the Mac OS command set in the debug configuration? When I try a simple pthreads program, I get 2 threads in the list. I made a new patch to fix a problem with the stack of each thread but it won't fix what you mentioned. I'll also attach the code I use to test.
Comment 10 Marc-André Laperle CLA 2010-01-06 00:54:19 EST
Created attachment 155393 [details]
pthreads test code
Comment 11 Mike Jackson CLA 2010-01-06 09:16:11 EST
(In reply to comment #9)

> 
> That's weird, this sounds like the original problem. Are you sure you used
> patch 151905 and used the Mac OS command set in the debug configuration? When I
> try a simple pthreads program, I get 2 threads in the list. I made a new patch
> to fix a problem with the stack of each thread but it won't fix what you
> mentioned. I'll also attach the code I use to test.

I didn't have the debugger setup correctly. I had set the debugger to the "gdb" option. After running the test code and comparing the debug configurations, I saw that I should change it to "gdb/mi". After doing that and then debugging my original code I do get the correct listing of threads. So that patch does seem to work.
Comment 12 Marc Khouzam CLA 2010-01-15 06:02:32 EST
(In reply to comment #3)

> Well, actually, DSF-GDB would also benefit from the above patch, since it has
> the same issue when it stops. The advantage is that DSF-GDB has a refresh
> button, which in turn uses -thread-list-ids, and only after pressing refresh,
> you see all threads.

So, this bug is still valid for DSF-GDB?  Is there a patch for it?
Comment 13 Marc-André Laperle CLA 2010-01-15 14:01:18 EST
(In reply to comment #12)
> So, this bug is still valid for DSF-GDB?  Is there a patch for it?

CDI uses "info threads" to generate a thread id list and to get the current thread id which doesn't work on Mac because the format is different and doesn't include the current thread id. DSF uses "-thread-list-ids" to generate the list which is parsed correctly and I don't see the current thread id being used in DSF. So, I'm not sure how DSF is affected by the wrong "info threads" parsing. From what I tested, the threads appear correctly in 7.0 but not in 6.0.2, I'm not sure why. Any idea what might have changed? Maybe it would be possible to backport the fix.

For reference, here's a comparison of info threads:

FSF-GDB:

info threads
&"info threads\n"
~"  4 Thread 0xb6a71b70 (LWP 16662)  0xb7fe2430 in __kernel_vsyscall ()\n"
~"* 1 Thread 0xb7e75b10 (LWP 16657)  main () at foo.c:35\n"

Apple-GDB:

info threads
&"info threads\n"
^done,threadno="2",target_tid="port# 0x417",frame={func="aFunc",args={{name="foo",value="0x0"}},file="../pthread-test.c",fullname="/Users/mark/Documents/dev/workspaces/eclipsecarbondev/testThreads/pthread-test.c",line="11",dir="/Users/mark/Documents/dev/workspaces/eclipsecarbondev/testThreads/Debug"},threadno="1",target_tid="port# 0xa0f",frame={func="main",args={},file="../pthread-test.c",fullname="/Users/mark/Documents/dev/workspaces/eclipsecarbondev/testThreads/pthread-test.c",line="26",dir="/Users/mark/Documents/dev/workspaces/eclipsecarbondev/testThreads/Debug"}

"info thread" includes the current thread id:

info thread
&"info thread\n"
~"Thread 2 has current state \"WAITING\"\n"
~"\tMach port #0x417 (gdb port #0x4123)\n"
~"\tframe 0: "
~"\tpthread ID: 0x1000a2000\n"
~"\tsystem-wide unique thread id: 0xa8f\n"
~"\ttotal user time: 0x2677a8\n"
~"\ttotal system time: 0x12cc8\n"
~"\tscaled cpu usage percentage: 0\n"
~"\tscheduling policy in effect: 0x1\n"
~"\trun state: 0x3 (WAITING)\n"
~"\tflags: 0x0\n"
~"\tnumber of seconds that thread has slept: 0\n"
~"\tcurrent priority: 31\n"
~"\tmax priority: 63\n"
~"\tsuspend count: 0.\n"
^done,frame={func="aFunc",args={{name="foo",value="0x0"}},file="../pthread-test.c",fullname="/Users/mark/Documents/dev/workspaces/eclipsecarbondev/testThreads/pthread-test.c",line="11",dir="/Users/mark/Documents/dev/workspaces/eclipsecarbondev/testThreads/Debug"}

The stopped event also includes the thread id:
*stopped,time={wallclock="0.00945",user="0.00078",system="0.00152",start="1263574093.844442",end="1263574093.853893"},reason="breakpoint-hit",commands="no",times="1",bkptno="3",thread-id="2"

So, the CDI patch uses the thread-id from the stopped event and replaces info threads with -thread-list-ids.
Comment 14 Marc-André Laperle CLA 2010-01-15 14:34:30 EST
(In reply to comment #13)
> CDI uses "info threads" to generate a thread id list and to get the current
> thread id which doesn't work on Mac because the format is different and doesn't
> include the current thread id.

I just discovered that using -interpreter-exec console "info threads" gives a more useful output:

-interpreter-exec console "info threads"
* 2 port# 0x417 aFunc (foo=0x0) at ../pthread-test.c:11
  1 port# 0xa0f main () at ../pthread-test.c:26

Still, it is not parsed correctly. We could patch it in 7.0 but the threads are already displayed properly there. I might change the CDI patch, though.
Comment 15 Marc Khouzam CLA 2010-01-18 22:53:22 EST
(In reply to comment #13)

Sorry for the delay.

> (In reply to comment #12)
> > So, this bug is still valid for DSF-GDB?  Is there a patch for it?
> 
> CDI uses "info threads" to generate a thread id list and to get the current
> thread id which doesn't work on Mac because the format is different and doesn't
> include the current thread id. DSF uses "-thread-list-ids" to generate the list
> which is parsed correctly and I don't see the current thread id being used in
> DSF. 
> So, I'm not sure how DSF is affected by the wrong "info threads" parsing.
> From what I tested, the threads appear correctly in 7.0 but not in 6.0.2, I'm
> not sure why. Any idea what might have changed? Maybe it would be possible to
> backport the fix.

Are you using the same GDB for both tests?  I wouldn't expect to have a different behavior for this case.  Can you attach the "gdb traces" for both cases?

 
> For reference, here's a comparison of info threads:
> 
> FSF-GDB:
> 
> info threads
> &"info threads\n"
> ~"  4 Thread 0xb6a71b70 (LWP 16662)  0xb7fe2430 in __kernel_vsyscall ()\n"
> ~"* 1 Thread 0xb7e75b10 (LWP 16657)  main () at foo.c:35\n"
> 
> Apple-GDB:
> 
> info threads
> &"info threads\n"
> ^done,threadno="2",target_tid="port#
> 0x417",frame={func="aFunc",args={{name="foo",value="0x0"}},file="../pthread-test.c",fullname="/Users/mark/Documents/dev/workspaces/eclipsecarbondev/testThreads/pthread-test.c",line="11",dir="/Users/mark/Documents/dev/workspaces/eclipsecarbondev/testThreads/Debug"},threadno="1",target_tid="port#
> 0xa0f",frame={func="main",args={},file="../pthread-test.c",fullname="/Users/mark/Documents/dev/workspaces/eclipsecarbondev/testThreads/pthread-test.c",line="26",dir="/Users/mark/Documents/dev/workspaces/eclipsecarbondev/testThreads/Debug"}
> 
> "info thread" includes the current thread id:

I'm confused.  What is the output above compared to the one below?  How did you get different outputs?

> info thread
> &"info thread\n"
> ~"Thread 2 has current state \"WAITING\"\n"
> ~"\tMach port #0x417 (gdb port #0x4123)\n"
> ~"\tframe 0: "
> ~"\tpthread ID: 0x1000a2000\n"
> ~"\tsystem-wide unique thread id: 0xa8f\n"
> ~"\ttotal user time: 0x2677a8\n"
> ~"\ttotal system time: 0x12cc8\n"
> ~"\tscaled cpu usage percentage: 0\n"
> ~"\tscheduling policy in effect: 0x1\n"
> ~"\trun state: 0x3 (WAITING)\n"
> ~"\tflags: 0x0\n"
> ~"\tnumber of seconds that thread has slept: 0\n"
> ~"\tcurrent priority: 31\n"
> ~"\tmax priority: 63\n"
> ~"\tsuspend count: 0.\n"
> ^done,frame={func="aFunc",args={{name="foo",value="0x0"}},file="../pthread-test.c",fullname="/Users/mark/Documents/dev/workspaces/eclipsecarbondev/testThreads/pthread-test.c",line="11",dir="/Users/mark/Documents/dev/workspaces/eclipsecarbondev/testThreads/Debug"}
>
Comment 16 Marc-André Laperle CLA 2010-01-19 01:03:13 EST
Created attachment 156477 [details]
gdb traces

(In reply to comment #15)
> (In reply to comment #13)
> 
> Sorry for the delay.
> 
> > (In reply to comment #12)
> > > So, this bug is still valid for DSF-GDB?  Is there a patch for it?
> > So, I'm not sure how DSF is affected by the wrong "info threads" parsing.
> > From what I tested, the threads appear correctly in 7.0 but not in 6.0.2, I'm
> > not sure why. Any idea what might have changed? Maybe it would be possible to
> > backport the fix.
> 

> Are you using the same GDB for both tests?  I wouldn't expect to have a
> different behavior for this case.  Can you attach the "gdb traces" for both
> cases?

Yes, Apple's gdb-1346. I'm testing the previously attached pthreads code. I can reproduce the problem on both versions now. If I set a breakpoint before the second thread is created and then break into the second thread, it will only show one thread in the debug session unless I press the refresh button. So, it looks like gdb doesn't notify about the new thread and it shows up when refresh is pressed because -thread-list-ids is sent. Should -thread-list-ids be sent more often? I'm not sure what is the solution. 

About the attached traces, in this session, I put a break point before the second thread is created and another one in the second thread. When I hit the second break point, only one thread is displayed.

> I'm confused.  What is the output above compared to the one below?  How did you
> get different outputs?

Sorry I forgot some details. The output under FSF-GDB is the output from a "normal" gdb (7.0) running on Ubuntu 9.10. Under Apple-GDB is the output from the same command with Apple's gdb-1346. The last one is also from gdb-1346 but is "info thread" not "info threads". But using interpreter-exec with gdb-1346, we can get a similar output compared to the "normal" FSF-GDB output. It doesn't look like the wrong output from info threads is the what's affecting DSF but I think we should fix info threads by parsing the interpreter-exec output. I can see the "isCurrentThread" being used at some point. Also, it would lead to a cleaner CDI patch.
Comment 17 Marc-André Laperle CLA 2010-01-19 01:07:06 EST
(In reply to comment #16)
> About the attached traces, in this session, I put a break point before the
> second thread is created and another one in the second thread. When I hit the
> second break point, only one thread is displayed.

This is with CDT 6.0.2 but both versions have the problem.
Comment 18 Marc-André Laperle CLA 2010-01-26 00:14:43 EST
(In reply to comment #16)
> It doesn't
> look like the wrong output from info threads is the what's affecting DSF but I
> think we should fix info threads by parsing the interpreter-exec output. I can
> see the "isCurrentThread" being used at some point. Also, it would lead to a
> cleaner CDI patch.

I did a quick and dirty parsing of info threads with DSF and I can confirm that is doesn't fix the problem. I also tried to improve the CDI patch as I mentioned but MIInfo doesn't provide the necessary (abnormal) output so it can't be parsed. I guess the CDI patch will stay the same. I'm not sure how to proceed on the DSF front. I opened a bug with Apple's bug reporter about the missing [New Thread ...] but I'm not really counting on this.
Comment 19 Marc-André Laperle CLA 2010-02-01 23:12:28 EST
Created attachment 157863 [details]
DSF threads patch

The problem is that -thread-list-ids is cached so it always returns the same result unless the cache is flushed. So, the patch adds a listener for suspend events and flushes the cache.
Comment 20 Marc-André Laperle CLA 2010-02-01 23:21:13 EST
Created attachment 157865 [details]
DSF threads patch

Minor changes to the previous patch.
Comment 21 Nobody - feel free to take it CLA 2010-02-03 09:56:42 EST
Marc or Marc-Andre, can you please move the dsf related patches and discussions into a separate bug?
Comment 22 Marc-André Laperle CLA 2010-02-03 12:29:12 EST
(In reply to comment #21)
> Marc or Marc-Andre, can you please move the dsf related patches and discussions
> into a separate bug?

Sorry about that. I opened bug 301720.
Comment 23 Nobody - feel free to take it CLA 2010-02-03 12:32:25 EST
Thanks.
Comment 24 Marc-André Laperle CLA 2010-02-03 22:40:38 EST
Created attachment 158135 [details]
Updated CDI patch, fixes threads stacks

This patch includes a little modification in EventManager but I believe it makes sense for all platforms.
Comment 25 Marc-André Laperle CLA 2010-02-04 11:20:42 EST
(In reply to comment #24)
> Created an attachment (id=158135) [details]
> Updated CDI patch, fixes threads stacks
> 
> This patch includes a little modification in EventManager but I believe it
> makes sense for all platforms.

Uh, I'm not sure about this, it looks like it breaks bug 250037.
Comment 26 Nobody - feel free to take it CLA 2010-02-05 09:31:44 EST
Marc-Andre, please describe the problem with examples for -threads-list-ids and info threads for Mac.
Comment 27 Marc-André Laperle CLA 2010-02-07 14:26:57 EST
Created attachment 158393 [details]
Screenshot of the thread stack problem

Example of -thread-list-ids:

307-thread-list-ids
307^done,thread-ids={thread-id="2",thread-id="1"},number-of-threads="2",threads=[thread={thread-id="2",state="WAITING",mach-port-number="0x417",pthread-id="0x1000a2000",unique-id="0x26eb"},thread={thread-id="1",state="WAITING",mach-port-number="0xa0f",pthread-id="0x7fff7033dbe0",unique-id="0x26dc",workqueue="com.apple.main-thread"}],time={wallclock="0.00049",user="0.00021",system="0.00028",start="1265567563.847715",end="1265567563.848202"}

Example of info threads:
info threads
&"info threads\n"
^done,reason="breakpoint-hit",commands="no",times="1",bkptno="1",thread-id="2",threadno="2",target_tid="port# 0x417",frame={func="aFunc",args={{name="foo",value="0x0"}},file="../pthread-test.c",fullname="/Users/mark/Documents/dev/workspaces/eclipsecarbondev/testThreads/pthread-test.c",line="9",dir="/Users/mark/Documents/dev/workspaces/eclipsecarbondev/testThreads/Debug"},threadno="1",target_tid="port# 0xa0f",frame={addr="0x00007fffffe007c5",fp="0x00007fff5fbff6b0",func="__memcpy",args={}}
(gdb)


Example of -interpreter-exec console "info threads": (notice how there are no '~')

330-interpreter-exec console "info threads"
  2 port# 0x417 aFunc (foo=0x0) at ../pthread-test.c:9
* 1 port# 0xa0f main () at ../pthread-test.c:25

Example of info thread :
&"info thread\n"
~"Thread 2 has current state \"WAITING\"\n"
~"\tMach port #0x417 (gdb port #0x411b)\n"
...

As you can see, -thread-list-ids and info threads do not contain the current thread id. I tried to parse the result of -interpreter-exec console "info threads" but the MIInfo output is incomplete (because of the missing ~ ?). In the previous patch, I attempted to fallback on the thread id provided by the stopped event but it breaks bug 250037.
Comment 28 Marc-André Laperle CLA 2010-02-07 14:42:13 EST
Created attachment 158394 [details]
Updated CDI patch, fixes threads stacks

A better patch that doesn't break bug 250037

In EventManager.processSuspendedEvent:

int threadId = stopped.getThreadId();
currentTarget.updateState(threadId);

In Target.updateState(int newThreadId):

//get the new Threads.
currentThreadId = newThreadId;
currentThreads = getCThreads();

In Target.getCThreads:

names = info.getThreadNames();
currentThreadId = info.getCurrentThread();

Which means that the current thread id from info will overwrite the one from the stopped event, even if it's invalid. So this patch sets the current thread in info threads always to 0 and changes Target.getCThreads to this:

names = info.getThreadNames();
if (info.getCurrentThread() != 0) {
    currentThreadId = info.getCurrentThread();
}
Comment 29 Marc-André Laperle CLA 2013-07-12 22:09:36 EDT
Marking as WONTFIX. There is not enough interest to fix this in CDI (and on Mac OS on top of that!).