Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 263689

Summary: [commands] Deadlock: can't debug HelloWorld if Project is in directory with space
Product: [Tools] CDT Reporter: James Blackburn <jamesblackburn+eclipse>
Component: cdt-debug-dsf-gdbAssignee: Marc Khouzam <marc.khouzam>
Status: RESOLVED FIXED QA Contact: Marc Khouzam <marc.khouzam>
Severity: normal    
Priority: P3 CC: malaperle, nobody, pawel.1.piech
Version: 6.0   
Target Milestone: 7.0   
Hardware: PC   
OS: Mac OS X - Carbon (unsup.)   
Whiteboard:
Attachments:
Description Flags
Backtrace
none
backtrace 2
none
MI log
none
CDI-only patch
none
Using 'cd' instead of "-environment-cd"
marc.khouzam: iplog-
Path for paths with and without spaces
none
Another fix marc.khouzam: iplog-

Description James Blackburn CLA 2009-02-04 15:16:45 EST
Created attachment 124726 [details]
Backtrace

Build ID: 6.0M5

Steps To Reproduce:
Testing the 6.0M5 build on Mac OS X, I can't launch a debug session using DSF-GDB on HelloWorld if the path to the Project has a space in it.  The failure to launch the debugger leads to a hang of the IDE.

CDI has a similar issue -- it looks like Mac OS GDB isn't handling the quoted path correctly.

In DSF the launch never proceeds and Eclipse/CDT hangs. In CDI I can debug as normal...

It seems the crash is related to clicking on the Debug process in the Debug View. Backtrace attached.


More information:
Request for monitor: 'RequestMonitor (org.eclipse.cdt.dsf.concurrent.RequestMonitorWithProgress@251664): Status ERROR: org.eclipse.cdt.dsf.gdb code=10004 Failed to execute MI command:-environment-cd "/Users/james/Desktop/Families Together/hw"
Error message from debugger back end:
/Users/james/Desktop/Families: No such file or directory. null' resulted in an error.
Comment 1 Marc Khouzam CLA 2009-02-16 10:00:20 EST
I tried to create a path with a space in Linux.
So, I created a new workspace called "runtime with space" and I created a HelloWorld C project through the wizard.
When I try to open the src folder, my entire Eclipse hangs!
I couldn't even get to the debugging part.

but from the backend error you show, I'll try it straight with GDB and see what happens.
Comment 2 James Blackburn CLA 2009-02-16 10:04:47 EST
(In reply to comment #1)
> I tried to create a path with a space in Linux.
> So, I created a new workspace called "runtime with space" and I created a
> HelloWorld C project through the wizard.
> When I try to open the src folder, my entire Eclipse hangs!
> I couldn't even get to the debugging part.

Eek! I know it's a bad idea to use spaces in general, but we certainly shouldn't hang.

> but from the backend error you show, I'll try it straight with GDB and see what
> happens.

I thought the backtrace might be useful. The UI thread appears to be blocked on a queue, and never returns. Isn't it a bad idea to lock up the UI in this way -- especially if this is dependent on the output of an external process? (Apart from anything else you might expect this to be a performance issue?)
Comment 3 James Blackburn CLA 2009-02-16 10:14:02 EST
(In reply to comment #1)
> When I try to open the src folder, my entire Eclipse hangs!
> I couldn't even get to the debugging part.

I couldn't reproduce this (on Linux), could you file a bug with the a backtrace Marc?  My IDE's not quite on HEAD, but it does have all my project model changes we made. It'd be good to work out the root cause of your hang was... 

Comment 4 Marc Khouzam CLA 2009-02-16 10:17:24 EST
(In reply to comment #3)
> (In reply to comment #1)
> > When I try to open the src folder, my entire Eclipse hangs!
> > I couldn't even get to the debugging part.
> 
> I couldn't reproduce this (on Linux), could you file a bug with the a backtrace
> Marc?  My IDE's not quite on HEAD, but it does have all my project model
> changes we made. It'd be good to work out the root cause of your hang was... 

How do I trigger the backtrace once it hangs?
Thanks

Comment 5 James Blackburn CLA 2009-02-16 10:21:42 EST
(In reply to comment #4)
> How do I trigger the backtrace once it hangs?
> Thanks

On Unix you can do ctrl-\ to the eclipse process from the shell you started Eclipse on.  There are other ways too:
http://wiki.eclipse.org/index.php/How_to_report_a_deadlock

Comment 6 Marc Khouzam CLA 2009-02-16 11:12:21 EST
(In reply to comment #5)
> (In reply to comment #4)
> > How do I trigger the backtrace once it hangs?
> > Thanks
> 
> On Unix you can do ctrl-\ to the eclipse process from the shell you started
> Eclipse on.  There are other ways too:
> http://wiki.eclipse.org/index.php/How_to_report_a_deadlock
> 

Thanks!
I opened bug 265028
Comment 7 James Blackburn CLA 2009-06-16 14:52:12 EDT
Created attachment 139339 [details]
backtrace 2

Just to say this deadlock still occurs in RC4. Creating a project "hello world" then trying to debug it locks up every time (on Mac OS).  Attaching the backtrace from RC4 in case it's more helpful / up to date
Comment 8 Marc Khouzam CLA 2009-06-17 09:23:08 EDT
(In reply to comment #7)
> Created an attachment (id=139339) [details]
> backtrace 2
> 
> Just to say this deadlock still occurs in RC4. Creating a project "hello world"
> then trying to debug it locks up every time (on Mac OS).  Attaching the
> backtrace from RC4 in case it's more helpful / up to date

The connect button of the debug view uses a blocking call that seems to hang.  My guess is that the DSF executor is busy.

James, can you attach the gdb traces so I can see the output of GDB for this failed -environment-cd command?

Thanks

Comment 9 Marc Khouzam CLA 2009-06-17 09:37:04 EDT
On Linux, if I force an invalid -environment-cd command during the launch, I end up in a launch that is stuck and that I cannot kill, but the GUI is still responsive.
Comment 10 Marc Khouzam CLA 2009-06-17 11:08:46 EDT
(In reply to comment #9)
> On Linux, if I force an invalid -environment-cd command during the launch, I
> end up in a launch that is stuck and that I cannot kill, but the GUI is still
> responsive.

I've opened bug 280626 about this issue, which is different than the current bug.

Comment 11 James Blackburn CLA 2009-06-17 13:41:03 EDT
Created attachment 139457 [details]
MI log

It locks up properly on MacOS. I got the MI trace by turning on the dsf gdb debug switch.
Comment 12 Marc-André Laperle CLA 2009-11-06 22:44:20 EST
I downloaded apple's sources and while debugging their gdb I noticed that 
-environment-cd "/Users/james/Documents/eclipse/runtime-New_configuration/hello world" 
is converted to the CLI command
"cd  /Users/james/Documents/eclipse/runtime-New_configuration/hello world"
but it should be 
"cd  \"/Users/james/Documents/eclipse/runtime-New_configuration/hello world\""
so if you do
-environment-cd "\"/Users/james/Documents/eclipse/runtime-New_configuration/hello world\"" 
it gets converted to the correct CLI command and it works! But I don't see how to change this in dsf.gdb since there are no command.factories like in the standard one. Any ideas?
Comment 13 Marc-André Laperle CLA 2010-01-10 16:42:28 EST
Created attachment 155688 [details]
CDI-only patch

Patch for CDI 6.0.2 and 6.1 (aka 7.0). It looks like DSF would need API changes to fix this one. I can't use the new services factory since the problematic command is instantiated not in a service but in the launch (MIEnvironmentCD in FinalLaunchSequence).
Comment 14 Nobody - feel free to take it CLA 2010-02-03 10:43:13 EST
Moved the CDI patch to https://bugs.eclipse.org/bugs/show_bug.cgi?id=301706.
Comment 15 Marc Khouzam CLA 2010-02-04 14:09:57 EST
(In reply to comment #13)
> It looks like DSF would need API changes
> to fix this one. I can't use the new services factory since the problematic
> command is instantiated not in a service but in the launch (MIEnvironmentCD in
> FinalLaunchSequence).

One way forward would be to have the FinalLaunchSequence call a service to do the MIEnvironmentCD command.  The you could use the services factory.

In truth, the FinalLaunchSequence should not issue MI commands directly but should use services for everything.  We just never got around to fixing that.
Comment 16 Marc Khouzam CLA 2010-02-24 16:22:35 EST
Created attachment 160124 [details]
Using 'cd' instead of "-environment-cd"

(In reply to comment #12)
> I downloaded apple's sources and while debugging their gdb I noticed that

Good work debugging GDB.
 
> -environment-cd "/Users/james/Documents/eclipse/runtime-New_configuration/hello
> world" 
> is converted to the CLI command
> "cd  /Users/james/Documents/eclipse/runtime-New_configuration/hello world"
> but it should be 
> "cd  \"/Users/james/Documents/eclipse/runtime-New_configuration/hello world\""
> so if you do
> -environment-cd
> "\"/Users/james/Documents/eclipse/runtime-New_configuration/hello world\"" 
> it gets converted to the correct CLI command and it works! 

Was this actually broken in CDI?  CDI was using 'cd' and not '-environment-cd', so maybe apple's GDB was handling it properly?

I'm asking because the simplest solution for DSF-GDB may simply be to use 'cd' instead of '-environment-cd'.  Although it is not very pretty, it would allow things work on all platforms without having to special case MacOS.

With the attached patch, things seem to work well on Linux, even if I have a space in my project name.

Can someone try it on Mac?
Comment 17 Marc-André Laperle CLA 2010-02-25 02:19:04 EST
(In reply to comment #16)
> Was this actually broken in CDI?  CDI was using 'cd' and not '-environment-cd',
> so maybe apple's GDB was handling it properly?

On Mac OS, from what I tested, the command needs to be 
cd "spaces in path"
or 
-environment-cd "\"spaces in path\""

Come to think of it, I'm not even sure it was broken in CDI since it used the first one. Both commands work for me in CDI at the moment. Maybe there's a special case I'm forgetting.

> I'm asking because the simplest solution for DSF-GDB may simply be to use 'cd'
> instead of '-environment-cd'.  Although it is not very pretty, it would allow
> things work on all platforms without having to special case MacOS.
> 
> With the attached patch, things seem to work well on Linux, even if I have a
> space in my project name.
> 
> Can someone try it on Mac?

This patch doesn't work for me (gdb-1346). The command is issued without the quotes and it is an error, although gdb doesn't report it (recall that recent apple-gdb versions will not output errors for CLI commands). Then again, adding quotes doesn't work on Linux. I don't think this simple solution is going to work, the command probably needs to be done in a service.
Comment 18 Marc Khouzam CLA 2010-02-25 10:39:32 EST
(In reply to comment #17)
> (In reply to comment #16)
> > Was this actually broken in CDI?  CDI was using 'cd' and not '-environment-cd',
> > so maybe apple's GDB was handling it properly?
> 
> On Mac OS, from what I tested, the command needs to be 
> cd "spaces in path"
> or 
> -environment-cd "\"spaces in path\""
> 
> Come to think of it, I'm not even sure it was broken in CDI since it used the
> first one. Both commands work for me in CDI at the moment. Maybe there's a
> special case I'm forgetting.

From the fact that MacOSMIEnvironmentCD existed, we can deduce that it worked.  You just found another way to do it using -environment-cd instead of plain cd.  You way is better because this class inherits from MICommand and not CLICommand.

> > I'm asking because the simplest solution for DSF-GDB may simply be to use 'cd'
> > instead of '-environment-cd'.  Although it is not very pretty, it would allow
> > things work on all platforms without having to special case MacOS.
> > 
> > With the attached patch, things seem to work well on Linux, even if I have a
> > space in my project name.
> > 
> > Can someone try it on Mac?
> 
> This patch doesn't work for me (gdb-1346). The command is issued without the
> quotes and it is an error, 
> Then again, adding quotes doesn't work on Linux. 

Now I understand that for MacOS, (and only MacOS) we have to force quotes, which is why CDI had MacOSMIEnvironmentCD in the first place.

> I don't think this simple solution is going to
> work, the command probably needs to be done in a service.

You're right, I don't see a unified solution.  Too bad...

Thanks for trying it out.
Comment 19 Marc Khouzam CLA 2010-02-27 22:35:57 EST
I've created a CommandFactory as part of bug 304146.  The patch of that bug also contains a solution to this current bug, as a proof-of-concept.

Marc-Andre, could you try the patch "proposed solution" of bug 304146 on MacOS when you have a chance, and let me know if it fixed the current bug?  Thanks.
Comment 20 Marc-André Laperle CLA 2010-02-28 02:52:33 EST
(In reply to comment #19)
> I've created a CommandFactory as part of bug 304146.  The patch of that bug
> also contains a solution to this current bug, as a proof-of-concept.
> 
> Marc-Andre, could you try the patch "proposed solution" of bug 304146 on MacOS
> when you have a chance, and let me know if it fixed the current bug?  Thanks.

This patch works for me. The new CommandFactory should be very useful!
Comment 21 Marc Khouzam CLA 2010-02-28 11:48:48 EST
(In reply to comment #20)
> (In reply to comment #19)
> > I've created a CommandFactory as part of bug 304146.  The patch of that bug
> > also contains a solution to this current bug, as a proof-of-concept.
> > 
> > Marc-Andre, could you try the patch "proposed solution" of bug 304146 on MacOS
> > when you have a chance, and let me know if it fixed the current bug?  Thanks.
> 
> This patch works for me. The new CommandFactory should be very useful!

Great!  Thanks for the quick test.
Comment 22 Marc Khouzam CLA 2010-03-03 09:55:46 EST
Fixed as part of bug 304146

However, I don't think anyone but James has seen the deadlock, so it's hard to know if the deadlock itself still occurs.  I'm hoping not.

I'll mark the bug as fixed, but James, if you get the deadlock, just re-open.
Comment 23 Marc Khouzam CLA 2010-03-03 18:55:33 EST
This fix failed after the commit of bug 304146.  
The below is copied from bug 304146 comment #9:

There is a bug when MIEnvironmentCD/MacOSMIEnvironmentCD is executed. It worked
when I tried the first patch. Could something else have changed in the
meantime?

The command executed is now:

-environment-cd \"/Users/mark/.../workspace/testDefines\"

but it should be (notice the quotes):

-environment-cd "\"/Users/mark/.../workspace/testDefines\""

I tried something like:
super(ctx, "\"\\\"" + path + "\\\"\"");

and it builds the correct string but it seems the quotes and backslashes are
escaped again further in the code because this is the resulting command:

-environment-cd \"\\\"/Users/mark/.../workspace/testDefines\\\"\"
Comment 24 Marc Khouzam CLA 2010-03-03 20:15:09 EST
(In reply to comment #23)
> This fix failed after the commit of bug 304146.  
> The below is copied from bug 304146 comment #9:
> 
> There is a bug when MIEnvironmentCD/MacOSMIEnvironmentCD is executed. It worked
> when I tried the first patch. Could something else have changed in the
> meantime?
> 
> The command executed is now:
> 
> -environment-cd \"/Users/mark/.../workspace/testDefines\"
> 
> but it should be (notice the quotes):
> 
> -environment-cd "\"/Users/mark/.../workspace/testDefines\""

Did the path you tried contain spaces?  The MICommand code automatically adds quotes when there are spaces in the option/parameter.

So, did the command work on MacOS but you didn't see the quotes, or did the command actually fail?
 
> I tried something like:
> super(ctx, "\"\\\"" + path + "\\\"\"");
> 
> and it builds the correct string but it seems the quotes and backslashes are
> escaped again further in the code because this is the resulting command:
> 
> -environment-cd \"\\\"/Users/mark/.../workspace/testDefines\\\"\"

That's the MICommand again which automatically escapes quotes.

One thing I've notices is that we don't properly build MICommands all the time. We often make everything an option instead of a parameter.  But I don't think it would cause a problem.
Comment 25 Marc Khouzam CLA 2010-03-03 20:27:24 EST
(In reply to comment #23)

I've tried on Windows, but triggering the MacOSCommandFactory.  GDB fails, but I see the following commands being sent:

(1) -environment-cd \"/path/without/any/spaces\"
(2) -environment-cd "\"/path/with spaces\""

For MacOS, do we need the double set of quotes of (2) all the time, or only when there are spaces?
Comment 26 Marc-André Laperle CLA 2010-03-03 20:36:46 EST
(In reply to comment #24)
> Did the path you tried contain spaces?  The MICommand code automatically adds
> quotes when there are spaces in the option/parameter.
> 
> So, did the command work on MacOS but you didn't see the quotes, or did the
> command actually fail?

Ok, when I tried the first patch, the path contained spaces, that's why it worked. It doesn't work for paths with no spaces.

(In reply to comment #25)
> For MacOS, do we need the double set of quotes of (2) all the time, or only
> when there are spaces?

Yes, they are needed, sorry I should have mentioned that. It should be:

-environment-cd "\"/path/without/any/spaces\""
-environment-cd "\"/path/with spaces\""

I'm not sure how I feel about MICommand modifing the string. There are probably other quirks with other gdb versions that will be harder to work around if MICommand does too much.
Comment 27 Marc-André Laperle CLA 2010-03-03 20:39:30 EST
Sorry, this would work too:

-environment-cd /path/without/any/spaces
Comment 28 Marc Khouzam CLA 2010-03-03 21:14:47 EST
(In reply to comment #26)

> Ok, when I tried the first patch, the path contained spaces, that's why it
> worked. It doesn't work for paths with no spaces.

Could you try with and without spaces with CDI and tell me what commands are being sent?  I'm wondering if the recent fix for CDI actually broke things...
Comment 29 Marc-André Laperle CLA 2010-03-03 21:23:54 EST
(In reply to comment #28)
> (In reply to comment #26)
> 
> > Ok, when I tried the first patch, the path contained spaces, that's why it
> > worked. It doesn't work for paths with no spaces.
> 
> Could you try with and without spaces with CDI and tell me what commands are
> being sent?  I'm wondering if the recent fix for CDI actually broke things...

Looks good with CDI:

-environment-cd "\"/Users/mark/Documents/dev/nospaces\""
-environment-cd "\"/Users/mark/Documents/dev/spaces in path\""

For DSF, we could do \" + path + \" only when there is a space.
Comment 30 Marc-André Laperle CLA 2010-03-03 21:28:30 EST
Created attachment 160879 [details]
Path for paths with and without spaces

This patch works for me for paths with and without spaces.
Comment 31 Marc Khouzam CLA 2010-03-03 21:44:54 EST
(In reply to comment #30)
> Created an attachment (id=160879) [details]
> Path for paths with and without spaces
> 
> This patch works for me for paths with and without spaces.

We could do that.  And I'm tempted.  But I'm worried there's a deeper problem.

Looking at CDI's MIEnvironmentCD, I see that they override the parameter handling, but we didn't copy that to DSF-GDB.  Same for MacOSMIEnvironmentCD.  But then WinMIEnvironmentCD reverts back to the original behavior.

We need to understand what CDI is doing, and do something similar.  MIBreakInsert has an example of overriding the parameter handling, but I'm not even sure about that one.

It's giving me a headache.  And it's too late at night for me.  Maybe tomorrow.
Comment 32 Marc Khouzam CLA 2010-03-04 14:34:11 EST
Created attachment 161013 [details]
Another fix

I've found Bug 57127 which explains why CDI overrides parametersToString() for MIEnvironmentCD.  However, as far as I can see, with GDB 6.5 and up, Linux is accepting the quotes, so I won't make the same override in DSF-GDB.

I've made the override only for MacOS.  So, this patch should always output

-environment-cd "\"path\""

with or without spaces in the path.

I've committed the patch to HEAD, since things were broken in HEAD anyway.

Marc-Andre, one more try?
Comment 33 Marc-André Laperle CLA 2010-03-04 17:20:07 EST
(In reply to comment #32)
> Created an attachment (id=161013) [details]
> Another fix
> 
> Marc-Andre, one more try?

It works now in both cases, thank you.
Comment 34 Marc Khouzam CLA 2010-03-04 17:58:32 EST
(In reply to comment #33)
> (In reply to comment #32)
> > Created an attachment (id=161013) [details] [details]
> > Another fix
> > 
> > Marc-Andre, one more try?
> It works now in both cases, thank you.

Thanks!