Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 530377

Summary: Fast succession of =breakpoint-created/deleted/modified messages corrupts breakpoints state
Product: [Tools] CDT Reporter: Jonah Graham <jonah>
Component: cdt-debug-dsf-gdbAssignee: Jonah Graham <jonah>
Status: RESOLVED FIXED QA Contact: Jonah Graham <jonah>
Severity: normal    
Priority: P3 CC: cdtdoug, jonah, umair_sair
Version: 9.3.0   
Target Milestone: 9.5.0   
Hardware: PC   
OS: Linux   
See Also: https://git.eclipse.org/r/116116
https://git.eclipse.org/c/cdt/org.eclipse.cdt.git/commit/?id=a8195048731655a422586856d1bc8801d1f0147e
Whiteboard:

Description Jonah Graham CLA 2018-01-26 08:12:45 EST
On GDB inserting/removing many breakpoints quickly, DSF's breakpoint manager and synchronizer get out of step with GDB. This causes error messages back from GDB as CDT tries to delete breakpoints that don't exist and can lead to the "tick" mark showing a breakpoint is installed when it no longer is. Internally the breakpoint's install count is not tracking GDB and it is happening because the breakpoint manager and synchronizer's states are not reentrant.

Steps to reproduce:
1) Create any C project (it just needs a main method)
2) Create a script to source from GDB, like so:
   python -c 'for i in range(2, 1000): print("b main"); print("del %d" % i)' > script
What that does is create a script the inserts and removes a breakpoint from main quickly, the beginning of the file looks like:
b main
del 2
b main
del 3
b main
del 4
3) Debug the program, letting it stop at the auto-inserted bp at main
4) In the debugger console do "source script"
5) At the end of the run, do "info break" and check the installed indicator tick in the breakpoints window. They won't match.
6) Observe the gdb traces, it should be only be a succession of =breakpoint-created/=breakpoint-deleted notifications, but at some random points CDT will try to re-delete breakpoints with -break-delete which will cause GDB to return an error like ~"No breakpoint number 2.\n"
Comment 1 Eclipse Genie CLA 2018-01-26 09:44:19 EST
New Gerrit change created: https://git.eclipse.org/r/116116
Comment 3 Umair Sair CLA 2018-03-06 11:59:57 EST
Hi Jonah,

There is a problem observed in cdt 9.3 that sometimes breakpoint with ignore count doesn't hit correct number of times. After investigation, I found that when breakpoint marker is updated (MIBreakpointsSynchronizer.targetBreakpointModified(..) calls ICBreakpoint.setIgnoreCount(..)), AutoBuildJob is triggered with breakpoint modification event and handled by MIBreakpointsManager.breakpointChanged(..) where it stores the breakpoint attributes immediately. These attributes are used later by MIBreakpointsManager.modifyBreakpoint(..) to create delta.

The problem starts when modifyBreakpoint(..) uses the current (latest) attributes of breakpoint to create delta by comparing it to breakpoint attributes from marker. The event might have triggered when ignore count was reduced to 'x', and marker will have 'x' ignore count. But when ignore count is get from latest attributes of breakpoint, ignore count is reduced further by that time, say 'x - 10'. So moving forward, GDBBreakpoints_7_0.modifyBreakpoint(..) also detects the ignore count is changed and "-break-after <breakpoint#> 'x - 10' " command is issued which causes undesired behavior.

The problem is random because it requires some reasonable delay (depending on system performance) between two notifications from gdb of breakpoint modification. This time allows MIBreakpointsManager.breakpointChanged(..) to run its scheduled dsfRunnable improperly.

With the fix of this bug (530377), I was expecting that the above explained issue must be solved. But it seems that with this change, the issue can be reproduced more reliably because of event queuing. I am running following code snippet.

#include <stdio.h>
#include <stdlib.h>

int main(void) {
	for (int i = 0; i < 40000; i++)
		puts("!!!Hello World!!!"); /* prints !!!Hello World!!! */
	return EXIT_SUCCESS;
}

Added breakpoint on puts call with ignore count 10000 and resumed. Execution completed in no time, when I checked the value of 'i', it was around 18000 and even after all breakpoint-modified messages have been appeared in gdb traces console, following keeps on appearing in gdb console.

592,412 48-break-after 1 7877
592,422 48^done
592,422 (gdb) 
596,832 49-break-after 1 6219
596,847 49^done
596,847 (gdb) 
....
....

And eclipse starts going into unresponsive state because queued events are still being processed and at the end I had to kill eclipse. Once I also got GC overhead limit exceeded in dsf thread.

The same scenario worked fine in cdt 9.3.3.

Can you please test on your side.
Comment 4 Jonah Graham CLA 2018-03-06 12:52:04 EST
(In reply to Umair Sair from comment #3)
This is very interesting because you are describing some related problems that I am actually working on.

I see your problem as two separate bugs, both separate from this bug.

1. Too many events causes CDT to misbehave in numerous ways, including OOM errors. See Bug 532076 for this.
2. DSF is changing the breakpoint properties incorrectly due to state corruption. See Bug 532077 for this.

I am working now in this area so hope to solve some of these issues. Thanks for testing what is there so far, hopefully you will have time to do some further tests as I progress things.

PS See Bug 532035 where I have added the ability to syhcronize the bp state in DSF with GDB explicitly.
Comment 5 Umair Sair CLA 2018-03-07 01:24:54 EST
(In reply to Jonah Graham from comment #4)
> I am working now in this area so hope to solve some of these issues. Thanks
> for testing what is there so far, hopefully you will have time to do some
> further tests as I progress things.
Yes, I'll get time to test.