Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 346129

Summary: Crash when scrolling using Traditional Renderer under 64-bit Redhat Enterprise Linux
Product: [Tools] CDT Reporter: Andre St. Laurent <st.laurent>
Component: cdt-memoryAssignee: Randy Rohrbach <Randy.Rohrbach>
Status: RESOLVED FIXED QA Contact: Ted Williams <ted>
Severity: normal    
Priority: P3 CC: cdtdoug, Randy.Rohrbach
Version: 8.0Flags: Randy.Rohrbach: review? (pawel.1.piech)
Target Milestone: ---   
Hardware: PC   
OS: Windows XP   
Whiteboard:
Attachments:
Description Flags
AbstractPane.java patch for CDT 7.0.2
none
AbstractPane.java patch for CDT 8.0
none
This patch corrects a problem in the the original 7.0.2 patch
cdtdoug: iplog+
This patch corrects a problem in the the original 8.0 patch cdtdoug: iplog+

Description Andre St. Laurent CLA 2011-05-17 13:51:27 EDT 
Build Identifier: 3.6.2

When displaying memory contents using the Traditional Renderer on 64-bit Redhat Enterprise Linux and then using the arrow keys to scroll the viewport contents, a crash or stack overflow can occur, due to an infinite recursion in the caret control logic; see AbstractPane.ensureCaretWithinViewport().

Reproducible: Always

Steps to Reproduce:
0. Start Eclipse on 64-bit RHEL
1. Display memory contents in the Memory or Memory Browser views
2. Set column count that exceeds viewable pane width
3. Position caret at top-left line of data pane; position 0
4. Press the Backspace or Left-Arrow key
Comment 1 Andre St. Laurent CLA 2011-05-17 14:04:23 EDT 
Created attachment 195899 [details]
AbstractPane.java patch for CDT 7.0.2

This patch fixes the infinite recursion issue with the Traditional Renderer when running on 64-bit Redhat Enterprise Linux.
Comment 2 Andre St. Laurent CLA 2011-05-17 14:05:19 EDT 
Created attachment 195900 [details]
AbstractPane.java patch for CDT 8.0

This patch fixes the infinite recursion problem in the Traditional Renderer running on 64-bit Redhat Enterprise Linux.
Comment 3 Randy Rohrbach CLA 2011-05-17 23:11:49 EDT 
I'm taking this one.

Randy
Comment 4 Andre St. Laurent CLA 2011-05-19 13:43:01 EDT 
Created attachment 196144 [details]
This patch corrects a problem in the the original 7.0.2 patch

I accidentally introduced a small regression with the original 7.0.2 patch, which is corrected in this updated patch.

Specifically, attempting to vertically-scroll the viewport by pressing the UP or DOWN arrow keys when the number of columns exceeds the size of the view would cause the caret positioning logic to get confused.  This revised patch fixes this problem.
Comment 5 Andre St. Laurent CLA 2011-05-19 13:44:10 EDT 
Created attachment 196145 [details]
This patch corrects a problem in the the original 8.0 patch

I accidentally introduced a small regression with the original 8.0 patch, which is corrected in this updated patch.

Specifically, attempting to vertically-scroll the viewport by pressing the UP or DOWN arrow keys when the number of columns exceeds the size of the view would cause the caret positioning logic to get confused.  This revised patch fixes this problem.
Comment 6 Randy Rohrbach CLA 2011-05-19 23:59:35 EDT 
Folks

   I looked for guidance on checking in this fix. It is a critical
   fix. Got no response. So I am checking it in to HEAD.

   I will follow up with 7.0.2 branch tomorrow.

Randy
781-364-2226

Pawel please review.

Randy
Comment 7 CDT Genie CLA 2011-05-20 00:23:08 EDT 
*** cdt cvs genie on behalf of rarohrba ***
Bug 346129 - stack overflow crash when moving in renderer.

[*] AbstractPane.java 1.7 http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.cdt/memory/org.eclipse.cdt.debug.ui.memory.traditional/src/org/eclipse/cdt/debug/ui/memory/traditional/AbstractPane.java?root=Tools_Project&r1=1.6&r2=1.7