Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 330556 - [typing] Indentation is messed up on paste
Summary: [typing] Indentation is messed up on paste
Status: VERIFIED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Text (show other bugs)
Version: 3.7   Edit
Hardware: All All
: P2 major (vote)
Target Milestone: 3.7 M4   Edit
Assignee: Dani Megert CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-11-18 07:21 EST by Deepak Azad CLA
Modified: 2010-12-07 08:20 EST (History)
5 users (show)

See Also:
daniel_megert: review-


Attachments
code snippet to paste in JE (4.50 KB, text/plain)
2010-11-18 07:21 EST, Deepak Azad CLA
no flags Details
Patch (8.61 KB, patch)
2010-11-23 03:48 EST, Rajesh CLA
no flags Details | Diff
Patch (2.19 KB, patch)
2010-11-25 13:32 EST, Rajesh CLA
daniel_megert: review-
Details | Diff
Snippet to paste into empty method body (477 bytes, text/plain)
2010-11-29 11:37 EST, Dani Megert CLA
no flags Details
patch (10.25 KB, patch)
2010-12-01 04:38 EST, Rajesh CLA
daniel_megert: review-
Details | Diff
patch (10.45 KB, patch)
2010-12-02 14:49 EST, Rajesh CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Deepak Azad CLA 2010-11-18 07:21:47 EST
Created attachment 183376 [details]
code snippet to paste in JE

>3.7 M3 (Works fine with M2)

- Copy the contents of the attached file and paste in the Java editor => The indentation is messed up
- Formatting the file fixes the indentation

Possibly a consequence of Bug 65317 ?
Comment 1 Deepak Azad CLA 2010-11-18 07:31:40 EST
Simpler/smaller test case

String[]a=new String[] {
	"X.java",
	"public class X extends B{\n" +
	"	public static int field1;\n" +
	"	public static X xfield;\n" +
	"	public void bar1(int i) {\n" + 
	"		field1 = 1;\n" +
	"	}\n" +
	"	public void bar2(int i) {\n" + 
	"		this.field1 = 1;\n" +
	"	}\n" + 
	"}\n" +
	"class A{\n" +
	"	public static X xA;\n" +
	"}\n" +
	"class B{\n" +
	"	public static int b1;\n" +
	"}"
};
Comment 2 Ayushman Jain CLA 2010-11-18 07:48:54 EST
This is causing much grief to us because in jdt/core we usually copy paste the console output into a junit. And there's no workaround for that.
Comment 3 Deepak Azad CLA 2010-11-18 07:53:28 EST
(In reply to comment #2)
> This is causing much grief to us because in jdt/core we usually copy paste the
> console output into a junit. And there's no workaround for that.

What is this then? :)
(In reply to comment #0)
> - Formatting the file fixes the indentation
Comment 4 Deepak Azad CLA 2010-11-18 07:58:35 EST
(In reply to comment #3)
> What is this then? :)
> (In reply to comment #0)
> > - Formatting the file fixes the indentation
sorry, but not the indentation.
Comment 5 Deepak Azad CLA 2010-11-18 07:59:08 EST
> sorry, but not the indentation.
of course I meant formatting
Comment 6 Dani Megert CLA 2010-11-18 08:21:21 EST
Probably caused by fix for bug 65317.

Rajesh, please take a look. Also: make sure that the indentation is the same when using Ctrl+Shift+F and Ctrl+I when everything is selected.
Comment 7 Rajesh CLA 2010-11-22 07:04:38 EST
(In reply to comment #6)
> Probably caused by fix for bug 65317.
> 
> Rajesh, please take a look. Also: make sure that the indentation is the same
> when using Ctrl+Shift+F and Ctrl+I when everything is selected.

Phew!! This was quite tricky. Till M2 this is what was happening -
# If you paste the following 4 lines -

String p= "public void bar2(int i) {\n" + 
" this.field1 = 1;}\n" +
		"public void bar3(int i) {\n" + 
		" this.field1 = 1;}\n" ;

The smart paste code parses till line 2 and since there is no change in indentation (there was no continuation indentation), it would just skip the rest of the lines. So the above would paste as is, even if lines 3/4 had wrong indents.

#With M3, if you paste the same you will get the following -

String p= "public void bar2(int i) {\n" + 
		" this.field1 = 1;}\n" +
		"public void bar3(int i) {\n" + 
				" this.field1 = 1;}\n" ;

Continuation indentation returns an indent for line 2 and 3 correctly, but in line 4, the heuristic scanner scans tokens inside of the string (I had no idea it did that) which causes indentation to be based on tokens such as braces, etc inside the string and not on continuation. There is no code currently that handles skipping scanning of strings and infact the heuristic scanner does not return " as a specific token. Will work on this later tonight.
Comment 8 Rajesh CLA 2010-11-23 03:48:05 EST
Created attachment 183642 [details]
Patch

With the attached patch Smart Paste should work. But the CTRL+I result is slightly different because each of these has separate setup code which causes the heuristic scanner to behave differently for the exact same lines of text. While I have tested with various samples, Deepak/Ayush if you guys can provide feedback that would be helpful. 
Here is what you should get with smart paste & Format -

String[]a=new String[] {
		"X.java",
		"public class X extends B{\n" +
				"    public static int field1;\n" +
				"    public static X xfield;\n" +
				"    public void bar1(int i) {\n" + 
				"        field1 = 1;\n" +
				"    }\n" +
				"    public void bar2(int i) {\n" + 
				"        this.field1 = 1;\n" +
				"    }\n" + 
				"}\n" +
				"class A{\n" +
				"    public static X xA;\n" +
				"}\n" +
				"class B{\n" +
				"    public static int b1;\n" +
				"}"
};

Here is what you should see with CTRL+I - 

String[]a=new String[] {
		"X.java",
		"public class X extends B{\n" +
		"    public static int field1;\n" +
		"    public static X xfield;\n" +
		"    public void bar1(int i) {\n" + 
		"        field1 = 1;\n" +
		"    }\n" +
		"    public void bar2(int i) {\n" + 
		"        this.field1 = 1;\n" +
		"    }\n" + 
		"}\n" +
		"class A{\n" +
		"    public static X xA;\n" +
		"}\n" +
		"class B{\n" +
		"    public static int b1;\n" +
		"}"
};
Comment 9 Markus Keller CLA 2010-11-25 10:04:29 EST
(In reply to comment #7)
> The smart paste code parses till line 2 and since there is no change in
> indentation (there was no continuation indentation), it would just skip the
> rest of the lines.

That was actually a feature, not a bug. The original behavior should be restored. The smartness of Smart Paste should only find the correct indentation of the whole pasted string. It should not try to do Correct Indentation behind the scenes.

To determine the right indent, you need to read the first 2 lines, because the first line is often copied without any indentation, so we need to peek at the second line to find out what the right indentation of the first line was. Indentations of lines 2..n should never be corrected individually (i.e. all lines should be shifted by the same amount).
Comment 10 Rajesh CLA 2010-11-25 13:32:36 EST
Created attachment 183875 [details]
Patch

This patch fixes smart paste so that it is consistent with Ctrl+I and New Line behaviors. The samples provided in this bug paste correctly. There is a broader issue where the formatter for string continuation does additional indentation in certain cases from the second continuation line. This has nothing to do specifically with paste and will have to be fixed so that it works for paste, ctrl+I and new line. I will do that separately as it will be a broader/non-trivial change given the current design of the Indenter/Scanner.
Comment 11 Rajesh CLA 2010-11-25 14:05:30 EST
(In reply to comment #9)
> (In reply to comment #7)
> > The smart paste code parses till line 2 and since there is no change in
> > indentation (there was no continuation indentation), it would just skip the
> > rest of the lines.
> 
> That was actually a feature, not a bug. The original behavior should be
> restored. The smartness of Smart Paste should only find the correct indentation
> of the whole pasted string. It should not try to do Correct Indentation behind
> the scenes.
> 
> To determine the right indent, you need to read the first 2 lines, because the
> first line is often copied without any indentation, so we need to peek at the
> second line to find out what the right indentation of the first line was.
> Indentations of lines 2..n should never be corrected individually (i.e. all
> lines should be shifted by the same amount).

Markus, didn't see your comment before sending in the patch(got a 'mid-air collision page' :).
As per the current code, if the second line does require indentation, then it
removes the partitioning and continues to compute the indentation for lines
3..n. Now, without the partitioning, the scanner returns tokens from within the
strings that contain code. I am not sure how one would compute the indentation
in this scenario?
Comment 12 Dani Megert CLA 2010-11-29 11:37:42 EST
Created attachment 184050 [details]
Snippet to paste into empty method body
Comment 13 Dani Megert CLA 2010-11-29 11:44:10 EST
I agree with Markus that we should not format the whole string but only peek into the existing code to find the right initial indentation level for the pasted string.

> Rajesh, please take a look. Also: make sure that the indentation is the same
> when using Ctrl+Shift+F and Ctrl+I when everything is selected.
This is still not the case, and also pasting in some scenarios results in bad indentation, e.g.
void foo() {
	<paste attachment 184050 [details] here>
}
==> indentation is destroyed
Comment 14 Rajesh CLA 2010-12-01 04:38:57 EST
Created attachment 184229 [details]
patch

For string continuation, smart paste, ctrl+I and format should match. Also added tests.
Comment 15 Dani Megert CLA 2010-12-01 09:58:26 EST
Comment on attachment 184229 [details]
patch

The code tries to be super-smart by indenting every line of the pasted string. However, at least up to now the idea of smart paste was simple to find the right indentation level for the whole clipboard contents.

Paste the following:
------%<------
foo();
foo();
	foo();
foo();
	foo();
foo();
foo();
------%<------

==> this should not change the alignment of line 3 and 5 relative to the other lines.
Comment 16 Rajesh CLA 2010-12-02 01:39:55 EST
(In reply to comment #15)
> (From update of attachment 184229 [details] [diff])
> The code tries to be super-smart by indenting every line of the pasted string.
> However, at least up to now the idea of smart paste was simple to find the
> right indentation level for the whole clipboard contents.
> 
> Paste the following:
> ------%<------
> foo();
> foo();
>     foo();
> foo();
>     foo();
> foo();
> foo();
> ------%<------
> 
> ==> this should not change the alignment of line 3 and 5 relative to the other
> lines.

The patch does 2 things -
1. Makes a small change to smart paste to shift removal of partitioning at the end.
2. String continuation handling (so that ctrl-I and format on selection will return same result).

For your above example, the 'String continuation handler' is not even executed, so it is not being super-smart.

If you copy/paste your example from this page into 3.7M2 or 3.6, every line will be processed (computeIndentation() is called) because line 2 is incorrectly indented, but the indentation of lines 3 & 5 do not get fixed. If you paste this into 3.7M3, the indentation of lines 3 & 5 get fixed due to fixes delivered in M3.

In summary -
# The idea of smart paste is still simply to find the indentation level of the clipboard contents.
# Like in 3.6, indentation is computed for lines 3..n only if line 2 is incorrectly indented.
# Iff indentation is computed for lines 3..n, then fixes made in M3 and in this patch will return different indentation results from before.
# Because line 2 has incorrect indentation, the result of pasting your above example is correct (in 3.6, indentation for lines 3/5 were being computed incorrectly).
Comment 17 Dani Megert CLA 2010-12-02 05:41:43 EST
># Because line 2 has incorrect indentation, the result of pasting your above
>example is correct (in 3.6, indentation for lines 3/5 were being computed
>incorrectly).
The clipboard should be moved left/right as block and not being touched. Only line 1 and 2 can be touched if needed.
Comment 18 Rajesh CLA 2010-12-02 14:49:20 EST
Created attachment 184390 [details]
patch

Made a change to smart paste so that indentation is computed only for lines 1 & 2. Lines 3..n are indented/unindented by the same amount as line 2. Tested this with various samples and compared the results with M2.
Comment 19 Dani Megert CLA 2010-12-03 02:58:14 EST
The fix is still not good as it now adds the indent to the line before the insertion point:

Paste the following:
------%<------
foo();
foo();
    foo();
foo();
    foo();
foo();
foo();
------%<------
just before the closing brace:
void foo() {
}
==>
void foo() {
	foo();
	foo();
	    foo();
	foo();
	    foo();
	foo();
	foo();
	}
BUG: last brace gets indented.
Comment 20 Dani Megert CLA 2010-12-03 03:07:52 EST
I've reverted all recently made indent fixes.
Comment 21 Deepak Azad CLA 2010-12-07 08:20:04 EST
Verified for 3.7M4 with I20101206-1800.