Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 317782 - TextTransfer generates incorrect UTF16 in pasteboard when copying strings from SWT application
Summary: TextTransfer generates incorrect UTF16 in pasteboard when copying strings fro...
Status: RESOLVED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: SWT (show other bugs)
Version: 3.5.2   Edit
Hardware: Macintosh Mac OS X - Carbon (unsup.)
: P3 normal (vote)
Target Milestone: 3.6.1   Edit
Assignee: Scott Kovatch CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-06-24 03:46 EDT by linyunzhi CLA
Modified: 2010-08-25 11:27 EDT (History)
5 users (show)

See Also:
Silenio_Quarti: review+


Attachments
CopyPasteTest.java (3.35 KB, text/plain)
2010-06-24 03:48 EDT, linyunzhi CLA
no flags Details
Native Carbon application CHZO858ABV (1.88 MB, application/octet-stream)
2010-06-24 03:49 EDT, linyunzhi CLA
no flags Details
patch for 3.5.2 (2.94 KB, patch)
2010-06-24 15:36 EDT, Silenio Quarti CLA
no flags Details | Diff
patch for 3.5.2 (4.20 KB, patch)
2010-06-24 15:56 EDT, Silenio Quarti CLA
no flags Details | Diff
patch for 3.5.2 (4.48 KB, patch)
2010-06-24 16:08 EDT, Silenio Quarti CLA
no flags Details | Diff
patch that only puts UTXT and UTF8 data in clipboard (4.48 KB, patch)
2010-06-24 16:13 EDT, Silenio Quarti CLA
no flags Details | Diff
patch that only puts UTXT and UTF8 data in clipboard (4.51 KB, patch)
2010-06-24 16:36 EDT, Silenio Quarti CLA
no flags Details | Diff
Patch from 3.6 (7.26 KB, patch)
2010-06-24 19:14 EDT, Scott Kovatch CLA
no flags Details | Diff
Patch for 3.5.2 (7.26 KB, patch)
2010-06-24 19:40 EDT, Scott Kovatch CLA
no flags Details | Diff
Revised patch (8.54 KB, patch)
2010-06-25 13:21 EDT, Scott Kovatch CLA
no flags Details | Diff
Revised patch for 3.5 branch (7.89 KB, patch)
2010-06-25 13:26 EDT, Scott Kovatch CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description linyunzhi CLA 2010-06-24 03:46:39 EDT
Build Identifier: 3.5.2

If we copy a string from SWT based application and then paste it to a native Carbon application, the native Carbon application could not get the correct data if inFlavorType is set to kUTTypeUTF16PlainText. This works with 3.4.2. If we roll back the changes in TextTransfer.java to 3.4.2, the copy/paste works. 

Reproducible: Always

Steps to Reproduce:
1. Run the CopyPasteTest.java application
2. Build and run the CHZO858ABV project
3. Copy a string from the CopyPasteTest application, this can be done either from Apple + C or clicking the buttons
4. Click "Edit" -> "myPaste" menu item to paste the data into the native CHZO858ABV application. Do NOT use Apple + V to paste. 
5. The paste fails.
Comment 1 linyunzhi CLA 2010-06-24 03:48:06 EDT
Created attachment 172576 [details]
CopyPasteTest.java
Comment 2 linyunzhi CLA 2010-06-24 03:49:18 EDT
Created attachment 172577 [details]
Native Carbon application CHZO858ABV
Comment 3 Silenio Quarti CLA 2010-06-24 14:28:30 EDT
The changes from 3.4.2 to 3.5.2 were done to fix bug#252109.
Comment 4 Silenio Quarti CLA 2010-06-24 14:39:40 EDT
Eclipse is putting "kUTTypeUTF16ExternalPlainText" data in the clipboard and the application is trying to get "kUTTypeUTF16PlainText".

Some how PasteboardGetItemFlavorFlags() says that kUTTypeUTF16PlainText format is in the clipboard but PasteboardCopyItemFlavorData fails to get data in that format.  It seems it cannot make the conversion from  kUTTypeUTF16ExternalPlainText to kUTTypeUTF16PlainText
Comment 5 Silenio Quarti CLA 2010-06-24 15:07:09 EDT
I was wrong. PasteboardGetItemFlavorFlags does not say that kUTTypeUTF16PlainText data is in the clipboard. So it does not try to convert between kUTTypeUTF16PlainText and kUTTypeUTF16ExternalPlainText for free.

I am not sure what to do here. Could the C application be changed to look for kUTTypeUTF16ExternalPlainText instead of kUTTypeUTF16PlainText?

If we change SWT to put kUTTypeUTF16PlainText data in the clipboard, I am afraid that we will be putting bug#252109 back.  I cannot check that because I do not have Entourage.   Scott,  do you believe this is true?
Comment 6 Scott Kovatch CLA 2010-06-24 15:17:59 EDT
(In reply to comment #5)
> If we change SWT to put kUTTypeUTF16PlainText data in the clipboard, I am
> afraid that we will be putting bug#252109 back.  I cannot check that because I
> do not have Entourage.   Scott,  do you believe this is true?

kUTTypeUTF16PlainText is 'utxt', not 'TEXT'. It was using 'TEXT' that caused 252109. I think we want to put 'ut16' and 'utxt' on the pasteboard.

I think the mistake in 252109 was that we took out 'utxt' and 'TEXT' and replaced them with 'ut16'. We just needed to remove 'TEXT' and use 'ut16' instead.

I think I have Entourage here, so I will check to make sure.
Comment 7 Silenio Quarti CLA 2010-06-24 15:36:19 EDT
Created attachment 172674 [details]
patch for 3.5.2 

This patch makes the C application work. Scott, please check Entourage?  Should the encoding for UTXT be something else?
Comment 8 Scott Kovatch CLA 2010-06-24 15:44:46 EDT
(In reply to comment #7)
> Created an attachment (id=172674) [details]
> patch for 3.5.2 
> 
> This patch makes the C application work. Scott, please check Entourage?  Should
> the encoding for UTXT be something else?

Okay, you beat me to it. I was getting ready to upload basically the same patch. Yes, this change still works when you copy from Eclipse and then paste into Entourage.

In Widget.java, we're asking the clipboard for 'TEXT' -- should that be 'utxt' as well?
Comment 9 Silenio Quarti CLA 2010-06-24 15:56:15 EDT
Created attachment 172679 [details]
patch for 3.5.2 

Yes and Widget.copyToClipboard() should put utxt and ut16 data to the clipboard otherwise Combo/Text copy would fail the same way.
Comment 10 Silenio Quarti CLA 2010-06-24 16:01:42 EDT
Oops, patch is wrong (1 sec)
Comment 11 Silenio Quarti CLA 2010-06-24 16:08:54 EDT
Created attachment 172681 [details]
patch for 3.5.2 

Ok, this is better.

I am wondering if we need to put both ut16 and utxt formats in the clipboard. It seems that utxt gets converted to ut16 for free. Would Entourage work if we only put utxt and utf8 data in the clipboard?
Comment 12 Silenio Quarti CLA 2010-06-24 16:13:40 EDT
Created attachment 172683 [details]
patch that only puts UTXT and UTF8 data in clipboard

Scott, please try this one with Entourage.
Comment 13 Silenio Quarti CLA 2010-06-24 16:36:04 EDT
Created attachment 172686 [details]
patch that only puts UTXT and UTF8 data in clipboard

This is the correct patch to add only UTXT and UTF8 to the clipboard.
Comment 14 Silenio Quarti CLA 2010-06-24 16:42:15 EDT
Comment on attachment 172686 [details]
patch that only puts UTXT and UTF8 data in clipboard

Ok, I just confirmed that free conversion does NOT happen either way. So only patch from comment#11 is correct.
Comment 15 Raji Akella CLA 2010-06-24 16:58:54 EDT
(In reply to comment #14)
Thanks Silenio & Scott! Yun Zhi if you can confirm this works for us we will get it into XPD build asap.
Comment 16 Silenio Quarti CLA 2010-06-24 17:34:20 EDT
Raji, we are still trying a few things. Please wait.
Comment 17 Raji Akella CLA 2010-06-24 17:35:38 EDT
OK, sorry for jumping the gun here.
Comment 18 Scott Kovatch CLA 2010-06-24 19:14:35 EDT
Created attachment 172695 [details]
Patch from 3.6

This patch reads and writes 'utxt' and 'utf8' to/from the clipboard. When we grab data with the 'utxt' flavor it's not in external representation (as opposed to 'ut16', so we have to pass false to CFStringCreateWithBytes so it can get the byte ordering correct.

Once I straightened that out, I changed the other places where we were still using 'ut16' and changed them to 'utxt' and not assuming external representation, for consistency. In practice it doesn't seem to matter based on the testing I did.
Comment 19 Scott Kovatch CLA 2010-06-24 19:32:14 EDT
See the file UTCoreTypes.h for a description of the various pasteboard types. Note that Cocoa fails this test for the 'copy' button because our Text objects are plain-text. If they use RTF then the 'ut16' format gets written to the clipboard. Otherwise, just 'utxt' is written.
Comment 20 Scott Kovatch CLA 2010-06-24 19:40:07 EDT
Created attachment 172699 [details]
Patch for 3.5.2

Use this patch for the R3_5_maintenance branch.
Comment 21 Raji Akella CLA 2010-06-25 11:27:49 EDT
Our Mac team has tested this privately and confirmed it fixes our problem. We will pull this into our builds.
Comment 22 Scott Kovatch CLA 2010-06-25 11:31:13 EDT
(In reply to comment #21)
> Our Mac team has tested this privately and confirmed it fixes our problem. We
> will pull this into our builds.

We may tweak it a bit more today based on review and more testing. Once we move it to resolved/fixed you'll know we're all done. Good to know that it's working, though.
Comment 23 Scott Kovatch CLA 2010-06-25 13:21:50 EDT
Created attachment 172791 [details]
Revised patch

One last change (I think). After more research we should be writing 'utxt' for Unicode and 'TEXT' (for the current encoding) to the clipboard, and let consumers convert to other encodings as needed.
Comment 24 Scott Kovatch CLA 2010-06-25 13:26:50 EDT
Created attachment 172793 [details]
Revised patch for 3.5 branch

Patch based on 3.5 maintenance branch.
Comment 25 Silenio Quarti CLA 2010-06-25 13:45:17 EDT
Patches look good.
Comment 26 Scott Kovatch CLA 2010-06-25 14:16:12 EDT
The last patches visible here are the final ones. We're going to put the fix in 3.6.1 as well.
Comment 27 linyunzhi CLA 2010-06-28 04:22:14 EDT
We have verified the patch for 3.5 branch and it fixes the problem. Thanks very much !
Comment 28 Scott Kovatch CLA 2010-06-28 12:17:45 EDT
Excellent! I will check this in to the 3.6 maintenance branch, and HEAD.
Comment 29 Scott Kovatch CLA 2010-06-28 13:34:05 EDT
Fixed in 3.6 branch and HEAD > 2010 06 28.