Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 495484 - Remove unique user tracking id in Neon
Summary: Remove unique user tracking id in Neon
Status: RESOLVED FIXED
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: p2 (show other bugs)
Version: 4.6.0 Neon   Edit
Hardware: All All
: P3 blocker with 4 votes (vote)
Target Milestone: Neon   Edit
Assignee: Pascal Rapicault CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on: 490112
Blocks:
  Show dependency tree
 
Reported: 2016-06-05 08:05 EDT by Markus Knauer CLA
Modified: 2016-06-14 05:45 EDT (History)
28 users (show)

See Also:
Mike_Wilson: pmc_approved+
daniel_megert: pmc_approved+
mober.at+eclipse: pmc_approved+
markus.kell.r: review+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Markus Knauer CLA 2016-06-05 08:05:34 EDT
In Eclipse Neon (4.6) M7 a user specific UUID has been introduced. This UUID is generated on the client (Eclipse) and is sent to eclipse.org servers in http requests.

This UUID is used in p2 requests (this bug), but also in Marketplace REST calls of MPC (bug 492916), and in AERI (bug 492917).

With the first announcement of this feature on the cross-project-issues mailing list [1] on 3 Jun 2016, a lot of concerns have been raised by the community that need to be solved before any mechanism like this can be released to the public.

Concerns raised are

- missing community involvement including review by the community
- unclear goals of the implementation
- open legal situation
- privacy
- opt-out versus opt-in

This list is certainly incomplete as the discussion is ongoing.

I hereby request to remove/revert the current implementation for Eclipse Neon (4.6) including a respin of Eclipse Platform RC4, and discuss a possible later inclusion based on community consensus.


[1] https://dev.eclipse.org/mhonarc/lists/cross-project-issues-dev/msg13204.html
Comment 1 Ian Skerrett CLA 2016-06-05 09:50:35 EDT
+1 The Eclipse community needs to have more of a discussion on adding a UUID.
Comment 2 Nobody - feel free to take it CLA 2016-06-05 11:55:58 EDT
This is a valuable thing to have but it should definitely not be opt-out, especially since the opt-out smells like it was deliberately left to be too difficult to be executed. Let's not kid ourselves here that an announcement on a Friday afternoon at 5pm after everything was completed and pushed is the right way to approach it.

There is potential to do lots of damage here. Eclipse is used by a lot of entities (persons or companies) which have varying degrees of privacy concerns, regardless of how right or wrong we think these concerns are. With opt-in you attend to everyone's concerns by letting them decide. If you decide for them you will definitely antagonize some to an un-repairable degree. You might think that it's an affordable price to pay ("we'll lose some but hey, we pushed it through") but I think it is not.

We get lots of flak fire for not announcing things much less important than this beforehand, so I'm baffled that this slid under the radar like this. We need to have an explicit rule about UUID-over-the-network style changes so this doesn't happen again.

In short, it would be great to have this data but having it opt-in is a solution to most concerns. AERI does a pretty good job at that.
Comment 3 Dani Megert CLA 2016-06-05 12:52:43 EDT
(In reply to Markus Knauer from comment #0)
> I hereby request to remove/revert the current implementation for Eclipse
> Neon (4.6) including a respin of Eclipse Platform RC4,

RC4 is out. As per our rules (https://www.eclipse.org/eclipse/development/plans/freeze_plan_4_6.php) the Eclipse PMC will have to decide how to proceed.
Comment 4 Dani Megert CLA 2016-06-05 13:02:43 EDT
(In reply to Dani Megert from comment #3)
> (In reply to Markus Knauer from comment #0)
> > I hereby request to remove/revert the current implementation for Eclipse
> > Neon (4.6) including a respin of Eclipse Platform RC4,
> 
> RC4 is out. As per our rules
> (https://www.eclipse.org/eclipse/development/plans/freeze_plan_4_6.php) the
> Eclipse PMC will have to decide how to proceed.

In the meantime, Pascal, please provide the necessary Gerrit changes that completely remove the feature.
Comment 5 Ed Willink CLA 2016-06-05 13:06:36 EDT
(In reply to Dani Megert from comment #3)
> RC4 is out.

Yes, but we are at +0. Providing an RC4a for the platform at +2/+3 would surely be a bug fix without needing a respin.
Comment 6 Dani Megert CLA 2016-06-05 13:10:50 EDT
(In reply to Ed Willink from comment #5)
> (In reply to Dani Megert from comment #3)
> > RC4 is out.
> 
> Yes, but we are at +0. Providing an RC4a for the platform at +2/+3 would
> surely be a bug fix without needing a respin.

My point was: we won't delay RC4. Once we have a fix and PMC approval we (the Platform) can respin and call it RC4a or whatever fits.
Comment 7 Mike Wilson CLA 2016-06-05 13:47:13 EDT
Agree that it should be removed before we ship. +1
Comment 8 Lars Vogel CLA 2016-06-05 15:17:39 EDT
+1 for removal until clarification of the open points is done
Comment 9 Dani Megert CLA 2016-06-05 15:46:01 EDT
+1 from me as well.
Comment 10 Matthias Sohn CLA 2016-06-05 18:08:01 EDT
+1 for removal of user specific UUID
Comment 11 Martin Oberhuber CLA 2016-06-05 18:10:50 EDT
+1 here as well, the negative perception of this needs to be addressed.
Comment 12 Pascal Rapicault CLA 2016-06-05 18:42:04 EDT
I'll commit something later tonight, removing everything that has been added.
Comment 13 Pascal Rapicault CLA 2016-06-05 22:14:25 EDT
I've pushed the change reverting the changes in p2 (https://git.eclipse.org/r/#/c/74611/)

The change in runtime needs to be committed https://git.eclipse.org/r/#/c/74610 

I've directly reverted the changes from the original gerrit review so no code has been written.
Comment 14 Martin Oberhuber CLA 2016-06-06 02:05:57 EDT
(In reply to Pascal Rapicault from comment #13)
> I've directly reverted the changes from the original gerrit review so no
> code has been written.

Hi Pascal,

is it possible that a 2nd change also needs to be reverted ? - Right now it seems to me that the code would still send an uuid, but unconditionally to whatever server is contacted (and not just eclipse.org)...

https://git.eclipse.org/c/equinox/rt.equinox.p2.git/log/bundles/org.eclipse.equinox.p2.transport.ecf/src/org/eclipse/equinox/internal/p2/transport/ecf/FileReader.java

It also seems to me that InternalPlatform would still produce an UUID for all plugins to consume, but the explanation message that says how to disable it would be removed.
Or am I mis-reading the changelog ?
Comment 15 Dani Megert CLA 2016-06-06 02:29:25 EDT
Another question regarding the unclear legal situation is: Do we also have to remove already written UUIDs?
Comment 16 Maximilian Koegel CLA 2016-06-06 03:08:32 EDT
Dani makes a good point, I think we need to automatically dispose of any already existing UUIDs by overwriting them. In addition any collected data associated with the UUID must be deleted by the Eclipse Foundation imho.
Is there any, Ian?

+1 for removing the change now and later for a transparent and open discussion if a UUID is really something we need and how users are informed of it and can agree to it.
Comment 17 Ian Skerrett CLA 2016-06-06 03:15:03 EDT
(In reply to Dani Megert from comment #15)
> Another question regarding the unclear legal situation is: Do we also have
> to remove already written UUIDs?

IANAL, but I think the safest would be to attempt to delete it. My guess is that this might not always be possible. The key thing is that we are not sending the UUID.
Comment 18 Ed Willink CLA 2016-06-06 03:42:53 EDT
(In reply to Maximilian Koegel from comment #16)
> Dani makes a good point, I think we need to automatically dispose of any
> already existing UUIDs by overwriting them. In addition any collected data
> associated with the UUID must be deleted by the Eclipse Foundation imho.
> Is there any, Ian?

Yes, But. This involves retaining code to write the UUID file, and involves new code to delete a file in the user's home directory. Both are risks.

The UUID code has and will perhaps never be in an official Eclipse release; just M7 through RC4. Better to keep it that way. The preliminary releases are mostly used by committers / well-informed users who can delete the file manually if they really care.

If the EF destroys as many historic logs of the UUID as practical, I think that represents a reasonable endeavor to rectify the problem.
Comment 19 Dani Megert CLA 2016-06-06 04:50:33 EDT
(In reply to Ed Willink from comment #18)
> (In reply to Maximilian Koegel from comment #16)
> > Dani makes a good point, I think we need to automatically dispose of any
> > already existing UUIDs by overwriting them. In addition any collected data
> > associated with the UUID must be deleted by the Eclipse Foundation imho.
> > Is there any, Ian?
> 
> Yes, But. This involves retaining code to write the UUID file, and involves
> new code to delete a file in the user's home directory. Both are risks.
> 
> The UUID code has and will perhaps never be in an official Eclipse release;
> just M7 through RC4. Better to keep it that way. The preliminary releases
> are mostly used by committers / well-informed users who can delete the file
> manually if they really care.

I agree with this. We should just remove the code and the Foundation should inform that they removed the data and explain how users that used a Neon dev build can remove the UUID from their system.
Comment 20 Dani Megert CLA 2016-06-06 05:00:17 EDT
(In reply to Martin Oberhuber from comment #14)
> (In reply to Pascal Rapicault from comment #13)
> > I've directly reverted the changes from the original gerrit review so no
> > code has been written.
> 
> Hi Pascal,
> 
> is it possible that a 2nd change also needs to be reverted ? - Right now it
> seems to me that the code would still send an uuid, but unconditionally to
> whatever server is contacted (and not just eclipse.org)...
> 
> https://git.eclipse.org/c/equinox/rt.equinox.p2.git/log/bundles/org.eclipse.equinox.p2.transport.ecf/src/org/eclipse/equinox/internal/p2/transport/ecf/FileReader.java

Correct. This needs to be reverted as well, so that it matches the version from commit 4c9d46150f2487b71ed0bacf9e0e178c33b6912e.


> It also seems to me that InternalPlatform would still produce an UUID for
> all plugins to consume, but the explanation message that says how to disable
> it would be removed.

Correct. The other commit also needs to be reverted, so that it matches the version from commit 34bc27b03afa073b72e6f1fe8e710d378d9a831c.
Comment 21 Markus Knauer CLA 2016-06-06 05:08:41 EDT
(In reply to Martin Oberhuber from comment #14)
> is it possible that a 2nd change also needs to be reverted ?

Good catch, I think you are right.

Others should verify, but I have the following list of commits, two of them need to be reverted. And as far as I can see the N&N needs to be updated, too.


Equinox p2
==========

commit 3bcec796ae83cd5400228185662275446056993f - send only to eclipse.org
https://git.eclipse.org/r/#/c/74611
REVERT DONE commit e421fd9ec19155d412a64851f348e23dbbd3a948

commit f64681eca06c4f8142bac7f0a1c91df8113a232d - add UUID
REVERT MISSING


Platform Runtime
================

commit 4561fb4325bb3edf0d40dff5dfca9a474273501e - property file comment
https://git.eclipse.org/r/#/c/74610
REVERT AWAITING REVIEW commit 416a69a750defc2c9abe695e6212ebab0ed5871c

commit b97ef3f4f4fc248edcabf714c36ad7980529f399 - UUID implementation
REVERT MISSING
Comment 22 Maximilian Koegel CLA 2016-06-06 06:01:45 EDT
(In reply to Dani Megert from comment #19)
> (In reply to Ed Willink from comment #18)
> > (In reply to Maximilian Koegel from comment #16)
> > > Dani makes a good point, I think we need to automatically dispose of any
> > > already existing UUIDs by overwriting them. In addition any collected data
> > > associated with the UUID must be deleted by the Eclipse Foundation imho.
> > > Is there any, Ian?
> > 
> > Yes, But. This involves retaining code to write the UUID file, and involves
> > new code to delete a file in the user's home directory. Both are risks.
> > 
> > The UUID code has and will perhaps never be in an official Eclipse release;
> > just M7 through RC4. Better to keep it that way. The preliminary releases
> > are mostly used by committers / well-informed users who can delete the file
> > manually if they really care.
> 
> I agree with this. We should just remove the code and the Foundation should
> inform that they removed the data and explain how users that used a Neon dev
> build can remove the UUID from their system.

OK, I agree, informing users about how to delete the UUID in case they used M7 to RC4 is probably better than keeping some of the code.
Comment 23 Eclipse Genie CLA 2016-06-06 08:12:09 EDT
New Gerrit change created: https://git.eclipse.org/r/74646
Comment 24 Markus Keller CLA 2016-06-06 08:15:32 EDT
(In reply to Eclipse Genie from comment #23)
> New Gerrit change created: https://git.eclipse.org/r/74646

Please review this suggested N&N change. The same update will be applied to /org.eclipse.platform.doc.user/whatsNew/platform_whatsnew.html
Comment 25 Markus Keller CLA 2016-06-06 08:35:09 EDT
> The same update will be applied to
> /org.eclipse.platform.doc.user/whatsNew/platform_whatsnew.html

Actually, we should just remove the item from the o.e.platform.doc.user bundle, since it's not going to be relevant for any released version.
But we need to update https://www.eclipse.org/eclipse/news/4.6/platform.php#uuid because there are already links in announcements that point to this entry.
Comment 26 Markus Keller CLA 2016-06-06 08:56:16 EDT
I accidentally pushed http://git.eclipse.org/c/platform/eclipse.platform.common.git/commit/?id=fbaf713c031abb4cdaf5a34254c02201a1771df4 because the EGit Staging view failed to recognize that it should push to Gerrit. I don't expect the removal from the help/infocenter N&N to be controversial, so I'll just leave it like this for the RC4a rebuild.
Comment 27 Eclipse Genie CLA 2016-06-06 09:32:40 EDT
New Gerrit change created: https://git.eclipse.org/r/74662
Comment 28 Eclipse Genie CLA 2016-06-06 09:37:20 EDT
New Gerrit change created: https://git.eclipse.org/r/74664
Comment 29 Eclipse Genie CLA 2016-06-06 09:40:54 EDT
New Gerrit change created: https://git.eclipse.org/r/74665
Comment 30 Pascal Rapicault CLA 2016-06-06 09:45:42 EDT
Here is the change that removes the generation of the uuid https://git.eclipse.org/r/#/c/74665/ this needs to be committed


The p2 change has already been committed.
http://git.eclipse.org/c/equinox/rt.equinox.p2.git/commit/?id=41688dd1563f4ccb6dd6921bef46c2a805c7d9da
Comment 32 Dani Megert CLA 2016-06-06 09:54:27 EDT
(In reply to Pascal Rapicault from comment #30)
> Here is the change that removes the generation of the uuid
> https://git.eclipse.org/r/#/c/74665/ this needs to be committed

Reviewed the change and verified that the code is the same as before the UUID was introduced.

 
> The p2 change has already been committed.
> http://git.eclipse.org/c/equinox/rt.equinox.p2.git/commit/?id=41688dd1563f4ccb6dd6921bef46c2a805c7d9da

Verified that the code is the same as before the UUID was introduced.
Comment 33 Dani Megert CLA 2016-06-06 09:56:13 EDT
Ian, you can either keep this bug open to track your work of communicating this to the community (see comment 19), or close this one as fixed and open a new bug for the communication.

Thanks everyone!
Comment 34 Markus Keller CLA 2016-06-06 10:48:57 EDT
I've pushed a temporary clarification of the situation to
https://www.eclipse.org/eclipse/news/4.6/platform.php#uuid . This entry will be removed completely from the Neon news after the rebuild is available and the eclipse-dev list has been informed about the revert.
Comment 35 Ian Skerrett CLA 2016-06-07 17:24:03 EDT
(In reply to Dani Megert from comment #19)
> (In reply to Ed Willink from comment #18)
> > (In reply to Maximilian Koegel from comment #16)
> > > Dani makes a good point, I think we need to automatically dispose of any
> > > already existing UUIDs by overwriting them. In addition any collected data
> > > associated with the UUID must be deleted by the Eclipse Foundation imho.
> > > Is there any, Ian?
> > 
> > Yes, But. This involves retaining code to write the UUID file, and involves
> > new code to delete a file in the user's home directory. Both are risks.
> > 
> > The UUID code has and will perhaps never be in an official Eclipse release;
> > just M7 through RC4. Better to keep it that way. The preliminary releases
> > are mostly used by committers / well-informed users who can delete the file
> > manually if they really care.
> 
> I agree with this. We should just remove the code and the Foundation should
> inform that they removed the data and explain how users that used a Neon dev
> build can remove the UUID from their system.

Just to follow-up on our next steps for removing the UUID. I plan to write a blog post about the UUID and how users can remove any existing UUID. I expect to have the blog posted early next week. The Eclipse Foundation will also do the work to delete any UUID data that has been received.  I hope this will be done in the next 1-2 weeks.
Comment 36 David Williams CLA 2016-06-07 17:51:33 EDT
(In reply to Ian Skerrett from comment #35)

> Just to follow-up on our next steps for removing the UUID. I plan to write a
> blog post about the UUID and how users can remove any existing UUID. I
> expect to have the blog posted early next week. The Eclipse Foundation will
> also do the work to delete any UUID data that has been received.  I hope
> this will be done in the next 1-2 weeks.

Thanks for letting us know, Ian. I just wanted to go on record to say I think it is a great idea if it helps understand our user community. I think people just feel better with the "opt-in" approach, but as we all know, even those people who decide not to "opt-in" are still providing some amount of data, admittedly less, just by using the internet, so I think people just need to understand what the planned use of the data is -- as well as how it is "protected" so someone could end up selling the data to one of those big-data aggregators -- you know, like Google. :)  Perhaps you can touch on those sorts of issues in your blog entry too? 

I do appreciate you discovered people's concern about it *before* we released! Thanks for that. Sincerely.
Comment 37 Ian Skerrett CLA 2016-06-13 16:06:46 EDT
(In reply to Ian Skerrett from comment #35)
> Just to follow-up on our next steps for removing the UUID. I plan to write a
> blog post about the UUID and how users can remove any existing UUID. I
> expect to have the blog posted early next week. The Eclipse Foundation will
> also do the work to delete any UUID data that has been received.  I hope
> this will be done in the next 1-2 weeks.

fyi, my blog post has been published. 

https://ianskerrett.wordpress.com/2016/06/14/uuid-removal-from-neon/
Comment 38 Markus Keller CLA 2016-06-14 05:01:48 EDT
The Neon N&N entry has been removed. Marking this bug as fixed.
Comment 39 Markus Knauer CLA 2016-06-14 05:45:34 EDT
A big thank you to everyone involved! I believe this was an important fix.