Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 379652 - Enable Gerrit for JDT/Core
Summary: Enable Gerrit for JDT/Core
Status: RESOLVED FIXED
Alias: None
Product: Community
Classification: Eclipse Foundation
Component: Git (show other bugs)
Version: unspecified   Edit
Hardware: PC Windows NT
: P1 critical (vote)
Target Milestone: ---   Edit
Assignee: Eclipse Webmaster CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 385536 (view as bug list)
Depends on:
Blocks: 384258 385707
  Show dependency tree
 
Reported: 2012-05-16 04:57 EDT by Tomasz Zarna CLA
Modified: 2013-01-03 06:18 EST (History)
13 users (show)

See Also:
srikanth_sankaran: review+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Tomasz Zarna CLA 2012-05-16 04:57:16 EDT
Please enable Gerrit for JDT/Core [1]. I would like project committers to
be able to bypass the Gerrit code review system and push changes directly to
the git repo.


[1] http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/
Comment 1 Eclipse Webmaster CLA 2012-05-16 11:10:11 EDT
I've enabled Gerrit for jdt/eclipse.jdt.core.git.  You can add it to the list of projects you watch:

https://git.eclipse.org/r/#/settings/projects

Upload your SSH public keys to Gerrit if you wish to use SSH:
https://git.eclipse.org/r/#/settings/ssh-keys

Your Gerrit-enabled repo URLs:
ssh://userid@git.eclipse.org:29418/jdt/eclipse.jdt.core.git
https://git.eclipse.org/r/p/jdt/eclipse.jdt.core.git

Gerrit@Eclipse docs:
http://wiki.eclipse.org/Gerrit#Logon

-M.
Comment 2 Srikanth Sankaran CLA 2012-05-17 01:06:47 EDT
Is there a reason why this bug's security settings should disallow anyone
other than reporter and those on CC list to view it ? (something I
hadn't noticed earlier)

This change disallowed git pushes due to ownership change and came in
the way of RC1 build input process and the release engineer had to stay
up until 1 AM to figure out what is going on and not being able to view
the bug report was impeding progress.

I was on the CC list - but I had no clue of any possible wider impact
due to enablement of Gerrit :-( otherwise I would have alerted the team.
Comment 3 Srikanth Sankaran CLA 2012-05-17 01:10:22 EDT
See https://bugs.eclipse.org/bugs/show_bug.cgi?id=379723.

Added Markus & Olivier to CC list as well, as they also occassionally
commit to JDT/Core.
Comment 4 Eclipse Webmaster CLA 2012-05-17 09:22:22 EDT
(In reply to comment #2)
> Is there a reason why this bug's security settings should disallow anyone
> other than reporter and those on CC list to view it ?

It was like that when I got it.  I don't see any reason for this bug to be restricted, since it doesn't cover any potential security issue, but perhaps Tomasz has some rationale?  

In any case I've removed the restrictions for now.

-M.
Comment 5 Markus Keller CLA 2012-05-17 14:41:01 EDT
(comment #0)
> I would like project committers to
> be able to bypass the Gerrit code review system and push changes directly to
> the git repo.

contradicts

(bug 379723 comment #7)
> Yes, because Gerrit needs to 'control' the repo completely. So that means no
> 'direct' committer access.

, so I wouldn't call this bug "fixed".

Tomasz, this request came at a bad time of the project cycle. During RC phases, no experiments are allowed.

Furthermore, the team lead (Srikanth) should be the only one requesting changes that affect all committers, and the webmaster should not take such wide-ranging requests from other people (especially not from non-committers on the project).

I'm not the JDT/Core team lead, but I think this should be reverted so that we can finish off the Juno release without uncertainties in the process. Srikanth, do you agree?
Comment 6 Eclipse Webmaster CLA 2012-05-17 14:50:34 EDT
(In reply to comment #5)
> (comment #0)
> > I would like project committers to
> > be able to bypass the Gerrit code review system and push changes directly to
> > the git repo.
> 
> contradicts
> 
> (bug 379723 comment #7)
> > Yes, because Gerrit needs to 'control' the repo completely. So that means no
> > 'direct' committer access.
> 

Actually it doesn't.  Gerrit supports the idea of allowing 'pushes' that bypass code review(no +1 required), but it doesn't like 'out of band'(non-Gerrit) pushes to the repo(although experience has demonstrated that it's possible).  That's what the change in group ownership prevents. 

> Furthermore, the team lead (Srikanth) should be the only one requesting changes
> that affect all committers, and the webmaster should not take such wide-ranging
> requests from other people (especially not from non-committers on the project).

I'll concede that I failed to consider that a non-project committer would ask for such a change.  But as the PL was cc'd it looked like a fairly typical 'The project wants X and the PL had the committer agitating for change file the bug, rather than file it themselves'.

I will happily revert the repo.

-M.
Comment 7 Srikanth Sankaran CLA 2012-05-17 22:11:47 EDT
(In reply to comment #5)

> I'm not the JDT/Core team lead, but I think this should be reverted so that we
> can finish off the Juno release without uncertainties in the process. Srikanth,
> do you agree?

Yes, please. This well intended request is better accommodated just past
3.8/4.2 shipment.

Now as none of the JDT/Core committers here have used Gerrit previously,
we need to be educated on what possible ripples a roll back could have
and what steps we need to take to react.
Comment 8 Eclipse Webmaster CLA 2012-05-18 08:46:06 EDT
Ok, I've removed JDT/Core from Gerrit.  If committers have updated their URLs to use the Gerrit URLs, they will need to revert to the originals(ssh://committer_id@git.eclipse.org/gitroot/jdt/eclipse.jdt.core.git) , beyond that I don't think there is anything else they need to do.

-M.
Comment 9 Ayushman Jain CLA 2012-05-18 08:48:43 EDT
(In reply to comment #8)
> Ok, I've removed JDT/Core from Gerrit.
Thank you! W00t! :)
Comment 10 Tomasz Zarna CLA 2012-05-21 05:30:51 EDT
(In reply to comment #4)
> > Is there a reason why this bug's security settings should disallow anyone
> > other than reporter and those on CC list to view it ?
> 
> It was like that when I got it.  I don't see any reason for this bug to be
> restricted, since it doesn't cover any potential security issue, but perhaps
> Tomasz has some rationale?

I don't remember ticking that checkbox, if I did that it was definitely by mistake.

(In reply to comment #5)
> Tomasz, this request came at a bad time of the project cycle. During RC phases,
> no experiments are allowed.

Sorry, didn't consider this as a potentially harmful experiment. Enabling Gerrit for Platform/Team[1] and Platform/Resources[2] went smoothly.

> Furthermore, the team lead (Srikanth) should be the only one requesting changes
> that affect all committers, and the webmaster should not take such wide-ranging
> requests from other people (especially not from non-committers on the project).

You're right, but I added Srikanth and Dani to +1 the request. Just like I did with the request for Platform/Team[1]. I was not expecting the webmaster to act that fast ;) According to instructions on how to enable Gerrit[3] anyone can file the bug but a confirmation from the PL is needed before proceeding.

Sorry for the trouble guys.
 
[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=371762
[2] https://bugs.eclipse.org/bugs/show_bug.cgi?id=371767
[3] http://wiki.eclipse.org/Gerrit#Enabling_Gerrit_for_your_Eclipse.org_Project
Comment 11 Markus Keller CLA 2012-05-21 06:43:12 EDT
(In reply to comment #6)
> Actually it doesn't.  Gerrit supports the idea of allowing 'pushes' that bypass
> code review(no +1 required), but it doesn't like 'out of band'(non-Gerrit)
> pushes to the repo(although experience has demonstrated that it's possible). 
> That's what the change in group ownership prevents. 

OK, so it seems that the "direct access" policy has changed since
(bug 371762 comment #2)
> You can continue to use your current URLs to bypass Gerrit.

Matt, could you please update http://wiki.eclipse.org/Gerrit#Enabling_Gerrit_for_your_Eclipse.org_Project ?

It should tell that "push changes directly to the git repo" doesn't mean that the old URLs continue to work, but that committers have to change their repo URLs and SSH keys. And it should have a fourth step:
- Negotiate a date/time when all committers have to switch their repo connection.
Comment 12 Dani Megert CLA 2012-05-21 10:33:19 EDT
> Matt, could you please update
> http://wiki.eclipse.org/Gerrit#Enabling_Gerrit_for_your_Eclipse.org_Project ?
> 
> It should tell that "push changes directly to the git repo" doesn't mean that
> the old URLs continue to work, but that committers have to change their repo
> URLs and SSH keys. And it should have a fourth step:
> - Negotiate a date/time when all committers have to switch their repo
> connection.

Would be good to track this in a separate bug.
Comment 13 Srikanth Sankaran CLA 2012-05-21 20:49:08 EDT
(In reply to comment #10)
> (In reply to comment #4)

> Sorry for the trouble guys.

Thanks for the clarification on this Tomasz. Don't beat yourself about
it, it took three cooks to dish this one out.

    - The idea should have been first proposed and discussed in
JDT/Core development list (https://dev.eclipse.org/mailman/listinfo/jdt-core-dev) before raising the request.
    - Webmaster should have checked with the team lead for a +1
    - Team lead should have had a clue about potential impacts :)

Just past 3.8 shipment, let us revisit this.
Comment 14 Eclipse Webmaster CLA 2012-05-22 16:21:32 EDT
(In reply to comment #11)

Ok, I've updated the Wiki page.

-M.
Comment 15 Srikanth Sankaran CLA 2012-07-18 04:49:01 EDT
Hello Webmaster, With things being quiescent, now it would seem is good 
time for this exercise. The team has also undergone basic orientation
on Gerrit (Thanks Tomek) and can jump onboard.

We want the option for project committers to be able to bypass the Gerrit
code review system and push changes directly to the git repo.

We can get onboard immediately, if a date is needed, let us say 20th Jul.

Thanks in advance for your help with this. Tom, please help team members
so teething trouble if any is minimized.
Comment 16 Eclipse Webmaster CLA 2012-07-18 11:14:52 EDT
Ok, done.  The URLs in comment 1 should be correct.

-M.
Comment 17 Tomasz Zarna CLA 2012-07-18 15:22:04 EDT
> ssh://userid@git.eclipse.org:29418/jdt/eclipse.jdt.core.git

This one works for me, I'm able to push and fetch from Gerrit using it.
Comment 18 Markus Keller CLA 2012-07-19 14:10:20 EDT
Looks like this broke the auto-tagging script in the SDK build, see bug 385536.
Comment 19 David Williams CLA 2012-07-19 14:14:19 EDT
Our builds are now failing, maybe related to this change. If you'll recall, our build ID was given "permission" to write tags to every repo in Eclipse. Not sure if that was via hooks, or ACLs, but comparing ACLs of jdt.core and jdt.core.binaries (just to pick one), I see e4Build is no longer a "user" in jdt.core. 

Can e4Build be added back? Will this somehow mess up "gerrit" because it assumes it has "complete control"?  



$ getfacl eclipse.jdt.core.git
# file: eclipse.jdt.core.git
# owner: gerrit
# group: cvs
# flags: -s-
user::rwx
group::rwx
other::r-x
default:user::rwx
default:group::rwx
default:other::r-x


       [14:08:37] david_williams@build:/home/data/git/jdt

$ getfacl eclipse.jdt.core.binaries.git/
# file: eclipse.jdt.core.binaries.git/
# owner: kmoir
# group: eclipse.jdt.core
# flags: -s-
user::rwx
user:e4Build:rwx
group::rwx
mask::rwx
other::r-x
default:user::rwx
default:user:e4Build:rwx
default:group::rwx
default:mask::rwx
default:other::r-x
Comment 20 Eclipse Webmaster CLA 2012-07-19 14:34:04 EDT
(In reply to comment #19)

> Will this somehow mess up "gerrit" because it  assumes it has "complete 
> control"?  

While it's been demonstrated that it is possible to commit code completely outside of Gerrit, the 'best practice' is to make such a thing impossible.

I'm willing to re-add the e4build user, but I'm be worried that there would be some conflict which would leave the repo a smoking hulk.

Could the Gerrit Hudson user be pressed into service for this?

-M.
Comment 21 David Williams CLA 2012-07-19 14:59:36 EDT
(In reply to comment #20)
> (In reply to comment #19)
> 
> > Will this somehow mess up "gerrit" because it  assumes it has "complete 
> > control"?  
> 
> While it's been demonstrated that it is possible to commit code completely
> outside of Gerrit, the 'best practice' is to make such a thing impossible.
> 
> I'm willing to re-add the e4build user, but I'm be worried that there would be
> some conflict which would leave the repo a smoking hulk.
> 
> Could the Gerrit Hudson user be pressed into service for this?
> 
> -M.

Well, remember, these are just tags, not code, so risk is probably less. (But, I agree not good ... good reason not to use Gerrit IMHO :) ... anything that needs "complete control" can't be too robust :) 

I'm not sure what "Gerrit Hudson user" is, but remember we are not really building on/in/from Hudson ... just cron jobs on build.eclipse.org with the e4Build Id. At a minimum, it sounds like, we'd have to get whole platform to move to Gerrit, and then do our builds on Hudson? Or ... give the e4Build ID some ability to SSH or sudo to Gerrit Hudson user, which a) doesn't sound secure and b) is a lot of work for this one "little" :) project. 

I'm willing to risk "direct access" if JDT Team is?
Comment 22 Jay Arthanareeswaran CLA 2012-07-19 15:04:40 EDT
(In reply to comment #20)
> I'm willing to re-add the e4build user, but I'm be worried that there would be
> some conflict which would leave the repo a smoking hulk.
> 
> Could the Gerrit Hudson user be pressed into service for this?

We had Gerrit enabled for JDT/Core briefly some time ago, and if I remember correct, we build was done in that state. Is that because the "e4build" user was available?
Comment 23 Eclipse Webmaster CLA 2012-07-19 15:08:25 EDT
Probably.

-M.
Comment 24 Srikanth Sankaran CLA 2012-07-19 19:40:16 EDT
(In reply to comment #21)

> I'm willing to risk "direct access" if JDT Team is?

So why is this not an issue for say Platform/Resource who I believe
have moved to Gerrit ?
Comment 25 Dani Megert CLA 2012-07-20 03:07:22 EDT
Something is screwed here. Other known projects which use Gerrit, don't have the permissions messed up. Here's how it should look like:

dmegert@build:/gitroot/platform> getfacl eclipse.platform.team.git/
# file: eclipse.platform.team.git/
# owner: kmoir
# group: eclipse.platform.team
# flags: -s-
user::rwx
user:gerrit:rwx
user:e4Build:rwx
group::rwx
mask::rwx
other::r-x
default:user::rwx
default:user:gerrit:rwx
default:user:e4Build:rwx
default:group::rwx
default:mask::rwx
default:other::r-x

Currently, committers with shell access can't even go and modify anything in jdt.core (e.g. hooks).
Comment 26 Dani Megert CLA 2012-07-20 03:08:40 EDT
(In reply to comment #25)
> Here's how it should look like:
Of course with the correct group:
# group: eclipse.jdt.core

And yes, the build user needs to be added as well.
Comment 27 Dani Megert CLA 2012-07-20 03:15:05 EDT
*** Bug 385536 has been marked as a duplicate of this bug. ***
Comment 28 Denis Roy CLA 2012-07-20 09:21:55 EDT
I've fixed the permissions:

build:/gitroot/jdt # getfacl eclipse.jdt.core.git/
# file: eclipse.jdt.core.git/
# owner: kmoir
# group: eclipse.jdt.core
# flags: -s-
user::rwx
user:gerrit:rwx
user:e4Build:rwx
group::rwx
mask::rwx
other::r-x
default:user::rwx
default:user:gerrit:rwx
default:user:e4Build:rwx
default:group::rwx
default:mask::rwx
default:other::r-x


In all honesty, I do not like these permissions where a web application, a build process _and_ committers all have full access to the repository.  To me it smells like a recipe for corruption.
Comment 29 David Williams CLA 2012-07-20 12:24:41 EDT
(In reply to comment #28)
> I've fixed the permissions:
> ...
> In all honesty, I do not like these permissions where a web application, a
> build process _and_ committers all have full access to the repository.  To me
> it smells like a recipe for corruption.

In case anyone was wondering, the fixed permissions did fix our build problem (that is, enabled our auto-tagging to work again). 

I think there is some risk of corruption, but don't think its the access, per se ... my concern is that we (most of us) expect the version in /gitroot to be THE one true canonical version of the repo but I've heard rumors that Gerrit has its own internal clone of the repo (which is what allows it to "roll back" changes, etc.) so I think the risk is if Gerrit considers _its_ internal repo to be the one accurate canonical version then the two could get "out of sync" (and hence appear corrupted) such as if a committer modified a part of a repo at the same time another committer used Gerrit to modify same part of the repo. Not sure it is likely, but sounds possible. 

Greatest thanks for your continued, patient, support.
Comment 30 Matthias Sohn CLA 2013-01-03 06:18:52 EST
(In reply to comment #29)
> (In reply to comment #28)
> > I've fixed the permissions:
> > ...
> > In all honesty, I do not like these permissions where a web application, a
> > build process _and_ committers all have full access to the repository.  To me
> > it smells like a recipe for corruption.
> 
> In case anyone was wondering, the fixed permissions did fix our build
> problem (that is, enabled our auto-tagging to work again). 
> 
> I think there is some risk of corruption, but don't think its the access,
> per se ... my concern is that we (most of us) expect the version in /gitroot
> to be THE one true canonical version of the repo but I've heard rumors that
> Gerrit has its own internal clone of the repo (which is what allows it to
> "roll back" changes, etc.) so I think the risk is if Gerrit considers _its_
> internal repo to be the one accurate canonical version then the two could
> get "out of sync" (and hence appear corrupted) such as if a committer
> modified a part of a repo at the same time another committer used Gerrit to
> modify same part of the repo. Not sure it is likely, but sounds possible. 

AFAIK there is no additional Gerrit internal repository, instead webmaster symlinks the canonical one under gitroot into Gerrit's file system tree so that Gerrit can access it. 

When allowing git write access over gitroot the main disadvantage is that it makes understanding permission settings more difficult since access over Gerrit is controlled by Gerrit permission settings whereas direct access over gitroot is controlled by file system permissions. Also Gerrit allows more fine grained permission control [1].

[1] https://git.eclipse.org/r/Documentation/access-control.html