Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 157931 - Fix the 16-group limit for NFS [was: User nickb has too many ACLs/groups on dev.eclipse]
Summary: Fix the 16-group limit for NFS [was: User nickb has too many ACLs/groups on d...
Status: RESOLVED FIXED
Alias: None
Product: Community
Classification: Eclipse Foundation
Component: CommitterTools (show other bugs)
Version: unspecified   Edit
Hardware: PC Linux
: P3 major (vote)
Target Milestone: ---   Edit
Assignee: Eclipse Webmaster CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-09-19 22:14 EDT by Nick Boldt CLA
Modified: 2007-11-09 19:36 EST (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Boldt CLA 2006-09-19 22:14:00 EDT
$ cvs ci -m "testing file permissions" features-emf.xml
Checking in features-emf.xml;
/cvsroot/callisto/org.eclipse.callisto.tools/build-home/features-emf.xml,v  <--  features-emf.xml
new revision: 1.18; previous revision: 1.17
cvs [commit aborted]: could not open lock file `/cvsroot/callisto/org.eclipse.callisto.tools/build-home/,features-emf.xml,': Permission denied

I've got group access to callisto-dev, so I should be able to update this file:

-r--rwxr--+ 1 david_williams callisto-dev 6159 2006-09-19 22:14 features-emf.xml,v
Comment 1 Dani Megert CLA 2006-09-20 03:14:02 EDT
The server is not managed by simple file permissions but by ACLs. Here's a copy of an older mailing list message from the webmaster:


4. ACLs and committer access
============================

Committers and Project Leads using the SSH console should beware of
using ls -l to determine file permissions and ownership.  We use Access
Control Lists (ACLs) on some directories to manage multiple group owners
of files and directories.

You can run getfacl (dirname) to see the complete set of owners and
permissions.
Comment 2 Eclipse Webmaster CLA 2006-09-20 09:47:19 EDT
Daniel Megert earns major points for remembering that.

That being said, Nick, you're in 20 groups now, which causes a problem for NFSv3's stupidly retarded limit of 16.

Anyway, you're good to go with Callisto, but the last four groups listed here could be problematic in the future.  If there are groups you don't need to be in, let me know and I'll remove them.

Groups (20): xsd,emf-dev,xsd-edit,emf-edit,emf-home,xsd-home,uml2-home,uml2-releng,emft-dev,emft-home,emftadmin,emft-eodm,emft-net4j,emft-releng,emft-jet,emft-build,callisto-dev,modeling-home,modelingadmin,emft-teneo
Comment 3 Nick Boldt CLA 2006-09-21 14:17:53 EDT
Actually, it's 21 groups, but several are overlapping or redundant. 

Project groups I need (committer for 2 projects): 

emf (is this the same as emf-dev? or the equivalent of a *admin account, dating to before the *admin convention was started?)
xsd (shouldn't this be called xsd-dev? or rolled into emf since they're the same project now?)

Web groups I need (5 websites):

emf-home, xsd-home, uml2-home, emft-home, modeling-home
(xsd-home could be rolled into emf-home; no need to have these owned by 2 different ACLs/groups; in fact, you could remove eclipse.org/xsd from being in CVS and replace it with an redirect to eclipse.org/emf)

Releng groups I need (2 projects - I guess EMF doesn't have a separate group for releng?):

uml2-releng
emft-releng

Admin groups I use (3 projects):

* callisto-dev
* emftadmin (create new folders in /cvsroot/technology/org.eclipse.emft/, which could be done by opening a bug instead)
* modelingadmin (create new folders in /cvsroot/modeling/, which could be done by opening a bug instead)

Groups I've probably never used (5):

emf-dev (how's that different from group emf?)
emft-dev (how's that different from emftadmin?)
emft-build (how is that different from emft-releng?)
xsd-edit (?)
emf-edit (?)

Groups that are useful, but not strictly required (4) - I'm not a committer, but having access makes doing releng work easier; perhaps if access could flow down from emftadmin I wouldn't need to be manually added to each new project as they're added:

emft-eodm, emft-net4j, emft-jet, emft-teneo

I'm also a committer for /cvsroot/eclipse/org.eclipse.releng.basebuilder/plugins/org.eclipse.build.tools/, but that apparently doesn't come a new group as well, just CVS access. ;-)
Comment 4 Nick Boldt CLA 2006-09-21 14:20:03 EDT
Renam, reassign, and downgrade bug as this is no longer a major Callisto issue.
Comment 5 Denis Roy CLA 2007-03-06 15:22:02 EST
I actually spent some time on this.  Here's what I did:

- removed the emf group, and assigned all owned files to emf-dev.  project-dev is a standard for us, so emf was redundant

- removed the xsd group, and assigned all owned files to emf-dev

- created emfadmin, as it's a standard for us.  Assigned ownership to download.eclipse.org:tools/emf to emfadmin

- removed emf-edit and xsd-edit, as no files we owned by these groups

- removed xsd-home, and assigned all owned files to emf-home

- removed emft-build, as it didn't have any assigned files

One would need to examing the modeling project's directory structure vs. modeling groups, but one serious confusion point is the presence of modeling files in tools CVS and tools downloads area.  I do believe you are working on this, but it would be great if modeling files could all be found under the modeling top-level project.

Thoughts?
Comment 6 Nick Boldt CLA 2007-04-14 15:31:25 EDT
(In reply to comment #5)
> I actually spent some time on this.  Here's what I did:

Looks great, thanks.

> One would need to examing the modeling project's directory structure vs.
> modeling groups, but one serious confusion point is the presence of modeling
> files in tools CVS and tools downloads area.  I do believe you are working on
> this, but it would be great if modeling files could all be found under the
> modeling top-level project.

There are two moves that ought to happen, but will probably not happen immediately as I've got lots of other migrations on the go already. http://wiki.eclipse.org/index.php/Modeling_Project_Releng_Plan

a) EMF: /cvsroot/tools/org.eclipse.emf, org.eclipse.emf.ecore.sdo, org.eclipse.xsd, org.eclipse.emf.releng.build --> /cvsroot/modeling/org.eclipse.emf/*
b) EMFT: /cvsroot/technology/org.eclipse.emft -> /cvsroot/modeling/org.eclipse.*/* (split into the projects where the incubating bits will eventually live after they exit incubation)

(a) isn't urgent for me because nothing would change in terms of the outside world... though I can see it's "unclean" from your point of view, and I agree. ;-)

(b) this one's a work-in-progress. EODM and OCL have already moved into MDT. JET, JETEditor will be moved into M2T soon; Query, Transaction and Validation are moving soon into EMF, once I solve jar signing. The remaining 3 components are still incubating so as with EMF (no outside visibility), they're a low priority for me. New projects (emf.compare and emf.jcrm) are going to start in /cvsroot/modeling instead of /cvsroot/technology, which means once all the current work is done, we'll have just emf.emf, emft.teneo, emft.net4j, and emft.cdo left to move.

Regarding tools downloads area (old EMF downloads), I tried to purge everything but there was some blowback (read: a few open bugs) when the old update manager site disappeared and when links to zips moved. So I believe the resolution was to have old copies in the old places for now. If symlinking worked via the web, I could move and link from the old place to save space. Is there any way to implement a half-way solution here with .htaccess, like having symlinks be valid in some cases? Or, could I use hard links instead? I've never used them before but this article suggests this might be the solution, if it's allowable: http://linuxgazette.net/105/pitcher.html.

Back on the topic of group ids, I recently noticed something is amiss:

$ ll /cvsroot/modeling/
total 1.5K
drwxrwxr-x+  3 root          8428 1.2K 2006-11-01 13:33 CVSROOT
drwxrwsr-x+  2 khussey mdt-releng   80 2006-11-21 17:29 org.eclipse.mdt.releng
drwxrwxr-x+  5 khussey       8428  176 2007-01-05 09:17 org.eclipse.m2m
drwxrwsr-x+ 12 khussey mdt-dev     456 2007-01-16 12:16 org.eclipse.mdt
drwxrwxr-x+ 10 root    cvs         304 2007-02-20 13:26 .
drwxrwsr-x+  7 pelder  m2t-dev     264 2007-02-21 09:37 org.eclipse.m2t
drwxr-xr-x  14 droy    root        432 2007-03-05 14:25 ..
drwxrwsr-x+ 10 emerks        8428  272 2007-03-23 17:05 org.eclipse.gmf
drwxrwsr-x+  6 emerks        8428  248 2007-04-09 17:44 releng-common
drwxrwsr-x+ 12 emerks        8428  520 2007-04-13 14:26 org.eclipse.emf

What is group 8428?

Also:

$ groups nickb
id: cannot find name for group ID 8069

Cheers,

Nick
Comment 7 Denis Roy CLA 2007-04-16 09:55:28 EDT
I've tried to fix the missing groups as best I could.
Comment 8 Denis Roy CLA 2007-08-21 08:57:05 EDT
Allow me to rename this bug to better reflect the problem.  Currently, NFSv3's lame authentication (or at the very least, the one we have chosen) has a 16-group limitation that users can belong to. When authenticating a user's access to a file, only a users first 16 groups are communicated to the NFS server, leading to "access denied" errors.

We have examined a number of options, and have determined that the path of least resistance is to simply upgrade to NFSv4 with Kerberos authentication.  We need to sandbox and test this correctly, but the move to NFSv4 is fairly low-risk, as we can simply revert to NFSv3 quite easily.
Comment 9 Denis Roy CLA 2007-10-19 09:37:52 EDT
Although switching to NFSv4 is low-risk, the implementation of Kerberos is not trivial in our environment. After a couple of days of messing around with it in sandboxes, I still haven't accomplished much, despite wading through tons of new terminology and command-line tools.  I'm sure it's probably just my lack of experience with Kerberos, but on to other things...

I downloaded a kernel patch that "fixes" the 16-group limit by simply anticipating which group the NFS server wants to see.  In sandbox tests, the patch worked like a charm and I have compiled and installed a custom kernel on node1. Using Kim Moir's account (she's in 75 groups) I was able to access files that were owned by the 47th and 75th group she is in (carbon-ui and orbit-home) without using clumsy user-based ACLs. The same tests failed on node2 (Access Denied).

Chances are we'll drop the Kerberos/NFSv4 idea for now and simply maintain a patched kernel. Kerberos would be really nice to have for the vastly improved security, but for now we need something that works with minimal complexity.
Comment 10 Denis Roy CLA 2007-11-09 19:36:35 EST
This is now fixed, and all the ia64 nodes (CVS, downloads, etc) are running a patched kernel that bypasses the 16-group limit. I tested with Kim Moir's account, and even with the 75th group, her write access was maintained.

We'll no longer be maintaining ACLs.