Biowulf High Performance Computing at the NIH
Managing Access Control Lists (ACLs) on the NIH HPC Systems


Access Control Lists (ACLs) is an extension of the traditional UNIX permission concept, and allows more complex and sophisticated access to files under Linux. Specifically, ACLs make it possible to grant indidividual users or groups access to single files or directories. Moreover, they afford selective control over read, write, and execute permissions. This gives much more sophisticated control for sharing data between users on our systems.

Currently, ACLs are available ONLY on our GPFS (/gs*) and VAST (/vf) filesystems. Our NFS file system (/spin1) does NOT support ACLs.

Filesystem Limitations

There are occasions where traditional UNIX permissions are not enough for sharing files with multiple users. If you want to share a file or a directory with multiple users who are not part of the same group, there is very little that can be done.

For users on the GPFS filesystems, ACLs allow setting access permissions on an per-individual and -group basis. At the moment, this is not possible on the NetApp (/spin1) filesystem.

You can determine what filesystem your /data directory resides in using the command pwd -P while currently in that directory:

[user]$ cd /data/user
[user]$ pwd -P
/gpfs/gsfs4/users/user

Also, checkquota -a will show the absolute path to your directories as well:

[user]$ checkquota -a
Mount                          Used      Quota  Percent    Files    Limit
/gs3/users/user:           332.5 GB   360.0 GB   92.37% 19952175      n/a
/home:                       9.2 GB    12.0 GB   76.92%   140220      n/a
/spin1/users/OtherGroup:     6.0 TB     8.5 TB   70.59%   594431 31876689

NOTE: /home directories do NOT SUPPORT ACLs. Additionally, they should not be shared between users.

back to top

Allow another user access

Let's create a file. By default, the file is only accessible to the user who created it:

[user]$ echo "This is the file" > file
[user]$ ls -l file
-rw-------  1 user group 17 Sep 18 17:33 file

Your friend does not have access to that file:

[friend]$ cat file
cat: file: Permission denied

The command setfacl can be used to create and set ACLs on a file (or directory):

[user]$ setfacl --modify user:friend:rw- file
$ ls -l file
-rw-rw----+ 1 user group 17 Sep 18 17:33 file

In this case, we are including the option --modify with the setfacl command. This option allows us to modify the ACLs of a file or directory.

NOTE: Notice that the standard group permissions have changed from --- to rw-. This is because the information used to enable ACLs is "stored" in the Unix group permissions. DO NOT MODIFY THE GROUP PERMISSIONS UNLESS YOU KNOW WHAT YOU ARE DOING. See below for more information.

ALSO NOTE: If you hadn't noticed, a '+' character is added to the Unix permissions string. This indicates that ACLs are set for that file or directory.

The command getfacl can be used to display ACLs on a file (or directory):

[user]$ getfacl file
# file: file
# owner: user
# group: group
user::rw-
user:friend:rw-
group::---
mask::rw-
other::---

Now your friend can read the file:

[friend]$ cat file
This is the file

back to top

Allow another group access

The same command setfacl can be used to add access to an arbitrary group as well:

[user]$ setfacl --modify group:Friends:rw- file
[user]$ getfacl file
# file: file
# owner: user
# group: group
user::rw-
user:friend:rw-
group::---
group:Friends:rw-
mask::rw-
other::---

If a user is a member of a group, and the user and group both have ACLs for a file or directory, individual ACLs for the user override that of the group:

[user]$ groups friend
friend : friend Friends
[user]$ setfacl --modify group:Friends:r--,user:friend:rw- file 
[user]$ getfacl file
# file: file
# owner: user
# group: group
user::rw-
user:friend:rw-
group::---
group:Friends:r--
mask::rw-
other::---

Now user friend can write to and delete the file, while all other members of the Friends group can only read, even though friend is a member of Friends.

Reversing the ACLs allows all members of the Friends group to write, but restrict friend to read-only:

[user]$ setfacl --modify group:Friends:rw-,user:friend:r-- file 
[user]$ getfacl file
# file: file
# owner: user
# group: group
user::rw-
user:friend:r--
group::---
group:Friends:rw-
mask::rw-
other::---

back to top

Propagation of Access Via Defaults

ACLs can be set to propagate permissions to newly created files and subdirectories. This is done using default ACLs.

Here, we create a directory, which initially is only accessible to the original user:

[user]$ mkdir my_shared_directory
[user]$ ls -ld my_shared_directory
drwx------ 2 user group 4096 Sep 18 17:41 my_shared_directory

Now we set ACLs to allow our friend to read and execute the directory, and set default ACLs so that any new files or directories retain the same ACLs from the parent:

[user]$ setfacl --modify u:friend:r-x my_shared_directory
[user]$ setfacl --modify d:u:friend:r-x my_shared_directory
[user]$ ls -ld my_shared_directory
drwxr-x---+ 2 user group 4096 Sep 18 17:41 my_shared_directory
[user]$ getfacl my_shared_directory
# file: my_shared_directory
# owner: user
# group: group
user::rwx
user:friend:r-x
group::---
mask::r-x
other::---
default:user::rwx
default:user:friend:r-x
default:group::---
default:mask::r-x
default:other::---
This procedure can also be used to set default group permissions. This may be especially useful for groups with shared data directories. The following commands will help ensure that group members have sufficient permissions to access new files and directories created by colleagues.
[user]$ setfacl --modify group:GroupName:r-X /data/GroupName/
[user]$ setfacl --modify d:group:GroupName:r-X /data/GroupName/

NOTE: Again, the standard Unix permissions for the group have been changed to r-x.

ALSO NOTE: Your umask value is ignored when new files or directories are created within a directory with default ACLs. If you want to keep a file from being accessible to those designated in the default ACLs, you will need to manually change the permissions with chmod.

One other thing. Keep in mind that ACLs for a given user or group are independent of default ACLs. You can modify the default ACLs by including the --default option to setfacl.

Now that we've got a directory set with default ACLs, if we create a file, it will by default be accessible to our friend:

[user]$ echo "This is the other file" > my_shared_directory/newfile
[friend]$ ls -l my_shared_directory/newfile
-rw-r-----+ 1 user group 23 Sep 18 17:45 my_shared_directory/newfile
[friend]$ getfacl my_shared_directory/newfile
# file: my_shared_directory/newfile
# owner: user
# group: group
user::rw-
user:friend:r-x			#effective:r--
group::---
mask::r--
other::---
[friend]$ cat my_shared_directory/newfile
This is the other file

NOTE: The newly created file newfile is read-only to our friend. Although the default ACLs allow executable access, the effective permission for friend is read-only (r--).

If the standard Unix permissions of newfile are changed to allow group execution, the mask is changed, and friend will obtain executable permission (note than only user can change the Unix permissions, since user is the owner of the file):

[user]$ chmod ug+x my_shared_directory/newfile
[user]$ getfacl my_shared_directory/newfile
# file: my_shared_directory/newfile
# owner: user
# group: group
user::rwx
user:friend:r-x
group::---
mask::r-x
other::---

The default access properties are propagated to all subdirectories as well:

[user]$ mkdir my_shared_directory/subdir
[user]$ echo "This is the bottom file" > my_shared_directory/subdir/another_newfile
[friend]$ ls -l my_shared_directory/subdir/another_newfile
-rw-r-x---+ 1 user group 24 Sep 18 17:46 my_shared_directory/subdir/another_newfile
[friend]$ cat my_shared_directory/subdir/another_newfile
This is the bottom file
[friend]$ getfacl my_shared_directory/subdir/another_newfile
# file: my_shared_directory/subdir/another_newfile
# owner: user
# group: group
user::rwx
user:friend:r-x
group::---
mask::r-x
other::---
default:user::rwx
default:user:friend:r-x
default:group::---
default:mask::r-x
default:other::---

back to top

Default group permissions

ACLs supplant and enhance the standard UNIX group permissions with a combination of ACL entries and the mask property. When ACLs are first applied, the standard UNIX group permissions were changed g+rw-:

-rw-rw----+ 1 user group 17 Sep 18 17:33 file

Changing the group permissions will invalidate the ACLs set for a file or directory, as this command alters the mask So, don't mess with the group permissions if you are working with ACLs.

If you need to change the default group permissions of a file or directory, leave the group blank in the setfacl command. For example, to remove write and execute access to a file for the default group, but allow read while maintaining full access access for the owner and all ACL entries:

[user]$ setfacl -m g::r-- file

back to top

Removing ACLs

ACLs can be removed with setfacl --remove-all. This command will remove all ACLs.

[user]$ setfacl --remove-all file

To selectively remove ACLs, use setfacl --remove:

[user]$ setfacl --remove user:friend file

or

[user]$ setfacl -x u:friend file

back to top

Troubleshooting

ACLs are bound to the file or directory. When that file or directory is moved with the mv command, or copied with the cp -p command, the ACLs travel with it. The pre-existing (or non-existing) ACLs on the source file/directory are not changed even if the target directory has default ACLs set. This can lead to mixtures of ACL settings on files within a single directory.

Reapplying ACLs on files and directories within a shared /data directory is necessary to overcome the problems associated with moving files and directories.

back to top

More Information

Other useful options to getfacl and setfacl:

To learn more about setfacl and getfacl, type:

man setfacl

or

man getfacl

Here are a few articles on ACLs from the internet: