HDFS-5625. Write end user documentation for HDFS ACLs. Contributed by Chris Nauroth.

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-4685@1567421 13f79535-47bb-0310-9956-ffa450edef68

hadoop-common-project/hadoop-common/src/site/apt/FileSystemShell.apt.vm

@@ -231,6 +231,29 @@ get
 
    Returns 0 on success and -1 on error.
 
+getfacl
+
+   Usage: <<<hdfs dfs -getfacl [-R] <path> >>>
+
+   Displays the Access Control Lists (ACLs) of files and directories. If a
+   directory has a default ACL, then getfacl also displays the default ACL.
+
+   Options:
+
+     * -R: List the ACLs of all files and directories recursively.
+
+     * <path>: File or directory to list.
+
+   Examples:
+
+     * <<<hdfs dfs -getfacl /file>>>
+
+     * <<<hdfs dfs -getfacl -R /dir>>>
+
+   Exit Code:
+
+   Returns 0 on success and non-zero on error.
+
 getmerge
 
    Usage: <<<hdfs dfs -getmerge <src> <localdst> [addnl]>>>
@@ -379,6 +402,52 @@ rmr
 
    Returns 0 on success and -1 on error.
 
+setfacl
+
+   Usage: <<<hdfs dfs -setfacl [-R] [{-b|-k} {-m|-x <acl_spec>} <path>]|[--set <acl_spec> <path>] >>>
+
+   Sets Access Control Lists (ACLs) of files and directories.
+
+   Options:
+
+     * -b: Remove all but the base ACL entries. The entries for user, group and
+       others are retained for compatibility with permission bits.
+
+     * -k: Remove the default ACL.
+
+     * -R: Apply operations to all files and directories recursively.
+
+     * -m: Modify ACL. New entries are added to the ACL, and existing entries
+       are retained.
+
+     * -x: Remove specified ACL entries. Other ACL entries are retained.
+
+     * --set: Fully replace the ACL, discarding all existing entries. The
+       <acl_spec> must include entries for user, group, and others for
+       compatibility with permission bits.
+
+     * <acl_spec>: Comma separated list of ACL entries.
+
+     * <path>: File or directory to modify.
+
+   Examples:
+
+      * <<<hdfs dfs -setfacl -m user:hadoop:rw- /file>>>
+
+      * <<<hdfs dfs -setfacl -x user:hadoop /file>>>
+
+      * <<<hdfs dfs -setfacl -b /file>>>
+
+      * <<<hdfs dfs -setfacl -k /dir>>>
+
+      * <<<hdfs dfs -setfacl --set user::rw-,user:hadoop:rw-,group::r--,other::r-- /file>>>
+
+      * <<<hdfs dfs -setfacl -R -m user:hadoop:r-x /dir>>>
+
+   Exit Code:
+
+   Returns 0 on success and non-zero on error.
+
 setrep
 
    Usage: <<<hdfs dfs -setrep [-R] [-w] <numReplicas> <path> >>>

hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-4685.txt

@@ -70,6 +70,8 @@ HDFS-4685 (Unreleased)
     HDFS-5914. Incorporate ACLs with the changes from HDFS-5698.
     (Haohui Mai via cnauroth)
 
+    HDFS-5625. Write end user documentation for HDFS ACLs. (cnauroth)
+
   OPTIMIZATIONS
 
   BUG FIXES

hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsPermissionsGuide.apt.vm

@@ -47,6 +47,10 @@ HDFS Permissions Guide
    client process, and its group is the group of the parent directory (the
    BSD rule).
 
+   HDFS also provides optional support for POSIX ACLs (Access Control Lists) to
+   augment file permissions with finer-grained rules for specific named users or
+   named groups.  ACLs are discussed in greater detail later in this document.
+
    Each client process that accesses HDFS has a two-part identity composed
    of the user name, and groups list. Whenever HDFS must do a permissions
    check for a file or directory foo accessed by a client process,
@@ -219,9 +223,173 @@ HDFS Permissions Guide
    identity matches the super-user, parts of the name space may be
    inaccessible to the web server.
 
+* ACLs (Access Control Lists)
+
+   In addition to the traditional POSIX permissions model, HDFS also supports
+   POSIX ACLs (Access Control Lists).  ACLs are useful for implementing
+   permission requirements that differ from the natural organizational hierarchy
+   of users and groups.  An ACL provides a way to set different permissions for
+   specific named users or named groups, not only the file's owner and the
+   file's group.
+
+   By default, support for ACLs is disabled, and the NameNode disallows creation
+   of ACLs.  To enable support for ACLs, set <<<dfs.namenode.acls.enabled>>> to
+   true in the NameNode configuration.
+
+   An ACL consists of a set of ACL entries.  Each ACL entry names a specific
+   user or group and grants or denies read, write and execute permissions for
+   that specific user or group.  For example:
+
++--
+   user::rw-
+   user:bruce:rwx                  #effective:r--
+   group::r-x                      #effective:r--
+   group:sales:rwx                 #effective:r--
+   mask::r--
+   other::r--
++--
+
+   ACL entries consist of a type, an optional name and a permission string.
+   For display purposes, ':' is used as the delimiter between each field.  In
+   this example ACL, the file owner has read-write access, the file group has
+   read-execute access and others have read access.  So far, this is equivalent
+   to setting the file's permission bits to 654.
+
+   Additionally, there are 2 extended ACL entries for the named user bruce and
+   the named group sales, both granted full access.  The mask is a special ACL
+   entry that filters the permissions granted to all named user entries and
+   named group entries, and also the unnamed group entry.  In the example, the
+   mask has only read permissions, and we can see that the effective permissions
+   of several ACL entries have been filtered accordingly.
+
+   Every ACL must have a mask.  If the user doesn't supply a mask while setting
+   an ACL, then a mask is inserted automatically by calculating the union of
+   permissions on all entries that would be filtered by the mask.
+
+   Running <<<chmod>>> on a file that has an ACL actually changes the
+   permissions of the mask.  Since the mask acts as a filter, this effectively
+   constrains the permissions of all extended ACL entries instead of changing
+   just the group entry and possibly missing other extended ACL entries.
+
+   The model also differentiates between an "access ACL", which defines the
+   rules to enforce during permission checks, and a "default ACL", which defines
+   the ACL entries that new child files or sub-directories receive automatically
+   during creation.  For example:
+
++--
+   user::rwx
+   group::r-x
+   other::r-x
+   default:user::rwx
+   default:user:bruce:rwx          #effective:r-x
+   default:group::r-x
+   default:group:sales:rwx         #effective:r-x
+   default:mask::r-x
+   default:other::r-x
++--
+
+   Only directories may have a default ACL.  When a new file or sub-directory is
+   created, it automatically copies the default ACL of its parent into its own
+   access ACL.  A new sub-directory also copies it to its own default ACL.  In
+   this way, the default ACL will be copied down through arbitrarily deep levels
+   of the file system tree as new sub-directories get created.
+
+   The exact permission values in the new child's access ACL are subject to
+   filtering by the mode parameter.  Considering the default umask of 022, this
+   is typically 755 for new directories and 644 for new files.  The mode
+   parameter filters the copied permission values for the unnamed user (file
+   owner), the mask and other.  Using this particular example ACL, and creating
+   a new sub-directory with 755 for the mode, this mode filtering has no effect
+   on the final result.  However, if we consider creation of a file with 644 for
+   the mode, then mode filtering causes the new file's ACL to receive read-write
+   for the unnamed user (file owner), read for the mask and read for others.
+   This mask also means that effective permissions for named user bruce and
+   named group sales are only read.
+
+   Note that the copy occurs at time of creation of the new file or
+   sub-directory.  Subsequent changes to the parent's default ACL do not change
+   existing children.
+
+   The default ACL must have all minimum required ACL entries, including the
+   unnamed user (file owner), unnamed group (file group) and other entries.  If
+   the user doesn't supply one of these entries while setting a default ACL,
+   then the entries are inserted automatically by copying the corresponding
+   permissions from the access ACL, or permission bits if there is no access
+   ACL.  The default ACL also must have mask.  As described above, if the mask
+   is unspecified, then a mask is inserted automatically by calculating the
+   union of permissions on all entries that would be filtered by the mask.
+
+   When considering a file that has an ACL, the algorithm for permission checks
+   changes to:
+
+     * If the user name matches the owner of file, then the owner
+       permissions are tested;
+
+     * Else if the user name matches the name in one of the named user entries,
+       then these permissions are tested, filtered by the mask permissions;
+
+     * Else if the group of file matches any member of the groups list,
+       and if these permissions filtered by the mask grant access, then these
+       permissions are used;
+
+     * Else if there is a named group entry matching a member of the groups list,
+       and if these permissions filtered by the mask grant access, then these
+       permissions are used;
+
+     * Else if the file group or any named group entry matches a member of the
+       groups list, but access was not granted by any of those permissions, then
+       access is denied;
+
+     * Otherwise the other permissions of file are tested.
+
+   Best practice is to rely on traditional permission bits to implement most
+   permission requirements, and define a smaller number of ACLs to augment the
+   permission bits with a few exceptional rules.  A file with an ACL incurs an
+   additional cost in memory in the NameNode compared to a file that has only
+   permission bits.
+
+* ACLs File System API
+
+   New methods:
+
+     * <<<public void modifyAclEntries(Path path, List<AclEntry> aclSpec) throws
+       IOException;>>>
+
+     * <<<public void removeAclEntries(Path path, List<AclEntry> aclSpec) throws
+       IOException;>>>
+
+     * <<<public void public void removeDefaultAcl(Path path) throws
+       IOException;>>>
+
+     * <<<public void removeAcl(Path path) throws IOException;>>>
+
+     * <<<public void setAcl(Path path, List<AclEntry> aclSpec) throws
+       IOException;>>>
+
+     * <<<public AclStatus getAclStatus(Path path) throws IOException;>>>
+
+* ACLs Shell Commands
+
+     * <<<hdfs dfs -getfacl [-R] <path> >>>
+
+       Displays the Access Control Lists (ACLs) of files and directories. If a
+       directory has a default ACL, then getfacl also displays the default ACL.
+
+     * <<<hdfs dfs -setfacl [-R] [{-b|-k} {-m|-x <acl_spec>} <path>]|[--set <acl_spec> <path>] >>>
+
+       Sets Access Control Lists (ACLs) of files and directories.
+
+     * <<<hdfs dfs -ls <args> >>>
+
+       The output of <<<ls>>> will append a '+' character to the permissions
+       string of any file or directory that has an ACL.
+
+       See the {{{../hadoop-common/FileSystemShell.html}File System Shell}}
+       documentation for full coverage of these commands.
+
 * Configuration Parameters
 
-     * <<<dfs.permissions = true>>>
+     * <<<dfs.permissions.enabled = true>>>
 
        If yes use the permissions system as described here. If no,
        permission checking is turned off, but all other behavior is
@@ -255,3 +423,9 @@ HDFS Permissions Guide
 
        The administrators for the cluster specified as an ACL. This
        controls who can access the default servlets, etc. in the HDFS.
+
+     * <<<dfs.namenode.acls.enabled = true>>>
+
+       Set to true to enable support for HDFS ACLs (Access Control Lists).  By
+       default, ACLs are disabled.  When ACLs are disabled, the NameNode rejects
+       all attempts to set an ACL.

hadoop-hdfs-project/hadoop-hdfs/src/site/apt/WebHDFS.apt.vm

@@ -752,6 +752,148 @@ Content-Length: 0
    {{{../../api/org/apache/hadoop/fs/FileSystem.html}FileSystem}}.setTimes
 
 
+** {Modify ACL Entries}
+
+  * Submit a HTTP PUT request.
+
++---------------------------------
+curl -i -X PUT "http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=MODIFYACLENTRIES
+                              &aclspec=<ACLSPEC>"
++---------------------------------
+
+  The client receives a response with zero content length:
+
++---------------------------------
+HTTP/1.1 200 OK
+Content-Length: 0
++---------------------------------
+
+  []
+
+  See also:
+  {{{../../api/org/apache/hadoop/fs/FileSystem.html}FileSystem}}.modifyAclEntries
+
+
+** {Remove ACL Entries}
+
+  * Submit a HTTP PUT request.
+
++---------------------------------
+curl -i -X PUT "http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=REMOVEACLENTRIES
+                              &aclspec=<ACLSPEC>"
++---------------------------------
+
+  The client receives a response with zero content length:
+
++---------------------------------
+HTTP/1.1 200 OK
+Content-Length: 0
++---------------------------------
+
+  []
+
+  See also:
+  {{{../../api/org/apache/hadoop/fs/FileSystem.html}FileSystem}}.removeAclEntries
+
+
+** {Remove Default ACL}
+
+  * Submit a HTTP PUT request.
+
++---------------------------------
+curl -i -X PUT "http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=REMOVEDEFAULTACL"
++---------------------------------
+
+  The client receives a response with zero content length:
+
++---------------------------------
+HTTP/1.1 200 OK
+Content-Length: 0
++---------------------------------
+
+  []
+
+  See also:
+  {{{../../api/org/apache/hadoop/fs/FileSystem.html}FileSystem}}.removeDefaultAcl
+
+
+** {Remove ACL}
+
+  * Submit a HTTP PUT request.
+
++---------------------------------
+curl -i -X PUT "http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=REMOVEACL"
++---------------------------------
+
+  The client receives a response with zero content length:
+
++---------------------------------
+HTTP/1.1 200 OK
+Content-Length: 0
++---------------------------------
+
+  []
+
+  See also:
+  {{{../../api/org/apache/hadoop/fs/FileSystem.html}FileSystem}}.removeAcl
+
+
+** {Set ACL}
+
+  * Submit a HTTP PUT request.
+
++---------------------------------
+curl -i -X PUT "http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=SETACL
+                              &aclspec=<ACLSPEC>"
++---------------------------------
+
+  The client receives a response with zero content length:
+
++---------------------------------
+HTTP/1.1 200 OK
+Content-Length: 0
++---------------------------------
+
+  []
+
+  See also:
+  {{{../../api/org/apache/hadoop/fs/FileSystem.html}FileSystem}}.setAcl
+
+
+** {Get ACL Status}
+
+  * Submit a HTTP GET request.
+
++---------------------------------
+curl -i -X PUT "http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=GETACLSTATUS"
++---------------------------------
+
+  The client receives a response with a {{{ACL Status JSON Schema}<<<AclStatus>>> JSON object}}:
+
++---------------------------------
+HTTP/1.1 200 OK
+Content-Type: application/json
+Transfer-Encoding: chunked
+
+{
+    "AclStatus": {
+        "entries": [
+            "user:carla:rw-", 
+            "group::r-x"
+        ], 
+        "group": "supergroup", 
+        "owner": "hadoop", 
+        "stickyBit": false
+    }
+}
++---------------------------------
+
+  []
+
+  See also:
+  {{{../../api/org/apache/hadoop/fs/FileSystem.html}FileSystem}}.getAclStatus
+
+
 * {Delegation Token Operations}
 
 ** {Get Delegation Token}
@@ -980,6 +1122,52 @@ Transfer-Encoding: chunked
   However, if additional properties are included in the responses, they are
   considered as optional properties in order to maintain compatibility.
 
+** {ACL Status JSON Schema}
+
++---------------------------------
+{
+  "name"      : "AclStatus",
+  "properties":
+  {
+    "AclStatus":
+    {
+      "type"      : "object",
+      "properties":
+      {
+        "entries":
+        {
+          "type": "array"
+          "items":
+          {
+            "description": "ACL entry.",
+            "type": "string"
+          }
+        },
+        "group":
+        {
+          "description": "The group owner.",
+          "type"       : "string",
+          "required"   : true
+        },
+        "owner":
+        {
+          "description": "The user who is the owner.",
+          "type"       : "string",
+          "required"   : true
+        },
+        "stickyBit":
+        {
+          "description": "True if the sticky bit is on.",
+          "type"       : "boolean",
+          "required"   : true
+        },
+      }
+    }
+  }
+}
++---------------------------------
+
+
 ** {Boolean JSON Schema}
 
 +---------------------------------
@@ -1387,6 +1575,23 @@ var tokenProperties =
 
 * {HTTP Query Parameter Dictionary}
 
+** {ACL Spec}
+
+*----------------+-------------------------------------------------------------------+
+|| Name          | <<<aclspec>>> |
+*----------------+-------------------------------------------------------------------+
+|| Description   | The ACL spec included in ACL modification operations. |
+*----------------+-------------------------------------------------------------------+
+|| Type          | String |
+*----------------+-------------------------------------------------------------------+
+|| Default Value | \<empty\> |
+*----------------+-------------------------------------------------------------------+
+|| Valid Values  | See {{{./HdfsPermissionsGuide.html}Permissions and HDFS}}. |
+*----------------+-------------------------------------------------------------------+
+|| Syntax        | See {{{./HdfsPermissionsGuide.html}Permissions and HDFS}}. |
+*----------------+-------------------------------------------------------------------+
+
+
 ** {Access Time}
 
 *----------------+-------------------------------------------------------------------+