AWS S3 Access Management

By Paul Heinlein | Sep 7, 2016

Access control on Amazon S3 is subtle and complex. Here’s a high-level overview of the access controls that can be placed on S3 buckets and objects.

Background

Amazon Simple Storage Service (S3) behaves like a WebDAV filesystem. It’s ideal for files that are written once (or rarely) and read many times. Unlike a block-storage device, an S3 bucket has no fixed size; you’ll never see a “filesystem full” message from S3. There are even tools that allow a computer to make an S3 bucket look like a mounted directory. This approach makes reading from and writing to S3 seem like normal file operations, but no one should mistake S3 for a high-performance filesystem. S3 involves very high latencies and slow write operations.

Terms:

Beyond a simple file store, S3 offers some powerful services. S3 can

Controlling Access

S3 access controls are as complex as the underlying service. ACLs can be placed on lifecycle management, replication, tagging, web site availability, versioning, and even on ACLs themselves. The upside is that access policies and controls can be very subtle and complex; the downside is that users are given enough rope to hang themselves.

Of First Importance

It’s crucial to know the difference between policies and ACLs.

It’d be cool to offer a tool that could read a bucket’s policy and then list files within the bucket that override that policy.

Canned ACLs

Amazon suspects that most access controls will be similar, so S3 offers a set of predefined grants, aka canned ACLs. They can easily make a bucket private (the default), public read-only, public read-write, read-only for authenticated users, read-only for a specific EC2 instance, etc.

Chances are, most buckets will be well served by these ACLs. My recommendation is to turn to them first. The canned ACLs are also easy to manage from the Web GUI; they can also be managed from the command line using the AWS command line interface:

aws s3api put-bucket-acl \
  --acl private --bucket elizasbucket
aws s3api put-object-acl \
  --acl public-read --bucket elizasbucket --key publicdir/index.html

Policies: the Available Knobs

An S3 policy has four main sections:

  1. Resources covered: buckets, objects
  2. Action requested (read, write, etc)
  3. Test of conditions (optional)
  4. Effect: either Allow or Deny the request

Resources

In an S3 policy, access controls can be applied to

The wildcard syntax for specifying buckets or objects looks similar to that used by command-line tools to specify files:

Action Requested

The S3 REST API Introduction details all of the many possible actions that can be request. Generally speaking, actions apply to

Conditions and Tests

AWS makes available at each request a variety of information for testing. At a high level, that information covers

All those those variables can be tested, e.g.,

The IAM Policy Variables Overview provides more details about the specific conditions that can be tested.

Command-line Example

First, create a policy that allows anonymous read-only access to the site-src subdirectory in the bucket named bucket-02, with the exception that the site-src/mgmt subdirectory and its contents are off-limits except to users from a couple specific IPv4 addresses.

{
  "Version": "2012-10-17",
  "Id": "Policy1473195819235",
  "Statement": [
    {
      "Sid": "Stmt1473195808073",
      "Effect": "Allow",
      "Principal": { "AWS": "*" },
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::bucket-02/site-src/*",
    },
    {
      "Sid": "Stmt1473195808074",
      "Effect": "Deny",
      "Principal": { "AWS": "*" },
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::bucket-02/site-src/mgmt",
        "arn:aws:s3:::bucket-02/site-src/mgmt/*"
      ],
      "Condition":{
        "NotIpAddress":{
          "aws:SourceIp": [ "104.131.178.116", "162.209.76.176" ]
        }
      }
    }
  ]
}

The second staza in the statement, which denies most access to the mgmt subdirectory, uses a double-negative to accomplish its goal. “Deny everyone access who doesn’t come from IPv4 address 104.131.178.116 or 162.209.76.176.”

Then apply the policy:

# create our bucket
aws s3 mb s3://bucket-02
# apply our policy file
aws s3api put-bucket-policy --bucket bucket-02 \
  --policy file://bucket-policy.json
# upload our content
aws s3 sync /tmp/site-src s3://bucket-02/site-src

Resources