Forem: Ravindu Nirmal Fernando

Understanding inbuilt AWS S3 security controls and methods — Part 4

Ravindu Nirmal Fernando — Fri, 03 Feb 2023 13:45:42 +0000

View the version published on Medium

This article is the 4th and final part of a series of articles on Inbuilt AWS S3 security controls and methods. In this article, we focus on the encryption mechanisms available for AWS S3 resources.

AWS S3 provides several encryption mechanisms for their customers. You have to select between each of those mechanisms based on the use case and the criticality of the data you store in S3. Hence it's really important to understand what is happening behind the scenes and apply the correct encryption mechanism to secure your data. These can be mainly categorized as Server Side Encryption and Client Side Encryption.

We will first go through each of the Server Side Encryption mechanisms in detail.

1 — AWS Server Side Encryption with S3 Managed Keys (SSE-S3)

As announced on Jan 05th 2023, AWS will encrypt all new objects within all the buckets with SSE-S3 encryption unless another encryption mechanism was defined. You can read the announcement here — https://aws.amazon.com/blogs/aws/amazon-s3-encrypts-new-objects-by-default/

This is the new default and easiest-to-implement encryption method within AWS S3. All you have to do is enable this and AWS S3 will handle data encryption and decryption.

Underneath the following process occurs during the encryption and decryption process.

Encryption

1 — The client uploads data.

2 — On the S3 side a plain text S3 data key is generated and then data is encrypted with that key. The encrypted data object is then stored. This encryption is symmetric encryption.

3 — Then the plain text S3 data key is encrypted with S3 Master Key creating an encrypted S3 data key. Then that encrypted S3 data key is stored along with the encrypted object data and the plain text S3 data key is removed from memory.

Decryption

1 — The client requests data.

2 — On the S3 side, the encrypted data key associated with the object is taken and decrypted using the S3 Master Key to obtain the plain text S3 data key.

3 — The S3 plain text data key is then used to decrypt the encrypted object data and this object data is returned to the client.

2 — AWS Server Side Encryption with KMS Managed Keys (SSE-KMS)

This method allows S3 to use AWS KMS service to generate data encryption keys. This allows greater flexibility as you have complete control to disable, rotate apply access control to the client-managed master key used to generate encryption keys.

Underneath the following process occurs during the encryption and decryption process.

Encryption

1 — The client uploads the data

2 — S3 service then requests data keys from KMS — Customer Master Key (CMK).

3 — On the KMS side the CMK generates the plaintext data key and the encrypted data key.

4 — Then both of those generated keys are passed to S3.

5 — Within S3. the data object is encrypted with the plain text data key and both the encrypted data object and the encrypted data key are stored on S3 while removing the plain text data key from memory.

Decryption

1 — Client requests data.

2 — S3 service will send the encrypted data key of the object to KMS.

3 — KMS will then decrypt the encrypted data key using the CMK and obtain the plain text data key.

4 — Plain text data will be returned back to S3.

5 — On the S3 side the plain text data key will be used to decrypt the encrypted data object.

6 — The decrypted data object will be sent back to the client.

3 — AWS Server Side Encryption with Customer provided Keys (SSE-C)

This method gives you the ability to provide your own master key. Your master key would be sent with your data to S3 and S3 will use that key to perform encryption for you.

Encryption

1 — The client will send the object data along with the customer key to S3. This will only work via an HTTPS connection.

2 — On the S3 side, the data object is encrypted using the key sent by the customer. Also, a salted HMAC value of the key sent is created for future validation. Then both of the encrypted data object and salted HMAC value will be stored within S3. The customer key sent will be removed from the memory.

Decryption

1 — The client will request the data along with the customer key from S3. This will only work via an HTTPS connection.

2 — On the S3 side, the salted HMAC value of the customer key will be used to validate the customer key sent with the request. Once validated that customer key will be used to decrypt the encrypted object data.

3 — The object data will be returned to the customer.

We will now go through each of the Client Side Encryption mechanisms in detail.

1 — AWS Client Side Encryption with KMS managed keys (CSE-KMS)

This method uses AWS KMS to generate data encryption keys. The AWS KMS is called by the client and encryption happens on the client side. Then the encrypted object will be sent to S3.

Encryption

1 — The client will communicate with AWS KMS via an AWS SDK and request data keys. They need to pass the CMK-ID of the CMK stored within KMS with the request.

2 — On KMS, the CMK associated with the requested CMK-ID will be used to generate a plain text data key and cipher blob associated with the plain text data key.

3 — Both the plain text data key and cipher blob key will be sent to the client.

4 — On the client side the data object will be encrypted using the plain text data key received in the above step.

5 — Then the encrypted data and the cipher blob key are sent to S3.

6 — On S3, the encrypted data object will be stored. The cipher blob data key will be stored as metadata of the encrypted data object.

Decryption

1 — The client requests data

2 — S3 will then send the encrypted data and cipher blob key to the client.

3 — The client will then send the cipher blob key to AWS KMS using AWS SDK.

4 — The cipher blob key combined with the CMK, the associated plain text data key will be generated.

5 — The plaintext data key will then be sent to the client.

6 — On the client side, the plain text data key will be used to decrypt the encrypted object data received from S3.

2— AWS Client Side Encryption with Customer provided keys (CSE-C)

This method allows you to use your own key to encrypt data.

Encryption

1 — The data object will be encrypted using client generated plain text data key.

2 —Customer-managed CMK will be used to encrypt the client-generated plain text data key.

3 — The encrypted data from step 1 and the encrypted data key from step 2 will be sent to S3.

4 — Then the encrypted data object will be stored in S3 and the encrypted data key will be stored as metadata of the encrypted data object.

Decryption

1 — The client requests the data.

2 — S3 will send both the encrypted data object and the encrypted key back to the client.

3 — The client will then use the customer-managed CMK to generate the plain text data key by decrypting the encrypted data key.

4 — Then the plain text data key will be used to decrypt the encrypted data object.

This article series will come to an end with this article. Though our main focus was to discuss about inbuilt security controls and methods within S3, there are many other services within AWS which can be used to improve and manage the overall security of your S3 resources. These services include AWS Security Hub, AWS GuardDuty, AWS Macie, AWS Trusted Advisor etc… You can read more about the S3 security best practices in detail by visiting — https://docs.aws.amazon.com/AmazonS3/latest/userguide/security-best-practices.html
AWS continuously takes measures to improve the security posture of AWS S3. But the customer also has a major responsibility in ensuring the security of their S3 resources are configured and managed with proper security controls and methods. I hope this article series helped you to understand the importance of that.

There are many other methods and controls that were not covered here and can be used to improve the security posture of AWS S3. Plus these concepts and controls are bound to be continuously updated for the better. Hence it's always better to refer to several resources and specifically official AWS documentation to have a more in-depth idea of AWS S3 security.

References

1 — https://docs.aws.amazon.com/AmazonS3/latest/userguide/security.html

Understanding inbuilt AWS S3 security controls and methods - Part 3

Ravindu Nirmal Fernando — Sun, 06 Nov 2022 18:01:15 +0000

This article is outdated. Updated version published on Medium

In part 2 of this article series, we mainly discussed the access control policies that can be used to secure access to S3 buckets. In this article, we will focus on the access control methods which include both features and settings which S3 offers you to secure data in S3. These controls are crucial when you need to share your buckets among different applications and users or require public access to those.

We are going to cover up following topics in this article,

1 - S3 Access Points

2 - Public Access management setting in S3

1 - S3 Access Points

S3 is heavily used for many use cases to store shared data sets. These data sets are being accessed by individuals, groups, and applications hence the S3 buckets which store these data sets should have shared access. The management of access to these shared buckets requires maintaining a bucket policy, IAM policy, or ACLs. These policies may include access controls to hundreds to thousands of different users, groups, and applications depending on the use case. But as the users and application sets grow, managing these individual policies will be difficult and time-consuming mainly due to their complexity. This will also make it difficult to audit the changes to the policies.

AWS released the S3 Access Points feature within AWS S3 to mitigate the complexities discussed above. The service allows users to manage and control access to shared S3 buckets. The S3 access points can be attached to a single bucket and configured with separate access policies for that particular access point. This allows you the ability to create different access points with different permissions for teams and applications to your shared S3 bucket.

You have to note the following points when using S3 Access Points,

You can attach an access point only to a single bucket whereas a bucket can have multiple access points attached to itself.
Access points allow only object operations like S3 GetObject and S3 PutObject etc… and not bucket-specific operations like S3 DeleteBucket.
Access points can be configured to accept traffic from specific VPCs and also include the Block Public Access setting which is enabled by default.
The access point policies can be used to only allow access to objects with a defined prefix or to objects with specific tags. These policies are defined in JSON. However, these policies are valid if controls allowed within access point policies are also allowed in bucket policies. Hence if you are planning to use access point policies to manage access control for your shared bucket, it's better to use bucket policy to delegate access control of the bucket to the access point as shown below.
(Refer - https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-points-policies.html)
```
{
    "Version": "2012-10-17",
    "Statement" : [
    {
        "Effect": "Allow",
        "Principal" : { "AWS": "*" },
        "Action" : "*",
        "Resource" : [ "Bucket ARN", "Bucket ARN/*"],
        "Condition": {
            "StringEquals" : { "s3:DataAccessPointAccount" : "Bucket owner's account ID" }
        }
    }]
}
```
When accessing the resources using access points, for S3 object operations, you can use the access point ARN in place of a bucket name. For requests requiring a bucket name in the standard S3 bucket name format, you can use an access point alias instead. Both of these details are available when you create an access point.

Read more about S3 Access Points by visiting → https://aws.amazon.com/s3/features/access-points/

2 - Public Access management setting in S3

Over the years we have seen many incidents where sensitive information residing within S3 buckets has been exposed. This was mainly due to unprotected settings on the S3 bucket level which allowed general public access to the data within the S3 bucket. The public access management feature is available within AWS S3 to mitigate these types of configuration issues. This setting can be accessed from the permissions tab within your buckets setting and it's enabled by default when you are creating a bucket. You will have to actively change this setting to allow public access to your bucket.

As shown below, you can completely turn off this setting if you require public access to your bucket, or you can select a combination of options that can be used to filter public access to your S3 buckets.

One thing to note here is that if you are to provide public access or cross-account access to the bucket using bucket policy or ACL, then access will still not be granted if the Block public access settings are enabled.

Let’s deep dive into the Encryption mechanisms provided by AWS S3 in the upcoming article on this article series.

References

1 — https://aws.amazon.com/s3/features/access-points/

2 — https://cloudacademy.com/course/increasing-your-security-posture-when-using-amazon-s3-1235

Understanding inbuilt AWS S3 security controls and methods — Part 2

Ravindu Nirmal Fernando — Sat, 29 Oct 2022 13:23:15 +0000

Originally published on Medium

This article is the second part of a series of articles on understanding inbuilt AWS S3 security controls and methods. You can access part 1 of the series by visiting here.

In this article, we will shift our focus toward the access control policies that can be defined to control access to your AWS S3 buckets. Those can be mainly divided into two main components as follows,

1 — Identity-based policies

2 — Resource-based policies

Let’s go through each one of them and understand how can we leverage those to control access to our S3 buckets.

1 — Identity-based policies

Identity-based policies are the ones that can be directly attached to a user, a group that the user belongs or a role via the IAM service in AWS. These can be either inline or managed. Given below is a sample example of an IAM policy that allows the assigned identity to Put an Object, Get an Object, List an Object, Delete an Object and Get Bucket location access to the awsexamplebucket and its internal objects. The second statement also allows the identity the ability to list all the bucket's permission.

One thing to note here that is identity-based policies cannot grant anonymous access to the bucket as it's always attached to a user.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AssignUserActions",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:ListBucket",
        "s3:DeleteObject",
        "s3:GetBucketLocation"
      ],
      "Resource": [
        "arn:aws:s3:::awsexamplebucket1/",
        "arn:aws:s3:::awsexamplebucket1"
      ]
    },
    {
      "Sid": "ExampleStatement2",
      "Effect": "Allow",
      "Action": "s3:ListAllMyBuckets",
      "Resource": "*"
    }
  ]
}

Likewise, there are a lot of actions, resource types, and condition keys to use within IAM policies, which offers you full flexibility in defining the permissions to your S3 buckets with identity policies.

To view all available actions, resource types, and condition keys on S3, refer https://docs.aws.amazon.com/service-authorization/latest/reference/list_amazons3.html

2 — Resource-based policies

As the name implies these policies provide access controls from the resource’s side, which are associated with either S3 objects or buckets. Due to this its required to also define what principal or in simple terms who will have the allowed/ denied access to that specific resource-based policy. From an S3 perspective, resource-based policies can be divided into two main types,

2.1 — Bucket Policies

Bucket policies can be used to grant IAM users and other AWS accounts permissions to your bucket and objects in it. It's defined at the individual bucket level.

Bucket policies can be added to a bucket when you click the bucket and within the Permissions tab in the AWS management console. You can also do the same programmatically as everything else if you were wondering. Bucket policies are defined in JSON. Given below is a sample bucket policy, which allows anonymous access (As the Principal is set to *) to Get objects from awsexamplebucket1. Going beyond this you can even set conditions into your bucket policies. You can refer to the link I attached above to view the available options.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "GrantAnonymousReadPermissions",
      "Effect": "Allow",
      "Principal": "",
      "Action": [
        "s3:GetObject"
      ],
      "Resource": [
        "arn:aws:s3:::awsexamplebucket1/"
      ]
    }
  ]
}

2.2 — ACLs

ACLs, allow you to control access to buckets and additionally to specific objects within bucket groupings and AWS accounts. ACL is a list containing grants identifying grantees and permissions associated with them. It is defined as Amazon S3–specific XML schema. Here is a sample view of the ACL settings by default to the bucket. You can access this by clicking on the S3 bucket and going to the permissions tab within the AWS management console.

By editing the above configurations, you can also add access to another AWS account as a grantee and set up a bucket and object-specific access. The object ACL only has two permissions as List, and Write from the object ACL, unlike Write, Read permission for the Bucket ACL. Read object allows the grantee to read the object data and its metadata. Read object permissions allows the grantee to read the object ACL. Write object permissions allow the grantee to write the ACL for the object.

As AWS now recommends, for most of the general use cases it's better to go off just with ACLs disabled option as we have discussed in Part 1. But it again may depend on your use case.

In a situation where all three of the controls are used to control access to an S3 bucket, S3 will view all these policies together and any permission conflict between policies will be handled in accordance with the principle of least privilege.

Few points to keep in mind related to occasions when permission conflicts are derived,

**- AWS S3 follows the principle of least privilege and by default, it defines that access is denied to an object even though an explicit denial is not mentioned within any policy.

Allow should exist within a policy that principal is associated or defined within a bucket policy or ALC. Access will be granted only if Allow is defined within a policy and Deny if not defined.
If a single Deny is defined as associated with the principal to a specific object, access will be Denied despite having an Allow in another policy. In simple terms, Deny always takes precedence over Allow when permissions are assessed.**

We have discussed multiple ways of controlling access to your S3 buckets using policies within this article. I hope by now, it's clear that you can use all the available policy types to control access to your AWS S3 bucket together.

Before finalizing, I would also like to highlight some key considerations that you have to keep in mind when planning to use different access control policies.

When to consider using Identity Based policies,

To centrally manage access control methods all within one service.
If you manage a large number of buckets and multiple permissions should be added to all those buckets, Identity-based policies will be ideal and easy to manage.

When to consider using Resource Based policies,

If you need to granularly control the security of your S3 buckets only within the S3 service itself and within the bucket level, bucket policies can be considered ideal.
If your policy file is a bit large, there are size constraints on IAM policies compared to bucket policies (IAM policies can be a maximum of two kilobytes in size for users, five kilobytes for groups, and 10 kilobytes for roles. However, bucket policies can reach a size of 20 kilobytes)

Based on your use case, security, compliance, and regulatory requirements you can leverage all the above access control policies together to achieve the maximum security you can obtain using S3 access control policies.

Let’s deep dive into other additional security controls and methods provided by AWS S3 in the upcoming articles on this article series.

References

1 — https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-control-overview.html

2 — https://docs.aws.amazon.com/AmazonS3/latest/userguide/how-s3-evaluates-access-control.html

3 — https://cloudacademy.com/course/increasing-your-security-posture-when-using-amazon-s3-1235

Understanding inbuilt AWS S3 security controls and methods - Part 1

Ravindu Nirmal Fernando — Sun, 23 Oct 2022 13:19:44 +0000

Originally published on Medium

Security is a key component and concept that you should focus on when it comes to data storage. Any misconfiguration in the security front of your sensitive data can bring a catastrophic impact on your business and its customers.

Since its inception in 2006, AWS S3 has been one of the prominent and go-to storage as a service and backup as a service for many software solutions across the world. Though lots of innovation happening around with AWS S3 service, we still see instances where a simple yet harmful misconfiguration in AWS S3 makes a way for large security breaches (https://www.zdnet.com/article/unsecured-aws-server-exposed-airport-employee-records-3tb-in-data/).

As stated in the Shared Responsibility Model documentation by AWS, Security and Compliance are shared responsibilities between AWS and the customer. AWS will inherently provide you with physical and environmental security for the services you consume. But when it comes to patch management, configuration management, and awareness and training, the responsibility is shared between both the customer and the Cloud Service Provider (CSP). Hence it's the same with AWS S3 service as AWS is responsible for maintaining the durability, availability, and physical security of data that resides in S3, its the customer’s responsibility to enable and configure the AWS S3 service (buckets and objects) with proper security controls and methods to ensure that the data resides on S3 are secure and cannot be accessed from unauthorized parties.

AWS has provided its customers with several security controls and methods to improve the security posture when using the AWS S3 service. This article series will go through all those in detail.

Before we go through the security controls and methods that AWS S3 offers, it’s really important to understand how resource ownership works in AWS S3. That will be the key focal point for part 1 of this article series.

These are a few points that you should grasp when it comes to principles of resource ownership by default.

Buckets and objects are considered resources in AWS S3. By default, when a bucket is created or an object is uploaded to Amazon S3 the ownership of that bucket and the object will go to the AWS account.
You can set up permissions to allow another AWS account to upload resources to a bucket created by your account. In such cases, by default, if that AWS account uploads an object resource to that bucket the ownership of that object resource will be with the AWS account which did the upload and not the AWS account that acts as the bucket owner. But this behaviour can be overridden by setting up the permissions in bucket configuration as shown below.

As you can see in the above figure, there are three settings that can be used to control the ownership of the objects uploaded to the bucket.

ACLs disabled

Bucket owner enforced (recommended) — In this setting ACLs are disabled, and the bucket owner automatically owns and has full control over every object in the bucket. ACLs are no longer considered when assessing permissions to data in an S3 bucket. Bucket policies can be used to define access control. Activating this setting will apply it to all the existing and new objects as well. This setting was introduced in November 2021.

ACLs enabled

Bucket owner preferred — This setting provides the bucket owner full control over new objects that other accounts write to the bucket with the bucket-owner-full-control canned ACL. If you activate this setting, you should ensure that you set up a bucket policy to grant a specific user or a role to upload objects into that bucket using the bucket-owner-full-control canned ACL. Refer to the sample bucket policy given below,

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Only allow writes to my bucket with bucket owner full control",
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::AccountA:role/AccountARole"
        ]
      },
      "Action": [
        "s3:PutObject"
      ],
      "Resource": "arn:aws:s3:::AccountB-Bucket/*",
      "Condition": {
        "StringEquals": {
          "s3:x-amz-acl": "bucket-owner-full-control"
        }
      }
    }
  ]
}

Object writer (default) — The AWS account that uploads an object owns the object, has full control over it and can grant other users access to it through ACLs.

You can read more about these controls by visiting the AWS Documentation.

For the majority of the use cases, AWS recommends disabling ACLs by choosing the bucket owner-enforced setting and using your bucket policy to share data with users outside of your account as needed.

With a solid grasp on the resources and resource ownership in AWS S3, let’s deep dive into understanding security controls and methods provided by AWS S3 in the upcoming articles on this article series.

References

1 — https://docs.aws.amazon.com/AmazonS3/latest/userguide/about-object-ownership.html

2 — https://aws.amazon.com/premiumsupport/knowledge-center/s3-bucket-owner-full-control-acl/

3 — https://aws.amazon.com/compliance/shared-responsibility-model/