Forem: Dustin Whited

Terraforming Resource Control Policies

Dustin Whited — Mon, 18 Nov 2024 19:07:39 +0000

AWS released a new type of organizational authorizational policy called Resource Control Policies (RCPs). This policy is considered a "resource-centric" policy, complementary to Service Control Policies "principal-centric" preventative guardrails.

See the AWS release blog for more info about RCPs: https://aws.amazon.com/blogs/aws/introducing-resource-control-policies-rcps-a-new-authorization-policy/

Terraforming

One of the first questions I ask with a new AWS feature or service is "how do I automate it?" For RCPs, I asked a similar question and found that the AWS provider in Terraform had support as of version 5.76.0.

First, ensure you have the policy type enabled on your organization via "ALL" or "RESOURCE_CONTROL_POLICY" like shown below. This will enable RCPs as well as attach RCPFullAWSAccess.

resource "aws_organizations_organization" "org" {
  feature_set          = "ALL"
  enabled_policy_types = [
    "AISERVICES_OPT_OUT_POLICY",
    "SERVICE_CONTROL_POLICY",
    "RESOURCE_CONTROL_POLICY",
    ]
}

Ensure your provider is up to 5.76. You could also use "~> 5.0".

See Version Constraints documentation for more info on how this works https://developer.hashicorp.com/terraform/language/expressions/version-constraints


terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
      version = "~> 5.76"
    }
  }
}

Now you can create a RCP. I'm using the confused deputy prevention example provided from aws-samples. Note that RCPs have similar limits to SCPs. You can only attach 5 per entity (OU and account), RCPFullAWSAccess counts against the limit, and they have a character limit of 5120 that includes whitespace.

data "aws_iam_policy_document" "rcp1" {
  statement {
    effect = "Deny"

    actions = [
      "s3:*",
      "sqs:*",
      "kms:*",
      "secretsmanager:*",
      "sts:*",
    ]

    resources = ["*"]

    principals {
      type        = "*"
      identifiers = ["*"]
    }

    condition {
      test     = "StringNotEqualsIfExists"
      variable = "aws:SourceOrgID"

      values = [
        aws_organizations_organization.org.id
      ]
    }

    condition {
      test     = "Null"
      variable = "aws:SourceAccount"

      values = [
        "false"
      ]
    }

    condition {
      test     = "Bool"
      variable = "aws:PrincipalIsAWSService"

      values = [
        "true"
      ]
    }
  }
}

resource "aws_organizations_policy" "rcp1" {
  name    = "rcp1"
  content = data.aws_iam_policy_document.rcp1.json
  type    = "RESOURCE_CONTROL_POLICY"
}

If you are combining multiple RCPs into a single aws_iam_policy_document, you may want to use minified_json instead of json on the content argument.

resource "aws_organizations_policy" "rcp1" {
  name    = "rcp1"
  content = data.aws_iam_policy_document.rcp1.minified_json
  type    = "RESOURCE_CONTROL_POLICY"
}

And finally, attach it to an entity. In this case, I am attaching it to an OU I have called "dev" under the organization root.

resource "aws_organizations_organizational_unit" "dev" {
  name      = "dev"
  parent_id = aws_organizations_organization.org.roots[0].id
}

resource "aws_organizations_policy_attachment" "rcp1_dev" {
  policy_id = aws_organizations_policy.rcp1.id
  target_id = aws_organizations_organizational_unit.dev.id
}

It's very nice to see Terraform provider support so quickly for a new release, especially on governance features.

Happy RCP'ing!

Introducing Amazon GuardDuty Detector Checker: Verify Enablement and Finding Publishing Configurations

Dustin Whited — Tue, 22 Oct 2024 17:33:47 +0000

I am excited to release my latest open-source project: Amazon GuardDuty Detector Checker. This Python-based tool addresses the challenge of validating the configuration of Amazon GuardDuty across multiple regions in your AWS environment. It can also aid in gathering information to import GuardDuty resources to Terraform.

The Problem

Manually checking each region for GuardDuty status, detector IDs, and publishing destinations is time-consuming and prone to oversight. Importing this all to Terraform helps wrangle GuardDuty as well as allow organizations to quickly roll out new features, such as the recently released malware protection for S3 https://aws.amazon.com/blogs/aws/introducing-amazon-guardduty-malware-protection-for-amazon-s3/

How Amazon GuardDuty Detector Checker Helps

Our tool simplifies this process by automatically:

Listing all enabled regions in your AWS account
Checking GuardDuty status in each enabled region
Retrieving GuardDuty detector IDs where available
Fetching publishing destinations and IDs for enabled detectors

This overview allows security teams to quickly identify gaps in GuardDuty coverage and ensure proper configuration across all regions.

Using the Tool

The Amazon GuardDuty Detector Checker is easy to use and requires minimal setup:

Install via PyPi with pip
- pip install guardduty-detector-checker
Run the script with an optional AWS profile name
- guardduty-detector-checker [-p PROFILE]

The tool outputs a JSON-formatted list of GuardDuty status across all enabled regions in the AWS account.

[
  {
    "us-east-1": {
      "abc123ABC123abc123ABC123abc123AB": [
        {
          "DestinationId": "abcABCdefDEFghiGHIjklJKLmnoMNO12",
          "DestinationType": "S3",
          "Status": "PUBLISHING"
        }
      ]
    }
  },
  {
    "us-west-2": {}
  }
]

These IDs can then be used in Terraform import blocks:

resource "aws_guardduty_detector" "this" {
  enable                       = true
  finding_publishing_frequency = "FIFTEEN MINUTES"
  datasources {
    kubernetes {
      audit_logs {
        enable = var.enable_kubernetes_audit_logs
      }
    }
    malware_protection {
      scan_ec2_instance_with_findings {
        ebs_volumes {
          enable = var.enable_malware_protection
        }
      }
    }
    s3_logs {
      enable = var.enable_s3_logs
    }
  }
}

import {
    to = aws_guardduty_detector.this
    id = "abc123ABC123abc123ABC123abc123AB"
}

resource "aws_guardduty_publishing_destination" "this" {
  detector_id     = aws_guardduty_detector.this.id
  destination_arn = var.my_bucket
  kms_key_arn     = var.my_kms_key
}

import {
    to = aws_guardduty_publishing_destination.this
    id = "abc123ABC123abc123ABC123abc123AB:abcABCdefDEFghiGHIjklJKLmnoMNO12"
}

The Amazon GuardDuty Detector Checker is available on my Github, where you can find documentation, contribute to the project, or adapt it to your specific needs.

The Importance of Internal Cloud Security Standards

Dustin Whited — Wed, 21 Dec 2022 18:51:04 +0000

Why You Need an Internal Standard

Internal standards contain prescriptive statements that provide engineers with a recipe to create repeatable secure processes. These statements declare how infrastructure or processes should be defined and are a baseline for security in a cloud environment.

Standards can also be written into reusable modules. When owned and maintained centrally, these modules provide developers with preconfigured templates that adhere to security standards. Converting the standard into policy as code allows immediate feedback to developers, linting the code as they are writing it. This allows developers to create business value quicker while reducing the amount of security remediation needed after deployment.

How to Write a Standard

Just like the standards themselves, how it is written will depend upon the needs of the organization. There are a few overall commonalities, however.

A standard should:

Be RFC2119 compliant. This means using words like MUST, and SHOULD to indicate requirements.
Be RFC8174 compliant. An add-on to the previous RFC, it specifies the required keywords’ capitalization.
Be precise when prescribing the requirements.
- “S3 buckets MUST use SSE-S3 or SSE-KMS encryption keys.”
Have different requirements for applications with different risk levels and criticality.
- Requiring the use of CloudHSM for a moderate risk workload is not a realistic view of the threats.
Have a key for each requirement that can be referred to by external systems. This is particularly important for organizations that use multiple cloud service providers (CSPs), as it helps to ensure the appropriate control is applied to the correct environment.
- AWS.IAM.1 - Identity and Access Management (IAM) users MUST NOT exist.
- GCP.GCE.1 - Identity Aware Proxy (IAP) MUST be used for all human remote connections to virtual machines.

Choose an accessible location to host the standard. This could be Confluence, a Google doc, or Github. Writing the standard should be a collaborative effort that includes the business owner, engineering, and security to ensure the requirements are realistic and align with the business’s risk appetite.

One size does not fit all in large complex environments. Many of the standards and benchmarks are written so that they can apply to a variety of environments. Take for example the CIS Benchmark for Amazon Web Services (AWS). There is a control for integrating CloudTrail with CloudWatch logs so that metric filters for alerts can be created. This control is irrelevant in an environment that has a security information and event management (SIEM) or other log aggregation and alerting tool.

How to Implement a Standard

Once an internal cloud security standard has been written, the next step is to implement and enforce its requirements. Requiring 30 new standards at once would create a large burden upon development teams to triage and remediate, pulling attention away from building business features. Enablement should be a planned and methodical process.

Create a Roadmap

Create a roadmap for enabling the controls. If there are any low effort, high impact items, prioritize those first. Often these are related to any publicly available resources to reduce and secure the attack surface. Some controls may have hidden complexity that is revealed by talking to the developer of the system.

Find the Baseline

The first step is to create detective capabilities to monitor compliance with the new control. In AWS Config is frequently used in tandem with Security Hub, but there is a large variety of Cloud Security Posture Management (CSPM) tools on the market that operate across multiple CSPs. This will not only provide a mechanism to detect compliance, but also give a general idea of the remediation work that needs to be done.

Facilitate Developer Enablement

The next step is to provide the development teams with the baselines and a target enforcement date. The standard should include documentation on how to implement the controls, and there should be a mechanism to provide feedback to the development teams as they are working on implementing the standards. This could be in the form of linting the code as it is written, developing preconfigured secure modules, or providing training for the engineering teams.

Implement Standards as Code

During the baseline phase, detective capabilities were created to detect compliance to controls. Because we do not want development teams to be in a constant state of remediation, for every detective capability, there should also be a preventative capability.

This preventative capability can be written as code, using policy as code tools such as Snyk, Terrascan, and OPA. Once the rules are written, they can be added to git repos as a pre-merge check. This creates a consistently applied process that is easy to maintain and audit.

Closing

Internal cloud security standards are essential for ensuring secure processes and reducing the amount of security remediation needed after deployment. Standards should be written in a precise and RFC-compliant manner and should be tailored to the risk appetite of the organization. Implementation should be done in a methodical way, with detective and preventative capabilities written as code and shared with development teams.

Threat Detection on EKS – Comparing Falco and GuardDuty For EKS Protection

Dustin Whited — Fri, 06 May 2022 19:40:01 +0000

Co-authored by Dustin Whited and Dakota Riley

Kubernetes adoption has grown, with managed services leading implementation according to a recent Cloud Native Computing Foundation (CNCF) survey. "79% of respondents use Certified Kubernetes Hosted platforms" with Amazon’s managed Kubernetes offering, Elastic Kubernetes Service (EKS), being the most popular of the hosted platforms.

The shared responsibility model means the customer is responsible for securing the workloads running on EKS. In January, 2022, AWS released new functionality for GuardDuty enabling threat detection on EKS clusters and aiding customers with securing their workloads. This new capability uses the Kubernetes Audit log to monitor for suspicious or malicious activity.

Falco, a CNCF project, is a common open source tool used to perform similar threat detection capabilities within Kubernetes clusters. Falco monitors system calls from the Linux kernel for the majority of its analysis. It is also preloaded with community maintained rule sets.

This blog will explore some commonalities and differences between Falco and the new GuardDuty feature.

Ease of implementation

GuardDuty Kubernetes protection is not enabled by default. Accounts that are in an AWS Organization can enable the feature through the GuardDuty delegated administrator account, or the detector can be updated on an individual account. Terraform support is pending at the time of writing, while Cloudformation and CDK support is fully implemented.

It is also worth noting that enabling EKS GuardDuty incurs additional pricing per one million audit events ingested. The cost is reasonable, but environments with many clusters or a high amount of activity may incur a surprise billing statement after the 30 day trial ends.

Falco can be installed with a chart as a daemonset in the cluster, or directly installed on the host operating system. Installing as a daemonset is the simplest way to get started with Falco, however installing on the OS insulates it from Kubernetes itself. Organizations that create hardened custom node AMIs may find it easiest to include Falco through the image build process.

Finding types

At a high level, EKS GuardDuty is ultimately more focused on detecting malicious activity against the EKS Control Plane, whereas by default Falco is better suited to detecting malicious activity on a container running on the cluster, or the host itself.

While there is some overlap in rules, GuardDuty and Falco detections are not congruent. At the time of writing, GuardDuty has 27 Kubernetes findings that leverage the Kubernetes Audit log. Importantly, regular GuardDuty findings for EC2 will also apply to the Kubernetes nodes. With over 68 community rules, Falco monitors syscalls from the Linux kernel by default and optionally can consume the Kubernetes Audit log - this requires configuring additional logic with a CloudWatch subscription filter on the EKS Audit Logs CloudWatch to feed it back into Falco.

Below we compare and contrast some notable detection capabilities of the two tools.

Anonymous Access:

EKS can be configured to allow anonymous access to its control plane (relevant GitHub Issue on AWS Containers Roadmap here). This configuration should almost never exist. This is an excellent "low-hanging fruit" detection provided by EKS GuardDuty upon that access being configured:

Policy:Kubernetes/AnonymousAccessGranted

It is also worth noting that EKS GuardDuty also includes findings for actual usage of that anonymous access - which can help inform your response, the findings themselves are categorized based on the specific APIs (which are not publicized by AWS) being called anonymously:

CredentialAccess:Kubernetes/SuccessfulAnonymousAccess

DefenseEvasion:Kubernetes/SuccessfulAnonymousAccess

Discovery:Kubernetes/SuccessfulAnonymousAccess

Impact:Kubernetes/SuccessfulAnonymousAccess

Persistence:Kubernetes/SuccessfulAnonymousAccess

Falco also has an anonymous access rule in its K8s Audit Logs ruleset, but requires the additional logic to feed into the engine:

Falco anonymous access rule

Syscalls and Malicious Runtime Activity:

Where Falco shines is its ability to ingest syscalls from containers on the cluster to detect malicious activity on the container itself. EKS GuardDuty currently only alerts on activities against the K8s Control Plane of the cluster. Highlighting a few default Falco rules below:

Contact EC2 Instance Metadata Service From Container - Specific to AWS - this finding indicates when a container makes a call to the IMDS endpoint, which would by default provide access to credentials for the EKS Node Instance Role. In most cases, this would be an indicator of lateral movement/recon attempts on a compromised container (see also: IAM Roles for Service Accounts and Restrict Access to the Node Instance Profile).

Contact K8s API Server From Container - This finding can be an indicator of lateral movement/recon attempts on a compromised container. Allowlisting legitimate access by management tools may be required in order to be a high fidelity alert.

Known Malicious IP:

GuardDuty supports Malicious IP Caller detection which will alert on malicious IP addresses invoking Kubernetes API calls. Similar to the Anonymous Access findings, EKS GuardDuty has a number of Malicious IP Caller finding types that will trigger based on the Kubernetes API being called. For example, Discovery:Kubernetes/MaliciousIPCaller and Impact:Kubernetes/MaliciousIPCaller both trigger on malicious IPs, but are dependent on the specific API being called.

Falco requires a static deny list of malicious IPs or domain names, which is not optimal to maintain up to date threat intelligence.

Cryptomining:

While not part of EKS GuardDuty specifically, the EC2 finding CryptoCurrency:EC2/BitcoinTool.B!DNS will alert on resources within the cluster contacting Crypto Mining domains. GuardDuty is automatically updated with the latest threat intel feeds from Crowdstrike and ProofPoint so it will always have up to date information.

Falco has a rule to detect crypto miner domains but it is not enabled because Falco will invoke a DNS lookup to determine if it is in the list, which may trigger alerts external to Falco. This list is also not a true threat intel feed. One advantage to using Falco for this use case is that it will provide helpful metadata about the container in question making the calls to cryptoming domains, where GuardDuty will only provide the EC2 Instance and associated IP addresses and require additional enrichment to triage further.

Customizability

Customization can be a game changer for threat detection tooling, especially when it comes to fine tuning the fidelity of the findings. On the other hand, not all teams have the bandwidth to tune findings or implement custom detections. A look below at the customization ability of both tools:

GuardDuty findings cannot be customized. Some post-processing of the finding is possible with external integrations like Lambda – for example to reduce the severity level of the finding. While findings are not able to be customized, GuardDuty supports importing both custom threat lists or trusted IP lists, as well as native suppression rule capabilities.

Existing Falco rules can be disabled or modified. New supplemental rules can be written to detect environment specific use cases. Falco is not just limited to Kubernetes syscalls or audit logs - recently Falco released a plugin capability, allowing Falco to be extended to consume additional event sources. One such example of this is the AWS CloudTrail Plugin.

Integrations

Alerts need to have an action or notification method to bring value to a security organization. Integrating GuardDuty and Falco with external systems to notify a responder or take action is achieved in a couple different ways.

GuardDuty findings are sent to AWS services like SecurityHub and the default Eventbridge eventbus within the administrator account. Both allow consumption into a SIEM of choice, direct alerting with Chatbot, or even taking action with a Step Function workflow. For an example of auto-remediation using a Step Function workflow see our previous blog on this topic.

Falco can emit its findings to stdout, a file, syslog, or custom endpoints with "Program Output" using bash. Alternatively, Falcosidekick is an extra app that receives Falco alerts from multiple clusters and forwards them to a variety of outputs concurrently. Falcosidekick has a large number of custom destinations, so it may reduce the need to write custom integrations for existing destinations like a SIEM.

Overall, both GuardDuty and Falco will require some level of investment into integrating the alerts with existing systems. The only exception is if SecurityHub is the primary location as GuardDuty sends findings there by default.

Wrap up

GuardDuty EKS protection offers very quick centralized enablement and is managed by AWS for updates to findings. Enabling this functionality is a new best practice and should be enabled organization-wide as it only costs money when an EKS cluster is present and has activity.

Falco offers an increased level of customization and granularity that GuardDuty does not. Falco should be enabled in organizations with the maturity to tune and monitor the alerts, as well as create defined runbooks for responders.

In the end, it is not usually an either-or situation for GuardDuty EKS and Falco, but rather each is a milestone on a maturity roadmap. The two tools complement each other to supplement a defense in depth strategy securing container workloads on AWS.

The information presented was current as of date of publication 05/06/2022

Auto Remediation with Eventbridge, Step Functions, and the AWS SDK Integration

Dustin Whited — Fri, 01 Apr 2022 18:55:55 +0000

In late 2021, AWS released a new feature for Step Functions (SFN) allowing AWS SDK calls directly from the workflow. Previously, a Lambda function or SSM document would be invoked by the SFN State Machine to make AWS SDK calls. This new feature allows low to no code workflows within AWS.

This is important because unmaintained code may introduce vulnerabilities into the environment and create tech debt. Ultimately, the responsibility for keeping code up to date in a cloud environment lies squarely with the customer. This new Step Functions SDK integration reduces the operational burden on security teams by transferring that responsibility to the Cloud Service Provider. This blog will explore an example of functionless auto remediation of a public resource exposure using the SFN SDK integration and Terraform.

Use Case: EBS Snapshot Made Public

All write events that are recorded by CloudTrail are also sent to the Default EventBridge Event bus. Read-only events are not currently sent to the default event bus and cannot trigger a workflow. Modifying a snapshot results in a write CloudTrail event, making this remediation workflow possible.

Modifying a snapshot to be public results in the following sample CloudTrail event:

{
  "input": {
    "version": "0",
    "id": "00000000-0000-0000-0000-000000000000",
    "detail-type": "AWS API Call via CloudTrail",
    "source": "aws.ec2",
    "account": "123456789012",
    "time": "2022-01-01T00:00:00Z",
    "region": "us-east-1",
    "resources": [],
    "detail": {
      "awsRegion": "us-east-1",
      "eventID": "1111",
      "eventName": "ModifySnapshotAttribute",
      "eventSource": "ec2.amazonaws.com",
      "eventTime": "2022-01-01T00:00:00Z",
      "eventType": "AwsApiCall",
      "eventVersion": "1.05",
      "recipientAccountId": "123456789012",
      "requestID": "1111",
      "requestParameters": {
          "attributeType": "CREATE_VOLUME_PERMISSION",
          "createVolumePermission": {
              "add": {
                  "items": [
                      {
                          "group": "all"
                      }
                  ]
              }
          },
          "snapshotId": "snap-1111"
      },
      "responseElements": {
          "_return": true,
          "requestId": "1111"
      },
      "sourceIPAddress": "111.111.111.111",
      "userAgent": "Mozilla/2.0 (compatible; NEWT ActiveX; Win32)",
      "userIdentity": {
          "accessKeyId": "1111",
          "accountId": "123456789012",
          "arn": "arn:aws:sts::123456789012:assumed-role/example-role/example-user",
          "principalId": "1111",
          "sessionContext": {
              "attributes": {
                  "creationDate": "2022-01-01T00:00:00Z",
                  "mfaAuthenticated": "true"
              },
              "sessionIssuer": {
                  "accountId": "123456789012",
                  "arn": "arn:aws:iam::123456789012:role/example-role",
                  "principalId": "1111",
                  "type": "Role",
                  "userName": "example-role"
              },
              "webIdFederationData": {}
          },
          "type": "AssumedRole"
        }
      }
  }
}

This sample event makes it a little easier to write an Event Pattern, but a pattern could also be created from the information in the API specification to match fields and values.

Sharing an EBS volume snapshot publicly modifies an existing snapshot with the ModifySnapshotAttribute API call. The “group”: “all” key-value pair is added through CreateVolumePermission during the API call.

This solution will monitor for invocations of this call and, if it meets the Event Rule criteria “group”: “all”, forward the event to a State Machine which will call the ModifySnapshotAttribute API to make the snapshot private again.

Writing an Event Pattern to Match

This example will use Terraform to manage the Infrastructure as Code (IAC). Using Terraform will enable the ability to scale and deploy this solution in a predictable and repeatable state across multiple regions and AWS accounts.

Event Patterns have a few fields that are always present. Writing the CloudWatch Event Rule event_pattern will use keys from from the above CloudTrail event. The source will be the service the event originates from, aws.ec2, and detail-type, which is AWS API Call via CloudTrail.

In Terraform, the event rule and pattern is created as so:

resource "aws_cloudwatch_event_rule" "public_snapshot" {
  name        = "public-snapshot"
  description = "Capture events when Snapshots are made public"

  event_pattern = <<EOF
{
  "source": ["aws.ec2"],
  "detail-type": ["AWS API Call via CloudTrail"],
  "detail": {
    "eventSource": ["ec2.amazonaws.com"],
    "eventName": [
        "ModifySnapshotAttribute"
    ],
    "requestParameters": {
      "createVolumePermission": {
        "add": {
          "items":
            {
            "group": ["all"]
            }
        }
      }
    }
  }
}
EOF
}

Contained within the “detail” key is the CloudTrail log like shown above. Fields eventSource, and eventName help filter to the ec2:ModifySnapshotAttribute permission.

The requestParameters section does not have to explicitly match everything contained in the log, and in this case, it is looking to match “group”: “all”. When writing event patterns, the fields specified must match the event to trigger the rule.

Multiple values can be specified within square brackets ( “[ ]” ), comma delimited, and the pattern will match on an “OR” basis to any of the values within.

See Create Event Patterns in the AWS Documentation website for more information on the logic used to match events.

Triggering a State Machine

Step Function workflows can be invoked directly from Eventbridge event rules. The full event object is sent as the payload.

The event rule will require an IAM role to invoke the state machine. It requires a trust policy or “assume role policy” allowing events.amazonaws.com to assume the role and the action states:StartExecution on the yet to be created state machine. The state machine will be created later, but is easily referenced here.

resource "aws_iam_role" "event_public_snapshot" {
  name = "public-snapshot-events-role"

  inline_policy {
    name   = "public-snapshot-events-policy"
    policy = data.aws_iam_policy_document.events_policy.json

  }

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "events.amazonaws.com"
        }
      },
    ]
  })
}

data "aws_iam_policy_document" "events_policy" {
  statement {
    effect  = "Allow"
    actions = ["states:StartExecution"]
    resources = [aws_sfn_state_machine.public_snapshot.arn]
  }
}

An event target in Terraform requires only 4 things: the rule referenced, a name for the target, the IAM role created above, as well as the state machine ARN reference.

resource "aws_cloudwatch_event_target" "sfn" {
  rule      = aws_cloudwatch_event_rule.public_snapshot.name
  target_id = "public-snapshot-to-sfn"
  arn       = aws_sfn_state_machine.public_snapshot.arn
  role_arn  = aws_iam_role.event_public_snapshot.arn
}

Remediating the Public Snapshot

Like the event target, the state machine will need an IAM role with the appropriate permissions to make the snapshot private.

The state machines IAM role is similar to that of Events, this one with a states.amazonaws.com trust policy and the ec2:ModifySnapshotAttribute action. Resource requires a wildcard here as snapshot ARNs can be dynamically generated and the automation should encompass any snapshot created in the account.

resource "aws_iam_role" "sfn_public_snapshot" {
  name = "public-snapshot-sfn-role"

  inline_policy {
    name   = "public-snapshot-sfn-policy"
    policy = data.aws_iam_policy_document.sfn_policy.json

  }

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "states.amazonaws.com"
        }
      },
    ]
  })
}

data "aws_iam_policy_document" "sfn_policy" {
  statement {
    effect    = "Allow"
    actions   = ["ec2:ModifySnapshotAttribute"]
    resources = ["*"]
  }
}

The state machine itself is fairly simple. After it begins, it will run a single stage, which removes the public permission, and then exits.

Creating the definition of a state machine has a learning curve. Step Functions has a visual workflow designer in the AWS console that can help reduce some of the learning curve.

There are some more complicated things to tie tasks together, transforming outputs into inputs, and forking the logic, but none of these are necessary for this example. Another helpful resource for in-depth information on Amazon States Language is hosted at https://states-language.net/

resource "aws_sfn_state_machine" "public_snapshot" {
  name     = "public-snapshot"
  role_arn = aws_iam_role.sfn_public_snapshot.arn

  definition = <<EOF
{
  "Comment": "Removes group:all from snapshots",
  "StartAt": "RemoveAllPermission",
  "States": {
    "RemoveAllPermission": {
      "Type": "Task",
      "Resource": "arn:aws:states:::aws-sdk:ec2:modifySnapshotAttribute",
      "Parameters": {
        "SnapshotId.$": "$.detail.requestParameters.snapshotId",
        "CreateVolumePermission": {
            "Remove": [ { "Group": "all" } ]
        }
      },
      "End": true
    }
  }
}
EOF
}

Walking through the state machine definition, it begins with a comment about what the state machine’s purpose is.

StartAt refers to the starting point of the state machine and in this case, is the RemoveAllPermission task.

RemoveAllPermission uses an AWS SDK resource in the form of arn:aws:states:::aws-sdk:${SERVICE}:${API} . All of the currently available service integrations are listed in the AWS documentation.

The keys in parameters match those required by the API call, and the SnapshotId is retrieved from the CloudTrail Event in JSON format.

Deploying

The full Terraform code is hosted in a Github snippet and includes all code above. Paired with an AWS provider block, this example can be deployed for testing.

https://gist.github.com/dgwhited/ce2f3570f5f7e79b2477456a62b2db38

###########################
#### Eventbridge event ####
###########################

resource "aws_cloudwatch_event_rule" "public_snapshot" {
  name        = "public-snapshot"
  description = "Capture events when Snapshots are made public"

  event_pattern = <<EOF
{
  "source": ["aws.ec2"],
  "detail-type": ["AWS API Call via CloudTrail"],
  "detail": {
    "eventSource": ["ec2.amazonaws.com"],
    "eventName": [
        "ModifySnapshotAttribute"
    ],
    "requestParameters": {
      "createVolumePermission": {
        "add": {
          "items":
          {
            "group": ["all"]
          }
        }
      }
    }
  }
}
EOF
}

resource "aws_cloudwatch_event_target" "sfn" {
  rule      = aws_cloudwatch_event_rule.public_snapshot.name
  target_id = "public-snapshot-to-sfn"
  arn       = aws_sfn_state_machine.public_snapshot.arn
  role_arn  = aws_iam_role.event_public_snapshot.arn
}

###########################
##### Eventbridge IAM #####
###########################

resource "aws_iam_role" "event_public_snapshot" {
  name = "public-snapshot-events-role"

  inline_policy {
    name   = "public-snapshot-events-policy"
    policy = data.aws_iam_policy_document.events_policy.json

  }

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "events.amazonaws.com"
        }
      },
    ]
  })
}

data "aws_iam_policy_document" "events_policy" {
  statement {
    effect    = "Allow"
    actions   = ["states:StartExecution"]
    resources = [aws_sfn_state_machine.public_snapshot.arn]
  }
}

###########################
###### State Machine ######
###########################

resource "aws_sfn_state_machine" "public_snapshot" {
  name     = "public-snapshot"
  role_arn = aws_iam_role.sfn_public_snapshot.arn

  definition = <<EOF
{
  "Comment": "Removes group:all from snapshots",
  "StartAt": "RemoveAllPermission",
  "States": {
    "RemoveAllPermission": {
      "Type": "Task",
      "Resource": "arn:aws:states:::aws-sdk:ec2:modifySnapshotAttribute",
      "Parameters": {
        "SnapshotId.$": "$.detail.requestParameters.snapshotId",
        "CreateVolumePermission": {
            "Remove": [ { "Group": "all" } ]
        }
      },
      "End": true
    }
  }
}
EOF
}

resource "aws_iam_role" "sfn_public_snapshot" {
  name = "public-snapshot-sfn-role"

  inline_policy {
    name   = "public-snapshot-sfn-policy"
    policy = data.aws_iam_policy_document.sfn_policy.json

  }

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "states.amazonaws.com"
        }
      },
    ]
  })
}

data "aws_iam_policy_document" "sfn_policy" {
  statement {
    effect    = "Allow"
    actions   = ["ec2:ModifySnapshotAttribute"]
    resources = ["*"]
  }
}

Other Uses

This example showed a single task that took action and ended upon receiving a qualified event. Other helpful extensions could be notifying a slack channel or user or creating tickets that action was taken. The Step Functions AWS SDK integration is a very powerful tool that can be used to create reactive controls that maintain a secure baseline in the event a preventative control should fail.

The information presented in this article is accurate as of March 02, 2022.