Forem: vikasbanage

AWS Resource Control Policies (RCPs) Explained: A Practical Guide to Resource-Level Security

vikasbanage — Wed, 14 Jan 2026 08:54:12 +0000

Modern AWS environments are built for scale — multiple accounts, shared teams, and resources that need to operate across boundaries. While this flexibility enables agility, it also introduces a critical governance challenge: even when IAM policies and SCPs are correctly configured, resource policies can still be misconfigured, resulting in overly broad access or unintended exposure.

That gap is exactly what Resource Control Policies (RCPs) were created to address.

RCPs let you enforce non-negotiable rules directly on resources, such as:

This resource can never be shared outside the organization
Only approved identities can assume roles here
Even admins can’t break these rules

They don’t replace IAM or SCPs — they fill the last missing governance layer.

Layer	What it controls
IAM	Which principals can perform which actions on which resources, within organizational guardrails.
SCP	The maximum permissions IAM principals in an account or OU can ever have; they restrict but do not grant access.
RCP	The maximum permissions that can ever apply to specific resources in accounts or OUs, regardless of which principal calls them.

Let's now jump to hands-on for RCP, how it works.

Note : It is highly advisable to test RCP on dev/test account before applying. Don't apply directly at Root level.

Hands-On

In this demo, we will be covering two scenarios:

Enforce HTTPS-only across all S3 buckets
Prevent external accounts from decrypting KMS keys. or Disable cross-account use of KMS keys.

Scenario #1 : Enforce HTTPS-only across all S3 buckets

I have created S3 bucket and intentionally set over permissive access as follows. I have uploaded simple file which has text RCP HTTPS test


    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "S3rcppolicy",
            "Effect": "Allow",
            "Principal": "*",
            "Action": [
                "s3:GetObject",
                "s3:GetObjectVersion",
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::XXXX-XXXXX-demo-XXXXXXX/*"
        }
    ]
}

Now if you access files using HTTP and HTTPs, both will work :

HTTP Test:

HTTPS test

Now, let's apply RCP at account level to make sure only HTTPs access is allowed

After attaching RCP, you can see below HTTP requests gets AccessDenied (highlighted part) but HTTPS works.

As you have also observed that, once we apply RCP its applicable to current and new resource policies as well. So while applying RCP make sure that which resources are intentionally shared across org.

Scenario #2 : Prevent external accounts from decrypting KMS keys.

This example blocks the decryption of AWS KMS (Key Management Service) keys for any principal except whitelisted principal in policy.

Part of this demo I have create KSM key and given access for decrypt to other account in KMS policy.

{
      "Sid": "Enable IAM User Permissions",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::714XXXXXXXXX:root"
      },
      "Action": "kms:*",
      "Resource": "*"
    },
    {
      "Sid": "Allow use of the key",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::599XXXXXXXXX:root"
      },
      "Action": [
        "kms:Encrypt",
        "kms:Decrypt",
        "kms:ReEncrypt*",
        "kms:GenerateDataKey*",
        "kms:DescribeKey"
      ],
      "Resource": "*"
    },

I'm encrypting one file using KMS key in AccountA(714XXXXXXXXX) and will try to decrypt it using assuming IAM role from AccountA(714XXXXXXXXX) and AccountB(599XXXXXXXXX).

Without RCP applied, we able to decrypt it.

AccountA(714XXXXXXXXX)

AccountB(599XXXXXXXXX)

Next, we apply the RCP directly to Account A (714XXXXXXXXX). With this policy in place, access is restricted so that only users or roles from Account A can perform the allowed actions, regardless of how individual resource or IAM policies are configured.

{
  "Effect": "Deny",
  "Principal": "*",
  "Action": [
    "kms:Decrypt",
    "kms:Encrypt",
    "kms:GenerateDataKey"
  ],
  "Resource": "*",
  "Condition": {
    "StringNotEquals": {
      "aws:PrincipalAccount": [
        "714XXXXXXXXX"
      ]
    }
  }
}

If I perform same operation for decryption, I get error for AccountB(599XXXXXXXXX) with explicit Deny in RCP.

Before implementing Resource Control Policies - RCPs

RCPs are useful feature from AWS to design data perimeter in AWS, it protect data, block breaches, and enforce real cloud perimeters — but if deployed carelessly, they can also break production workloads. We should must create and test RCPs before applying to critical workloads.

Before you deploy any RCP, make sure it passes through a formal test cycle:

What to test

Use AWS IAM Access Analyzer to understand:

Which resources are currently public
Which are shared externally
Which identities depend on cross-account access

This helps you avoid breaking legitimate access paths.

Then validate policy logic using:

IAM Policy Simulator
Controlled sandbox accounts
Dev OUs before prod

Also test interactions between SCPs and RCPs.

Remember: a deny from either one will block access.

Other High-Impact RCP Use Cases to Explore

AWS maintains a fantastic repository of real-world RCP patterns.

🔐 Enforce Org-Only STS Access
🔑 Lock Down OIDC Providers (GitHub Actions, etc.)
🗄️ Block External Sharing of Data Stores

When used thoughtfully, RCPs close the last remaining gaps around data access, cross-account trust, and misconfigurations.

Good security isn’t about trusting people — it’s about designing systems that remain safe even when mistakes happen.

Thanks for reading — hope this helps you build safer, more resilient AWS environments.

CloudWatch Investigations: Your AI-Powered Troubleshooting Sidekick

vikasbanage — Sun, 04 Jan 2026 20:04:30 +0000

Remember those 3 AM incidents when you’re frantically switching between dashboards, digging through logs, and wondering if you should just restart everything? We all have been through the situation where we worked in non-business hours, weekends, midnights to troubleshoot production issues and its quite energy draining task. What if in this GenAI world we get AI assistant that works 24*7 and guide us through the chaos.

Enter CloudWatch Investigations – a generative AI-powered feature that’s changing how we handle incidents in AWS environments.

When something breaks, instead of you jumping between CloudWatch metrics, logs, deployment history, CloudTrail, X-Ray, and health dashboards, CloudWatch Investigations does the first round of detective work for you.

It uses generative AI to scan your system’s telemetry and quickly surface:

the metrics that look suspicious
the logs that matter
recent deployments or config changes
and even possible root-cause hypotheses, especially when multiple resources are involved

All of this is presented visually, so you can see how things are connected instead of guessing.

It's like having an extra team member who's been staring at your system architecture 24/7 .

Let's get started to check how to configure CloudWatch investigation.

Getting Started

In AWS Console, Go to CloudWatch → AI Operation (left pane), if you are configuring account for first time, you need to do setup
- Creating Investigation Group
  - Configure Retention days: For how much time you would like keep investigation. Please note that The retention cannot be changed once it is configured.
  - Customise Encryption: You can have customer managed key for encryption. But make sure you give permission to access key.
  - Next, it will create new role with required permission for investigation. These will be read-only permission which needed for investigation. You can also create new role here. By default, it uses : AIOpsAssistantPolicy, AmazonRDSPerformanceInsightsFullAccess and AIOpsAssistantIncidentReportPolicy**

Once the investigation group is created, you will be able to see Optional Enhanced configuration

Under Enhanced integration, you will be able to include tags related to application. This will help Cloudwatch to narrow down investigation. This is quite useful setting as its efficient to narrow down investigation.
Access to CloudTrail event, to help CloudWatch investigations better discover relevant change events.
Optionally, X-ray, Application Signals and EKS access entries.

DEMO

Now, we have configuration ready, let's start with demo. Part of this blog, I have simple Event booking app as follows.

User books appointment by providing details and selecting available slots, admin approves/rejects requests.

Disruption in Application

Part of this demo I have modified Lambda role permission where I have removed KMS permission.

Now imagine scenario, suddenly users started reporting errors they are not able see the slots, it throws an errors. Also Admin not able to see any appointments at all.

As we know application design, entry point for application is CloudFront, we start checking Cloudfront and see there is increase in 5xx errors.

Starting Investigation

Under CloudWatch metrics 5xx, you can start already start investigation why its throwing 5xx errors.

It will pick the timestamp automatically or you can adjust from what time you would like to start investigation.

Once investigation is started, it will take 10-15 minutes to finish investigation, we can also view progress of investigation. But instead, we can always start communicating user/business or start other parallel activities.
On completing investigation, it correctly pointed out what went wrong and why we are getting 5xx errors 🥳🥳🥳

As we can use under Root Cause Summary, it was IAM configuration issue which was causing issue.

ANALYSIS: This failure pattern represents an IAM configuration issue rather than a service degradation, as evidenced by the specific KMS permission errors and the NEW occurrence pattern indicating a recent permission change affecting the eventap staging service components.

We have root cause in 15 minutes. This is huge advantage for anyone who works on production system and need to keep system running.

Going one step further, instead of checking metrics every time and start investigating. We can have CloudWatch Alarm in place, as soon as resources metrics gets ALARM start, CloudWatch Investigation will get automatically started and I did same.

I hope this blog gives you a good idea of how you can get started with CloudWatch Investigations. There may be moments where you don’t fully agree with the AI’s suggestions — and that’s perfectly fine. You’re always in control. You can accept what makes sense, discard what doesn’t, and guide the investigation in the right direction.

The beauty is that you can start small with zero setup, and then gradually level up by adding richer telemetry, cross-account visibility, and automation runbooks. Over time, this leads to fewer guesswork-driven fixes, faster MTTR, and much calmer incident calls — even at 3 AM.

Instead of panic-driven troubleshooting and endless tab-hopping across metrics, logs, and dashboards, you get context first: what changed, what’s related, and what’s most likely broken.

Thanks for reading, and happy troubleshooting 🚀

How to Use Amazon SNS Data Protection Policies to Prevent Sensitive Data Leakage

vikasbanage — Sat, 29 Nov 2025 19:05:06 +0000

When we build things using event-driven architecture, we almost always run into Amazon SNS and for good reason. It’s simple, scalable, and makes it incredibly easy to fan out messages to multiple subscribers.

Imagine a fintech or healthcare application that sends transaction alerts or patient updates via SMS or email. These messages may accidentally include sensitive information such as account details, names, or dates of birth. Encrypting SNS topics and applying strict access controls helps ensure compliance, but it’s equally important to prevent sensitive data from leaking into messages themselves.
That’s where ** SNS Message Data Protection** comes in — because your architecture isn’t just about fast delivery, it’s also about secure delivery.

In this blog, I will walk through how we can protect personal and sensitive information while sending notifications through SNS using Data Protection Policies.

Data Protection in SNS

Amazon SNS uses data protection policies to identify and manage sensitive data (like PII and PHI) in message payloads with Predefined or custom data identifiers by using Machine learning and pattern matching.

Each policy allows you to define operations based on detection:

Audit – Log findings without interrupting delivery
De-identify – Mask or redact sensitive data
Deny – Block messages containing sensitiv e data

A policy is defined in JSON format and includes elements like:

DataDirection (Inbound/Outbound)
Principal (IAM identity publishing/subscribing)
DataIdentifier (e.g., name, phone number)
Operation (Audit, De-identify, Deny)

Only one data protection policy per SNS topic is allowed, but it can have multiple statements. This helps organizations enforce privacy controls and reduce compliance risks.

Why should I use message data protection?

Introducing SNS Data Protection into your governance, risk, and compliance programs helps you automatically detect, prevent, and control data leakage. It safeguards regulated data (PII/PHI) and reduces the overhead of building your own detection or masking pipeline.

Defining SNS topics with policy

We define three SNS topics — each configured with its own data protection policy: Audit, De-identify, and Deny. These sample policies demonstrate how SNS handles sensitive data under different rules.

Below are the three sample data protection policies used in this blog—Deny to block sensitive data, Audit to log sensitive content without stopping delivery, and De-identify to automatically mask regulated fields before the message is published.

Audit - Data protection policy

This policy detects email, date of birth, and credit card numbers.
If found, the finding is logged in CloudWatch but delivery continues.

One important thing to note, CloudWatch log group name in this case must have prefix /aws/vendedlogs/

{
  "Description": "Audit sensitive data without blocking delivery",
  "Version": "2021-06-01",
  "Statement": [
    {
      "DataDirection": "Inbound",
      "DataIdentifier": [
        "arn:aws:dataprotection::aws:data-identifier/EmailAddress",
        "arn:aws:dataprotection::aws:data-identifier/DateOfBirth",
        "arn:aws:dataprotection::aws:data-identifier/CreditCardNumber"
      ],
      "Operation": {
        "Audit": {
          "FindingsDestination": {
            "CloudWatchLogs": {
              "LogGroup": "/aws/vendedlogs/sns-audit/"
            }
          },
          "SampleRate": "99"
        }
      },
      "Principal": [
        "*"
      ],
      "Sid": "AuditSensitiveData"
    }
  ],
  "Name": "sns-audit-policy"
}

De-Identify : Data protection policy

This policy masks sensitive fields using # characters. Subscribers never see actual sensitive data.

{
  "Description": "Mask or redact sensitive data",
  "Version": "2021-06-01",
  "Statement": [
    {
      "DataDirection": "Inbound",
      "DataIdentifier": [
        "arn:aws:dataprotection::aws:data-identifier/EmailAddress",
        "arn:aws:dataprotection::aws:data-identifier/DateOfBirth",
        "arn:aws:dataprotection::aws:data-identifier/CreditCardNumber"
      ],
      "Operation": {
        "Deidentify": {
          "MaskConfig": {
            "MaskWithCharacter": "#"
          }
        }
      },
      "Principal": [
        "*"
      ],
      "Sid": "DeidentifySensitiveData"
    }
  ],
  "Name": "sns-deidentify-policy"
}

Deny : Data protection policy

This policy blocks the publish request entirely if sensitive data is detected.

{
  "Description": "Block messages containing sensitive data",
  "Version": "2021-06-01",
  "Statement": [
    {
      "DataDirection": "Inbound",
      "DataIdentifier": [
        "arn:aws:dataprotection::aws:data-identifier/EmailAddress",
        "arn:aws:dataprotection::aws:data-identifier/DateOfBirth",
        "arn:aws:dataprotection::aws:data-identifier/CreditCardNumber"
      ],
      "Operation": {
        "Deny": {}
      },
      "Principal": [
        "*"
      ],
      "Sid": "DenySensitiveData"
    }
  ],
  "Name": "sns-deny-policy"
}

Demo

I have created simple lambda function to test these topics:

import boto3
import os
import json
import logging

sns = boto3.client('sns')
logger = logging.getLogger()
logger.setLevel(logging.INFO)

def lambda_handler(event, context):
    message = {
        "patientId": "PAT123456",
        "name": "John Doe",
        "dob": "12-01-2012",
        "diagnosis": "Flu"
    }

    topics = ["AUDIT_TOPIC_ARN", "DEIDENTIFY_TOPIC_ARN", "DENY_TOPIC_ARN"]
    results = {}

    for topic_env in topics:
        topic_arn = os.environ.get(topic_env)

        try:
            response = sns.publish(
                TopicArn=topic_arn,
                Message=json.dumps(message)
            )
            results[topic_env] = {
                "status": "success",
                "messageId": response.get("MessageId")
            }
            logger.info(f"Published to {topic_env}: {response.get('MessageId')}")

        except sns.exceptions.InvalidParameterException as e:
            # Common case: DENY_TOPIC_ARN rejects sensitive fields
            logger.error(f"[{topic_env}] Sensitive data detected or invalid parameter: {str(e)}")
            results[topic_env] = {"status": "failed", "error": "Sensitive data not allowed"}

        except sns.exceptions.AuthorizationErrorException as e:
            logger.error(f"[{topic_env}] Access denied: {str(e)}")
            results[topic_env] = {"status": "failed", "error": "Access denied"}

        except Exception as e:
            # Catch any unexpected exception
            logger.error(f"[{topic_env}] Unexpected error: {str(e)}")
            results[topic_env] = {"status": "failed", "error": str(e)}

    return {
        "status": "completed",
        "results": results
    }

I'm sending below JSON where I have date of birth as personal information.

  {
        "patientId": "PAT123456",
        "name": "John Doe",
        "dob": "12-01-2012",
        "diagnosis": "Flu"
    }

On testing I get below results ,

For Audit, I do get message but its get logged under CloudWatch log group

For De-identify, we get masked message for date of birth

And for Deny, we get Access Denied error.

**_> Access denied: An error occurred (AuthorizationError) when calling the Publish operation: One or more data identifiers were found_**

Below is summary for three data protection policy types:

Policy Type	Purpose	What Happens When Sensitive Data Is Detected?	Impact on Message Delivery	Ideal Use Case
Audit	Monitor sensitive data exposure	Logs findings to CloudWatch using `/aws/vendedlogs/`	✔️ Delivered	Compliance monitoring, security insights, debugging sensitive data flow
De-Identify	Mask or redact sensitive data before delivery	Sensitive fields are replaced with `#` or a chosen mask	✔️ Delivered (masked)	Sending events to analytics systems, external subscribers, or downstream apps that shouldn't see PII
Deny	Prevent sensitive data from being published	Publish request is blocked with an `AuthorizationError`	❌ Not delivered	Strict compliance environments (PCI/HIPAA), preventing accidental PII leakage

I hope this walkthrough gives you a clear understanding of how SNS Data Protection policies work and where they can be applied across real-world scenarios. By using these capabilities, teams can significantly reduce compliance risks, strengthen data governance, and build more secure, trustworthy systems—without sacrificing the speed or scalability of their event-driven architecture.

Stay secure. Stay responsible. Keep building.

How to Automate IAM Best Practices in CI/CD with IAM Access Analyzer

vikasbanage — Sat, 19 Apr 2025 05:15:09 +0000

Managing IAM (Identity and Access Management) policies securely is one of the most important parts of working with AWS. Developers may accidentally create overly-permissive policies that grant more access than necessary — for example, allowing iam:PassRole to all roles, or opening up sts:AssumeRole without restriction. Without proper checks in place, these risky permissions can silently make their way into your production environment.

In a organizations with multiple accounts, the impact of such mistakes can multiply. That’s why having strong guardrails like IAM Access Analyzer become critical — ensuring that only safe and intentional access is allowed.

🛡️ Before we go further, it's important to understand that AWS Service Control Policies (SCPs) and Resource Control Policies (RCPs) are the first line of defence in any AWS multi-account setup. They define the maximum permissions any user or role can have, regardless of their IAM policies. This blog will not cover SCPs and RCPs, we’ll focus on validating IAM policies during development to ensure we follow least privilege and catch access issues early.

In this blog, we’ll explore how to bring IAM policy validation into CI/CD pipeline using AWS IAM Access Analyzer. This is an example of shift-left security — catching misconfigurations as early as possible, before anything gets merged or deployed.

We'll walk through:

Setting up a GitHub Actions workflow that validates IAM policies.
Using IAM Access Analyzer checks.
Building CloudFormation templates that intentionally pass and fail to understand how policy validation works in practice.

IAM Access Analyzer

IAM Access Analyzer is a security feature in AWS that helps ensure your IAM policies follow best practices and don’t grant unintended access. It supports several capabilities like:

Validating IAM policies against AWS best practices or security standards.
Identifying unused permissions
Analyzing external sharing of AWS resources
Detecting public access to resources.
Generating policies based on CloudTrail access logs

In this blog, we’ll focus specifically on Custom policy checks, which help to validate IAM policies against your organizations security standards — like least privilege or restricted actions. IAM Access Analyzer provides custom policy check APIs to validate IAM policies:

- CheckNoNewAccess – Detects if a policy introduces new permissions compared to a baseline.
- CheckAccessNotGranted – Ensures specific actions or access are not allowed.
- CheckNoPublicAccess – Identifies if a resource policy allows public access.

👉 IAM Access Analyzer custom policy check is a paid feature. As of now, AWS charges $0.0020 per API call for these checks. For example, you make 10,000 calls each month to the IAM Access Analyzer APIs to run custom policy checks across 5 accounts signed up for consolidated billing with AWS Organizations, cost will be $0.0020*10,000 API calls = $20 per month

Now let's start with playing around IAM Access Analyzer. Github code for this blog.

Pre-Requisite

AWS Account
Github Account

CI-CD Flow

Developer pushes changes to repo.
Github action performs IAM Access Analyzer validation.
If policy created by developer violates, workflow gets failed.
If policy created by developer is as per security standard, workflow proceeds with new checks and depployment.

Hands-On

Step 1 - Configure AWS and Github

In order to, configure Github action, we need to first integrate AWS Account and Github, basically we need to establish trust. To do that follow below steps:

If you already have github configured with AWS, you can skip this step.

Login into AWS account, go to IAM → Identity providers →Add provider.
- For the provider URL: Use https://token.actions.githubusercontent.com
- For the "Audience": Use sts.amazonaws.com
Once done, create a role for GitHub. We need to set trust policy for role as below. You need to replace values as per your accounts.
- Principal should be ARN of Identity provider you configured above.
- For Condition, paste github repo link.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::012345678910:oidc-provider/token.actions.githubusercontent.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "token.actions.githubusercontent.com:aud": "sts.amazonaws.com",
          "token.actions.githubusercontent.com:sub": "repo:<githuborg>/<github-repo>:*"
        }
      }
    }
  ]
}

For this role, I have assigned below permission :

AmazonS3ReadOnlyAccess
IAMAccessAnalyzerReadOnlyAccess

Depending on your scenario, you can add more permission like creating IAM roles, policies.

Step 2 - Configure Github Secrets

While running Github action, we will be using role which get assumed to perform necessary actions. But instead of directly hardcoding role ARN we will be configuring it as secrete. For that

Go to your repo → Setting → Secrete & Variable → New repository secretes. Enter name for secrets and paste role ARN

We will be creating two secretes:

IAM Role : This will be used by Github action.
Reference policy Object S3 URL : This will be reference policy which we will be using for validation. As good practice, I will be storing this reference policy on restricted S3 bucket. Part of workflow, reference policy gets downloaded. You can create dedicated S3 bucket for this or use existing one, make sure github role have permission to access it.

Step 3 - Reference Policy

I will be using below reference policy. This reference policy allows all actions by default but explicitly denies a list of CloudFormation actions (like CreateStack, UpdateStack, DeleteStack, etc.) on stacks with names under platformteam/*. It’s designed to protect critical infrastructure owned by the platform team from unauthorized changes, even if general CloudFormation access is permitted elsewhere.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "*",
            "Resource": "*"
        },
        {
            "Effect": "Deny",
            "Action": [
                "cloudformation:CancelUpdateStack",
                "cloudformation:ContinueUpdateRollback",
                "cloudformation:CreateChangeSet",
                "cloudformation:CreateStack",
                "cloudformation:DeleteChangeSet",
                "cloudformation:DeleteStack",
                "cloudformation:DescribeChangeSet",
                "cloudformation:DescribeChangeSetHooks",
                "cloudformation:DescribeStackEvents",
                "cloudformation:DescribeStackResource",
                "cloudformation:DescribeStackResourceDrifts",
                "cloudformation:DescribeStackResources",
                "cloudformation:DescribeStacks",
                "cloudformation:DetectStackDrift",
                "cloudformation:DetectStackResourceDrift",
                "cloudformation:ExecuteChangeSet",
                "cloudformation:GetStackPolicy",
                "cloudformation:GetTemplate",
                "cloudformation:GetTemplateSummary",
                "cloudformation:ListChangeSets",
                "cloudformation:ListStackResources",
                "cloudformation:RecordHandlerProgress",
                "cloudformation:RollbackStack",
                "cloudformation:SetStackPolicy",
                "cloudformation:SignalResource",
                "cloudformation:TagResource",
                "cloudformation:UntagResource",
                "cloudformation:UpdateStack",
                "cloudformation:UpdateTerminationProtection"
            ],
            "Resource": "arn:aws:cloudformation:*:*:stack/platformteam/*"
        }
    ]
}

Step 4 - Github Workflow

With below Github workflow:

Runs on pull requests and pushes to the main branch
Assumes AWS IAM role using OIDC authentication
Downloads reference policies from S3 using GitHub secrets
Performs two IAM Access Analyzer checks:
- ✅ CloudFormation Access Check – Ensures no new permissions are introduced (CHECK_NO_NEW_ACCESS)
- ✅ Principal Check – Verifies only approved principals can assume the role (CHECK_NO_NEW_ACCESS on trust policy)

name: cfn-policy-validator-workflow
on:
  pull_request:
    types: [opened, review_requested]

  push:
    branches:
      - 'main'

permissions:
  id-token: write
  contents: read
  issues: write

jobs: 
  cfn-iam-policy-validation: 
    name: iam-policy-validation
    runs-on: ubuntu-latest
    permissions: write-all
    steps:
      - name: Checkout code
        id: checkOut
        uses: actions/checkout@v4

      - name: Configure AWS Credentials
        id: configureCreds
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ secrets.OIDC_IAM_ROLE }}
          aws-region: us-east-1
          role-session-name: GitHubSessionName

      - name: Fetch reference policy from s3
        id: getReferencePolicy
        run: |
          aws s3 cp ${{ secrets.REFERENCE_IDENTITY_POLICY_CLOUDFORMATION }} ./cloudformation-policy-reference.json
          aws s3 cp ${{ secrets.REFERENCE_IDENTITY_POLICY_PRINCIPAL }} ./allowlist-account-principal.json
        shell: bash

      - name: CloudFormation-Access-Checks
        id: run-aws-check-no-new-access
        uses: aws-actions/cloudformation-aws-iam-policy-validator@v1.0.1
        with:
          policy-check-type: 'CHECK_NO_NEW_ACCESS'
          template-path: './cloudformation-sample/cfn-iam-pass-role-sample.yaml'
          reference-policy: './cloudformation-policy-reference.json'
          reference-policy-type: "IDENTITY"
          region: us-east-1
      - name: New-Principal-Checks
        id: run-aws-check-no-new-access-assumerole
        uses: aws-actions/cloudformation-aws-iam-policy-validator@v1.0.1
        with:
          policy-check-type: 'CHECK_NO_NEW_ACCESS'
          template-path: './cloudformation-sample/cfn-iam-role-principal-sample.yaml'
          reference-policy: './allowlist-account-principal.json'
          reference-policy-type: "RESOURCE"
          region: us-east-1

CloudFormation Template - Failed

Now below template, grants CloudFormation permissions to all stacks (Resource: ''), violating the reference policy by potentially allowing changes to `platformteam/` stacks.

# lambda-pass-template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Resources:
  LambdaFriendlyRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: LambdaSafeCFRole
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
            Action: sts:AssumeRole
      Policies:
        - PolicyName: CFNonPlatformTeamAccess
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - cloudformation:CreateStack
                  - cloudformation:UpdateStack
                  - cloudformation:DeleteStack
                Resource: '*'

After pushing changes to main (ideally pushing to main is not best practice but for this blog only i'm doing), we will see github workflow gets failed as policy violates our reference. policy.
When a custom policy check fails, IAM Access Analyzer returns the statement ID (Sid) and statement index of the specific policy statement that caused the failure. The index is zero-based (starting from 0).

CloudFormation Template - Pass

With below template, a lambda function CloudFormation permissions scoped to devteam/* stacks, staying compliant with the reference policy by avoiding access to protected platformteam/* stacks.

# lambda-pass-template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Resources:
  LambdaFriendlyRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: LambdaSafeCFRole
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
            Action: sts:AssumeRole
      Policies:
        - PolicyName: CFNonPlatformTeamAccess
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - cloudformation:CreateStack
                  - cloudformation:UpdateStack
                  - cloudformation:DeleteStack
                Resource: arn:aws:cloudformation:us-east-1:*:stack/devteam/*

I hope this example gave you an idea about implementation and how it works. If you would like play more on this, you can find reference repo.

Validating IAM policies early in the development process helps prevent security risks before they reach production. By integrating IAM Access Analyzer with GitHub Actions, you can enforce least privilege, block overly-permissive changes, and shift security left — all without slowing down development.

Thanks, cloud builder! Now go forth and validate those IAM policies like a pro. 🔐🛠️

How to monitor Unused Amazon EBS Volumes

vikasbanage — Thu, 30 Jan 2025 05:05:17 +0000

EBS storage is a fundamental component of cloud storage, often used as the primary storage attached to EC2 instances. Even though an EBS volume is attached to an instance, it has a separate lifecycle. If we don’t monitor EBS storage for unused volumes, cloud costs can escalate significantly.

Unless the Delete on Termination option is selected during instance creation, terminating an EC2 instance detaches the EBS volume but doesn’t delete it. Specially in development and testing environments, where EC2 instances are frequently launched and terminated, this often results in a large number of unused EBS volumes.

These unused EBS volumes continue to accrue charges in your AWS account, regardless of whether they are being used.

So why Delete Unused EBS Volumes:

EBS volumes incur charges based on storage usage. Removing unused volumes reduces unnecessary costs.
Regularly cleaning up unused resources simplifies account management and reduces clutter.
Unused volumes may contain sensitive or outdated data. Deleting these volumes can prevents accidental exposure.

In this blog, we will explore how to configure an AWS Config Rule and set up an automatic remediation action using AWS Systems Manager Automation to delete unused EBS volumes. Main goal of this blog is to bring awareness about unused EBS volumes in AWS environment. Deleting EBS volume can be optional step.

Note: It is not advisable to use this solution directly in production. It is always good to test solution and also decide what data strategy within team/business like retention period etc.

Solution Overview

Detects unused EBS volumes using the AWS-managed Config Rule.
Automatically creates a snapshot of the volume. I would recommend this step if you are going to delete unused volumes.
Deletes the unused volume.

Implementation Steps

You can refer this Github repo and deploy required resources using cloudformation.

After cloning this repo, you can deploy Cloudformation template via console. It takes two parameter:

Config rule name - String
IsSnapshotRequired - Boolean.

This Cloudformation will deploy:

IAM Role which will be assumed during detection and remediation.
AWS Config Rule with remediation action.

Remediation is AWS Managed automation document so we don't build our own as of now.

One you deployed cloudormation template, confirm config rule creation by going AWS Config console. AWS Config - Rules

Test the Solution

To test the solution, I'm creating an EC2 instance with 2 EBS volumes attached. While creating an EC2 instance, make sure for EBS volume you select Delete on Termination value No. To see this option, you need to click Advance under Storage(volume) option while creation an EC2 instance.

After creating an EC2 instance, in few minute AWS Config rule get evaluated automatically. You should able to see volumes are in Compliant state as they are attached to EC2 instance.

Now delete EC2 instance. Once EC2 instance deleted, EBS volume status changed to Available. Since volumes are not attached to any EC2 instance now, config rule make these volumes Noncompliant and start deleting it by taking snapshot.

You can check status of remediation by going to System Manager -> Automation

Below you can also EBS snapshot has also been created.

Conclusion

I hope this blog gives you an idea how we can leverage AWS Config and AWS Systems Manager to manage not attached EBS volumes. If you don't want to delete volume, that can also done. You can just make volumes Noncompliant, just fetch list and send it to respective teams. Feel free to modify solution as per your need.

Stay secure, optimised in Cloud!!!

How to Set Up Cross-Account EventBridge

vikasbanage — Mon, 06 Jan 2025 15:43:27 +0000

Introduction

Amazon EventBridge is a powerful event bus service that makes it easier to build event-driven architectures. It allows you to connect different AWS services or even external SaaS applications through a simple and scalable setup.

While EventBridge is incredibly versatile, its ability to target endpoints or consumers is typically restricted to the same AWS account.
The exception is an event bus in a different account, which can be a valid target.

To achieve this, events must be pushed from the source account's event bus to the destination account's event bus. This cross-account communication is essential for managing critical events and ensuring centralized visibility and control.

In this blog, we'll demonstrate this with an example: when a security service in AWS is disabled, it generates an event that is forwarded to a central monitoring account event bus, triggering an alarm. This approach helps maintain a robust, scalable, and secure event-driven architecture across AWS accounts.

Architecture and Workflow

In given architecture :

We have two accounts workload account (source account for event) and central-monitoring account (destination account).
If Security service like SecurityHub disable in workload account, event rule on default event bus will get triggered.
This event rule has target and respective permission to send event to central-monitoring account event custom bus. You can configure default event bus as well.
Custom event bus in central-monitoring account have similar event rule as above. This event rule triggers Lambda function , that can enrich event and send it to SNS.
On SNS, if you subscription conifgured, you will get notification.

Deployment

To create resources via Cloudformation you can refer this Github repo.
Part of this stack below resources get created

Central Account - Receiver

Custom event bus with permission for workload i.e. source account.

{
  "Version": "2012-10-17",
  "Statement": [{
    "Sid": "AllowSourceAccountPutEvents",
    "Effect": "Allow",
    "Principal": {
      "AWS": "arn:aws:iam::99999999999:root"
    },
    "Action": "events:PutEvents",
    "Resource": "arn:aws:events:us-east-1:111111111:event-bus/CentralMonitoringBus"
  }]
}

Note: Here in principle I have specified root of workload account. It means any entity from workload account can publish message. My advice would be to have specific role.

Event rule that matches event related to security hub

{
  "Version": "2012-10-17",
  "Statement": [{
    "Sid": "AllowSourceAccountPutEvents",
    "Effect": "Allow",
    "Principal": {
      "AWS": "arn:aws:iam::714258651552:root"
    },
    "Action": "events:PutEvents",
    "Resource": "arn:aws:events:us-east-1:11111111:event-bus/CentralMonitoringBus"
  }]
}

Lambda function that will be the target of above rule. This lambda will extract details from event and send it to SNS.

Workload Account - Source

Event rule on default event bus. It get trigger when security hub is disable. It has target set to custom event bus of central account.

Demo

In workload account, I disabled security hub, this triggered event rule in workload account.

This event sent further details to central account where event rule triggers lambda function that extract details from event like Accountid, service which is getting disabled. These details will get send to SNS. For this demo I have created Email subscription under SNS topic. So I get notification over an email. As per your need you can set communication channel.

Conclusion

I hope this blog gives you an idea how we can set cross account communication between AWS accounts via even bridge. Event-bridge is one of the important service in AWS which we can use effectively to build scalable application.

Guide to AWS Certifications: Choosing the Right Path for Your Role

vikasbanage — Sun, 29 Dec 2024 17:13:31 +0000

As cloud computing continues to grow, AWS certification is one of the important step for IT/Non-IT professionals who want to advance their careers. AWS provides a range of certifications designed for various roles.

In this blog, we’ll explore the AWS certification paths for several key positions, helping you decide which certifications to pursue based on your career goals.

If you are from Non-IT background, working in sales and marketing then it is always good to Start with AWS Cloud Practitioner Certification. AWS Certified Cloud Practitioner that validates foundational knowledge of AWS Cloud and terminology.
If you are from IT background, having 1–3 years of IT/STEM experience. I would suggest to skip AWS Certified Cloud Practitioner and start with Solution Architect Associate or Developer Associate certification. In below diagrams, I have denoted Cloud Practitioner as an optional with dotted border.
Below I have list down the Role and corresponding certification path.

Also if you would like to understand eco system of AI services in AWS , AWS AI Practitioner is good way to start your AI journey.

Below I have list down the Role and corresponding certification path.

Solutions Architect

As a solutions architect you need to design, develop, and manage cloud infrastructure and resources. Collaborate with DevOps to migrate applications to the cloud.

Application Architect

As an Application architect, you need to make cloud native application scalable, reliable, and manageable across the entire enterprise.

Cloud Data Engineer

As a Cloud Data Engineer, you need to use Cloud native services to automate the collection and processing of structured and semi-structured data, and monitor the performance of data pipelines.

Cloud-Native-Software Development Engineer

As a Software Development engineer, you develop, build, and maintain software across various platforms and devices. You need understand about serverless development options, what cloud-native option that can be used effectively for software development, release process.

Systems Administrator

As an Systems Admin you install, upgrade, and maintain computer components and software on Cloud VMs, integrate automation processes. Understand what are the various option on cloud to automate the process of patching, making sure systems are compliant.

Cloud DevSecOps Engineer

As a DevOps engineer you need to design, deploy, and operate large-scale global hybrid cloud environments. Advocate for end-to-end automated CI/CD DevOps pipelines.

Cloud engineer

As a Cloud engineer, you need implement and operate an organization’s networked computing infrastructure. Set up security systems to ensure data safety.

Cloud Security Architect

Design and implement enterprise cloud solutions. Apply governance to identify, communicate, and minimize business and technical risks. Similar to engineer, you should be aware security services.

Network Engineer

Design and implement computer and information networks, including local area networks (LAN), wide area networks (WAN), intranets, and extranets.

MLOps Engineer

Build and maintain AI and ML platforms and infrastructure, ensuring scalability, reliability, and efficiency across diverse environments to support data scientists and engineers. Design, implement, and operationally support AI/ML model activity and deployment pipelines, including continuous integration/continuous deployment (CI/CD) workflows, monitoring, and performance optimization to streamline production-grade machine learning solutions.

Machine Learning Engineer

As ML engineer, you design machine learning systems, models, and frameworks. Build artificial intelligence (AI) systems to automate predictive models.

Data scientist

Develop and implement AI/ML models to solve complex business problems. Train and fine-tune models using large datasets to optimize accuracy and efficiency, ensuring alignment with business objectives. Evaluate model performance through rigorous testing and validation, deploying them into production environments

I hope this blogs guides you deciding which certification path to choose to get desired role.

AWS EBS Encryption Simplified : Protecting Your Cloud Data Effectively

vikasbanage — Thu, 12 Dec 2024 20:31:05 +0000

When it comes to data storage on the AWS Cloud, AWS offers a variety of services tailored to meet different needs. Two of the most widely used are Amazon S3 (for object storage) and Amazon Elastic Block Store (EBS) (for block storage). If you need a block device for mounting on instances, with fast data access and long-term persistence, EBS is the go-to choice.

Amazon EBS is tightly integrated with services like EC2 and RDS, making it a reliable and versatile option for many workloads.

But here’s the question we all need to ask ourselves: What Should We Care About When Storing Data on AWS?

The simple answer: Security.

AWS follows a shared responsibility model:

AWS’s Responsibility: "Security of the Cloud"
Customer’s Responsibility: "Security in the Cloud"

This means that any data we store on AWS services, including EBS, is our responsibility to protect. If an attacker gains access to your AWS environment, unencrypted data can be an easy target.

So, how do we secure data stored on EBS?

EBS Encryption.

Encrypting your EBS volumes ensures that your data is protected at rest. Encryption also secures all backups created from the volume and snapshots copied from it.

In this blog, we’ll explore two key methods for encrypting EBS volumes:

Default Encryption: Encrypt new volumes automatically during creation.
Encrypt Existing Non-Encrypted Volumes: Add encryption to volumes that were initially created without it.

Enable Default encryption

By default, when account gets created this setting is disabled. We can enable this easily by going to EC2 Dashboard.

Go to EC2 Dashboard → Under Account Attribute — Data protection and security → Manage

Once you click on Manage, you should able to enable encryption by just selecting checkbox. One important thing to note here, KMS Key you will be using. Here I have selected default key, but I would suggest to create Customer Managed KMS key, make sure KMS key policy that should give permission to role/user which going to be used by EC2/Application.

Once you enabled this setting, whenever you create EC2 instance it’s corresponding EBS volume will get encrypted with above key.

BUT, what about the volume which are created without encryption ?

Encrypting existing Non-Encrypted Volume

It may happen that while creating EC2 instance or EBS we didn’t created volume with encryption. This should not be a problem if we discovered earlier before attacker or auditor discovers ;)

Encrypting a non-encrypted volume is a five step process:

Take snapshot of non-encrypted volume.
Copy & Encrypt Snapshot
Create volume from Encrypted Snapshot
Stop EC2 instance & Detach non-encrypted volume.
Attached encrypted volume & Start EC2 instance.

As a part of this blog, I have done this process in my test environment. I would highly recommend to performance this steps in your test environment first and test your application. If all things work in test , only then proceed on production.

Let’s start encrypting a non-encrypted volume.

As a part of this blog, I have spin-up EC2 instance, installed apache server on it and added simple HTML page. One thing also note, Availability Zone in which you created EC2 instance. EBS and EC2 are zone specific. You cannot attach EBS from AZ-1 to EC2 instance in AZ-2.

So make sure to note-down the Availability Zone of EC2, EBS , it will be easy in performing steps.

As you can see in above screenshot, the volume which is attached to EC2 instance is not encrypted.

1. Take snapshot of non-encrypted volume
Part of this step, go to EBS volume attached to EC2 instance. You can easily do this by selecting EC2 → Go to Storage Tab →Click on corresponding EBS volume which start with vol-

In this step it also good to note down device name for respective volume, in this case it /dev/xvda

You will get navigated to below page where you can select from Action to Create Snapshot.

You can find snapshot section on left pane of AWS console. Go to Snapshot console, click on arrange by creation date (it will be easy for us to check recent snapshot if you have too many snapshots.). You will find recent snapshot creation in process.

After some time depending on size, you should able see the status Completed and progress Available. It means snapshot created successfully.

2. Copy & Encrypt Snapshot

Once snapshot is created successfully from step one, it’s time to copy snapshot and also encrypt. To do that, in snapshot console, select snapshot we created earlier → Action → Copy Snapshot.

During this copy process, you can also encrypt snapshot. For encryption you can provide default KMS key or customer managed KMS key. Make sure KMS key you will be selecting should have KMS key policy which grant access to corresponding EC2 role/ user who will be access application.

Here I have selected default KMS key which have permission to users and roles belongs this account only.

Click on Copy Snapshot, this operation should take some time to complete. Depending on size, time can vary. After 2 minute (in my case), I was able to see encrypted snapshot available.

3. Create volume from Encrypted Snapshot
Now we have encrypted snapshot available, we can create encrypted volume. Select encrypted snapshot → Action → Create Volume from Snapshot.

You will get pop-up after you click Create Volume from Snapshot.

On first option, you can select Volume time GP2, GP3, io1/2 etc. But I would recommend not to change volume type in this case. Keep the volume same as it was before like if current EC2 is on GP2 keep GP2 only, if it is on GP3, keep GP3 only. Our goal here is to encrypt volume only, not to play with performance or other parameters of volume. So if size was 8 GiB, keep 8GiB only. In your case if it is 100GiB, keep it that only.

So jumping on to availability zone, make sure to select correct availability zone. If your current EC2 instance is in us-east-1a, you select us-east-1a. In my case I have EC2 instance in us-east-1f so I selected here us-east-1f.

Click on Create Volume. Volume should be available quickly under Volume console. As you can see, Volume state is Avaialble, it means it is not attached to any instance and the one above which is In-use attached to EC2 instance but it is not encrypted.

So our next step is to attached this encrypted volume to EC2 instance. But before that we need to detach Non-encypted volume from EC2.

4. Stop EC2 instance & Detach non-encrypted volume

As this step leads to application downtime or non-avaialblity. I would suggest to have communication to user who access application. Or best way to have high-available application which is behind Load Balancer and do one server at a time.

In this step, we are going to detach non-ecrypted volume. Before that it is recommended to stop EC2 instance first. Go to corresponding EC2 instance → Instance State →Stop Instance.

Once instance stopped, go to corresponding EC2 volume attached to EC2 instance. It should take you to the volume console, select volume → Actions → Detach Volume. You can confirm when pop-up occured.

Once done, volume state should change to available, may be you also need refresh to reflect state.

5. Attached encrypted volume & Start EC2 instance
In this step we will be attaching encrypted volume to EC2 instance.

Select encrypted volume → Actions → Attach volume.

You will get below pop-up.

In instance drop down, select the instance you stopped in pervious step, you can copy instance-id from EC2 console and just search if there are too many instance in drop-down. If you are not able to see instance, check volume availability zone, it may happen EC2 and EBS volume you created from snapshot are in different AZs.

In device name filed, make sure to enter same name that you have noted down in step-1. In my case it is /dev/xvda

Click on attach volume.

Once volume is attached. Volume state will change In-Use.

Go to EC2 console and start EC2 instance. If the instance is not started, check have you mentioned correct device name as before.

Wait of system checks to finish.

Try accessing your application. In my case I was successfully able to access application 🚀🚀🚀

I hope you found this blog useful. Happy Cloud Computing 🚀

Protect Sensitive Data on AWS: A Beginner’s Guide to Amazon Macie

vikasbanage — Wed, 11 Dec 2024 20:41:49 +0000

The exponential growth of data has enabled businesses to innovate more in their products and services, making them more personalized. As organizations adopt Cloud technology like Amazon Web Services (AWS) to modernize their data capabilities and innovate around it, they face the complexity of managing vast information spread across multiple AWS accounts. Each account corresponds to distinct business units and use cases, adding layers of complexity not only in volume but also in the sensitivity and diversity of the data involved.

This involvement ranges from health information (PHI) and payment card industry (PCI) data to personally identifiable information (PII) and proprietary organizational intellectual property. A data breach involving this information can cause significant financial and reputational losses. Therefore, identifying and safeguarding this sensitive data scattered across accounts becomes one of the critical challenges.

In AWS, Amazon Macie emerges as a critical solution in addressing this challenge. Macie plays a pivotal role in answering below fundamental questions that are essential for robust data security and compliance

- Understanding Data Storage: What data do I have in my S3 buckets, and where is it located?

- Assessing Data Exposure: How is my data being shared and stored, and is it publicly or privately accessible?

- Near Real-Time Data Classification: How can I classify my data in near real-time to ensure constant vigilance?

- Identifying Sensitive Information: What PII or PHI might be exposed publicly, and how can I mitigate this risk?

- Automating Compliance and Security: How do I build and implement workflow remediation to meet my specific security and compliance needs?

What is Amazon Macie?

Amazon Macie is a data security service that uses machine learning and pattern matching to identify sensitive data in Amazon S3, offering insights into security risks and automated protection.

It continuously evaluates S3 buckets using built-in & custom criteria for potential data security or privacy concerns providing findings and detailed statistics for informed decision-making.
Macie integrates with Amazon EventBridge and AWS Security Hub, enhancing its capabilities for monitoring and remediation of data security issues.

Use Cases of Amazon Macie

- Compliance Monitoring and Reporting: Organisations subject to regulations like GDPR, HIPAA, or PCI-DSS can use Macie to automatically discover and classify sensitive data, ensuring compliance by identifying where this data resides and how it’s being used or accessed.

**- Intellectual Property Protection: **Companies can leverage Macie to detect and protect intellectual property stored in S3 buckets, ensuring that proprietary information is not inadvertently exposed or accessed by unauthorised users.

- Mergers and Acquisitions (M&A) Data Security Assessmentss: During M&A activities, Macie can be used to quickly assess the data security posture of acquired or merging entities, identifying sensitive data and ensuring that it complies with corporate policies and regulations.

- Educational Institutions Protecting Student Information: Schools and universities can use Macie to safeguard student records and sensitive information, ensuring compliance with education-related privacy regulations.

- Healthcare Data Management: Healthcare organisations can employ Macie to secure patient data, classifying and protecting health information in accordance with HIPAA and other health data protection standards.

Getting Started with Amazon Macie

Enabling Amazon Macie is a very easy task. Amazon Macie is a regional service, so you need to make sure you select the respective region as per your need from the top left corner in AWS Console.

In order to get started with Amazon Macie, AWS Console Mavie Getting Started → Enable Macie.

In multi-account environments, you can monitor Macie’s usage across your organisation in AWS Organizations through the usage page of the delegated administrator account. For this blog, I’m using a single account set up.

Part of pre-requisite, we need to create S3 bucket to store results. For every object analysed, Macie logs details in ‘sensitive data discovery results,’ including objects it couldn’t analyse due to errors. These results are stored for 90 days, but for longer retention, you can configure Macie to save these results in an S3 bucket.

Just to understand how macie shows findings, you can generate sample findings in Macie. Go to Macie Console → Setting → Generate Sample finding

To view finding, you can go to the Finding page on the same console.

As you can see, in above screenshots Macie generates sample findings related to financial, personal data. You can drill more by selecting specific to finding, it will show bucket, total number of sensitive data, type of information.

Let’s create job now.

You can create a custom job where you can define specific bucket, criteria. Go to Job → Create Job

Select bucket to scan → Refine scope, you can select frequency of job, type of files to scan. For demo purposes, I choose One time job. But depending on your requirement we can schedule a job Daily, weekly and monthly.

In this job, I will be checking the date of birth, I’m choosing a custom managed identifier – “DATE_OF_BIRTH”. So this job should be able to detect these keywords in files : bday,b-day, birth date, birthday, date of birth, dob

I created the sample file with below content and uploaded it to the bucket.

Date of Birth: 1961-04-21
Future Date: 2024-03-04
--------------------
date of birth: 1912-01-20
Future Date: 2024-03-06
--------------------
dob: 1956-05-25
Future Date: 2024-03-16

This One-Time job will get executed once you finish with creation. However, you can also execute this job any time after creation. In few minutes, Macie shows below findings

Above we just see the example of Custom Managed Identifier – date of birth. These identifiers are defined by AWS. You can find a list here.

But let’s say you want to define some customer criteria which is not in the AWS list, you can do it using Custom Data Identifier. With custom data identifiers, you can define detection criteria that reflect your organisation’s particular scenarios, intellectual property, or proprietary data—for example, employee IDs, customer account numbers, or internal data classifications.

Macie Benefits

Compliance Assurance: With its capability to discover and classify sensitive data according to various regulatory standards, Macie assists organisations in meeting compliance requirements for regulations such as GDPR, HIPAA, and PCI-DSS, thereby mitigating the risk of compliance-related penalties.
Automated monitoring and actions: Amazon Macie seamlessly integrates with other AWS services like Amazon EventBridge, AWS Security Hub, and AWS Lambda, facilitating the creation of automated workflows which enables quick response and remediation actions.
Global Data Visibility: It provides organisations with a unified view of their sensitive data across multiple AWS regions and accounts, enhancing the ability to manage data security on a global scale. This visibility is crucial for multinational companies dealing with data residency and sovereignty issues.

Macie pricing

Amazon Macie uses a usage-based pricing model, charging based on the volume of data processed for sensitive data discovery.
First-time users receive a 30-day free trial.
During the free trial, Amazon Macie provides an estimate of monthly costs after the trial ends.
Costs can be controlled by limiting the amount of data scanned, such as:
- Excluding CloudTrail logs from scans.
- Focusing on files with specific extensions.
- Scanning files based on tags.

Conclusion

In a world where data breaches can lead to significant financial and reputational losses, the role of Amazon Macie in safeguarding sensitive data is invaluable. Macie brings benefits that enhance an organisation’s security posture. I hope this blog gives you a good understanding of AWS’s data security service and its importance.

AWS Centralised Root Access Management : Simplifying Operations

vikasbanage — Sun, 01 Dec 2024 06:51:42 +0000

I’m sure many of us came across managing Root credentials for multiple account and setting up password for those accounts, setting MFA for those accounts. It is one of the manual process that I don’t find efficient and also sometimes can leads human error. And if we don't set up root credentials, we have critical finding in SecurityHub then ;)

Also, previously each AWS account was provisioned with root user credentials that granted unrestricted access which kind of contradictory to AWS principle "Least Privilege Access." ;)

But Recently AWS launched new capability in IAM where you can centrally manage root access for member accounts in AWS Organizations which I find really nice one. Even for root now, you can have short-term credentials and limited access. Let's explore this feature.

Key Features:

Newly created accounts in AWS Organizations come without root credentials by default, ensuring member accounts cannot sign in as the root user or recover passwords for it. So these are accounts are secure by default like no one can login with root until and unless org admin enables it.
After centralising root access, you can opt to delete root credentials from member accounts. This includes removing the root user password, access keys, signing certificates, and deactivating or deleting multi-factor authentication (MFA).
Centralised monitoring of root credential status across all member accounts aids in demonstrating compliance with security policies and regulatory requirements.

And one good thing is, the new capability does not provide full root access but allows temporary credentials for five specific actions:

Re-enabling Account Recovery: Reactivating account recovery without root credentials.
Auditing Root User Credentials: Read-only access to review root user information.
Deleting Root User Credentials: Removing console passwords, access keys, signing certificates, and MFA devices.
Unlocking an S3 Bucket Policy: Editing or deleting an S3 bucket policy that denies all principals.
Unlocking an SQS Queue Policy: Editing or deleting an Amazon SQS resource policy that denies all principals.

Let's get hands-on on this.

Prerequisites

Your AWS accounts must be managed under AWS Organizations.
Enable trusted access for AWS Identity and Access Management (IAM) in AWS Organizations.
The following permissions are necessary:
- iam:EnableOrganizationsRootCredentialsManagement
- iam:EnableOrganizationsRootSessions
- organizations:RegisterDelegatedAdministrator
- organizations:EnableAwsServiceAccess

Hands-On

Enabling Centralised Root Access

In the AWS Management Console, navigate to the IAM section, select "Root access management" from left pane and enable the desired features. In this demo, I have enabled all features where I can delete S3, SQS policy and root password recovery.
Additionally, if you want delegated admin for this type of activity, assign a dedicated member account as the delegated administrator for IAM to manage root access and perform privileged tasks, ensuring separation of duties and enhanced security.

Now you can either manage these root credential from delegated admin or management i.e Organizations account account.

Once enabled, you will get below view type. We can see in screenshot, for one account root user credentials are present and for other credentials are not present.

New Account Creation

Here I have created new account named - cloudgyan45-dataplatform and by default there are not credentials. Even if you try Forgot password wizard which we normally do for Root user, you will not able to reset password. I have tried and get below message:

Forgot password output : Got an email for password Reset:

But if I click on link, we get output :

Existing Account

Even for existing account, you will be able to delete root credentials and you don't need to manage root password anymore in your password storage or file ;)

Select existing account, on top right corner, click on Take privileged action

Once you delete root credential, you don't need worry about root logins as no one can logged in with root now. But make sure you haven't configured root credential or keys in your application. :)

Taking Privileged action

Now, let's try out last part, taking privileged action. Imagine scenario where by mistake someone put S3 bucket bucket policy as Deny to all or Deny to admin roles as well. Normally, in this case, we logged via root and we delete policy. Consider below kind of policy :

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyAllAccess",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::<bucket-name>",
        "arn:aws:s3:::<bucket-name>/*"
      ]
    }
  ]
}

I tried applying below policy to one of the bucket I have.

> Please note that I have created bucket for demo purpose. Don't try this on your production or any other major workload.

Apply S3 bucket policy:
Access is blocked to bucket
Now, take privileged action - delete bucket policy in this case by login into management account or delegated admin for root access management.
After deletion, we can access bucket again and also policy got deleted.

Things to consider while implementing feature

Make sure access to organisation or delegated admin account is very well protected.
Monitor CloudTrail events : AssumeRoot . This API operation generated someone try to take any action via root access management.

Centralising root access management in AWS Organisations is a powerful feature that simplifies administration and reduces security risks. Properly applied, this feature can significantly enhance your organisation’s security posture and operational efficiency.

Thank you reading this blog, appreciate your time !!

Amazon Inspector Integrations: Strengthening Cloud Security with Security Hub and EventBridge

vikasbanage — Tue, 12 Nov 2024 20:59:52 +0000

Learn how to leverage Amazon Inspector’s integrations for real-time alerts and automated responses.

In this part of our Amazon Inspector series, we’ll explore how to integrate Inspector with AWS Security Hub and EventBridge to enhance your cloud security strategy. By connecting these services, you can centralize vulnerability alerts and automate responses across your AWS environment. If you missed the earlier parts, check out Part 1: Introduction to Amazon Inspector and Part 2: ECR Scanning, CIS Scans, and SBOM for a complete understanding. Now, let’s dive into the power of integration!

We are going to see how we can integrate Inspector with

AWS SecurityHub
Event Bridge

SecurityHub Integration

AWS Security Hub gives you a centralized view of your AWS security posture, allowing you to assess your environment against industry standards and best practices. It gathers security data from multiple AWS accounts, services, and third-party tools, helping you analyze trends and focus on the most critical security issues. It can be integration with Amazon GuardDuty, Audit, Macie etc. You can view complete list here.

By default, inspector sends finding to SecurityHub. If you don't have SecurityHub enabled, you can enable it easily. SecurityHub offers 30 days of trial period.

Sign into Console, search for SecuritHub → Activate.
Under SecurityHub, in navigation → Integrations

In below screenshot, you can see by default it accepts findings.

If you don't want findings in security hub, you can stop it.

Since SecurityHub can have integration with lot of services, in order to filter findings for inspector, you can try below filter.

With inspector findings in SecurityHub, you will have complete security posture of you Cloud. 🚀🚀

EventBridge Integration.

One of my favorite service. It is like salt we need in each dish if we want to make some recipe. So in any type architecture you cannot forgot this service.

We can create event rule, that can get triggered on initial scan, trigger if there is critical vulnerability.
In this demo, I will be creating rule that gets triggered when initial scan is getting completed for EC2 instance and send summary over an email. But if you would like explore can find more patterns, explore here.

In below diagram, you can find possibilities to have alert in place.

For this blog, let's keep it simple. Create an rule for initial_scan and send it to SNS as target.

Sign-in into Console → Go to EventBridge → EvenBus → Rules → Create Rule
Enter details like name and description, keep default eventbus and Rule with event pattern.
Click Next, select Custom Pattern and paste below JSON:

{
  "source": ["aws.inspector2"],
  "detail-type": ["Inspector2 Scan"],
  "detail": {
    "scan-status": ["INITIAL_SCAN_COMPLETE"]
  }
}

Click Next → Select target as a SNS topic. In my case, I have SNS topic created with Email as subscription.
Click Next, review rule setting → finished.

Now we have rule created, create an EC2 instance. Inspector will do scan as soon as EC2 instance created. Once scan is done, you should get an alert on your preferred communication channel.

In my case, I have an email configured, I get summary over an email.

Based on your need, you can also try transforming event message with the help of lambda and send that message to channel.

Integrating Amazon Inspector with AWS Security Hub and EventBridge strengthens your security by centralizing alerts and automating responses. With these tools working together, you can monitor, prioritize, and act on vulnerabilities efficiently across your AWS environment.

Thank you for reading this blog, appreciate your time and passion.

Amazon Inspector Deep-Dive : CIS Benchmark, Container image and SBOM

vikasbanage — Tue, 12 Nov 2024 16:20:45 +0000

In the first part of our Amazon Inspector series, we covered basics Amazon Inspector and covered EC2 and Lambda scanning part.

Now, let’s explore more feature within Inspector: ECR scanning, CIS benchmarks, and SBOM generation. These features give you a more thorough view of your security posture, from container image vulnerabilities to best-practice configurations and software transparency. Whether you’re safeguarding your containerized workloads, ensuring compliance, or tracking your software components, Amazon Inspector has the tools to enhance your security strategy.

ECR Scan - Scanning Docker Images

Amazon Inspector scans container images in Amazon Elastic Container Registry (ECR) for software vulnerabilities, generating findings on package risks. When you enable Amazon Inspector as the preferred scanning service for your private registry, you have two options:

Basic Scanning: Configure repositories to scan images on push or perform manual scans.
Enhanced Scanning: Perform deeper scans at the registry level, detecting vulnerabilities in operating system and programming language packages.

Let's do quick demo. I assume here, you have activate ECR scan type in Amazon Inspector. If not, you can do it by going to Inspector → In the navigation pane, choose Account management → Select Scan Type and activate.

ECR Scan Demo

For this blog purpose, I have dockerise simple NodeJS app which has express dependency. I'm using outdate packages and image here.

app.js

const express = require('express');
const app = express();
const port = 3000;

// Basic route to trigger a response
app.get('/', (req, res) => {
    res.send('Hello, this is a vulnerable Node.js app!');
});

// Start the server
app.listen(port, () => {
    console.log(`App listening at http://localhost:${port}`);
});

package.json

{
  "name": "vulnerable-node-app",
  "version": "1.0.0",
  "description": "A vulnerable Node.js app",
  "main": "app.js",
  "dependencies": {
    "express": "3.0.0"
  }
}

Dockerfile

# Node.js vulnerable Dockerfile

# Use an outdated Node.js base image
FROM node:10

# Set working directory
WORKDIR /usr/src/app

# Copy package.json and install outdated dependencies
COPY package.json ./
RUN npm install

# Copy app source code
COPY . .

# Expose the app port
EXPOSE 3000

# Start the app
CMD ["node", "app.js"]

Build docker image and push to ECR repository. Within few minutes, Inspector will scan ECR repository and images and will generate findings.

Understanding Findings

To view findings Go to Inspector → Findings → Container Images

Findings will give us:

CVEs which are critical to fix.
On selection of specific CVEs, it gives which package is currently installed and which package will solve vulnerability.
Under remediation, it will give us hint what needs to be done.

Here fix is easy, we just need update LTS of NodeJs image and express.

CIS Scan

Amazon Inspector’s CIS scans assess your EC2 instance configurations against Center for Internet Security (CIS) benchmarks to ensure they meet security standards. You can run these scans on-demand or on a schedule after enabling EC2 scanning.
To target specific instances, create a scan configuration with instance tags and a CIS Benchmark level, which can be applied across multiple accounts if you’re a delegated administrator.

Configuring CIS Scan

To grant permissions to run CIS scans, attach AmazonSSMManagedInstanceCore and AmazonInspector2ManagedCispolicy to EC2 instance role.

Go to Inspector → CIS Scan → Create Scan
Enter required details. LEVEL_1 corresponds to foundational security and LEVEL_2 is for more critical workload for data security.
In this demo, I'm targeting EC2 instance which have tag CISScan=True

Below screenshot is for individual account.

If you would like to configure CIS scan from delegated admin i.e. from central account , it is also possible. From central setting you should able to specify more than one account, also manage setting centrally.

Understanding Scan

Within few minutes, CIS scan should be complete.

As we can see :

Under Resource Status you should able to see ❌ which means CIS checks failed, ➖ means resources is not evaluated and ✅ CIS checks are passed.
If you click on specific title, it gives you details about that CIS checks e.g. in my case we see journald should be configured to write logfiles to persistent disk.

Running CIS on Private EC2 instance

When running CIS scans on private instances, you’ll need VPC endpoints for Systems Manager services. Key endpoints include:

ssm.amazonaws.com
ssmmessages.amazonaws.com
ec2messages.amazonaws.com

Inspector uses OVAL (Open Vulnerability and Assessment Language) definitions for assessments, stored in Amazon S3. Allow listing these Amazon S3 buckets in VPCs ensures access to the required definitions for CIS scans:

inspector2.amazonaws.com
s3.amazonaws.com
ssm.amazonaws.com
ssmmessages.amazonaws.com

This setup allows Inspector to benchmark, assess, and secure EC2 instances according to CIS standards.

Software Bill of Materials - SBOM

A Software Bill of Materials (SBOM) lists all open-source and third-party components in your codebase. Amazon Inspector generates SBOMs for monitored resources, which can be exported in CycloneDX or SPDX formats to an Amazon S3 bucket. Note that SBOM export is not currently supported for Windows EC2 instances.

Why SBOMs are important :

SBOM provides a detailed inventory of software components, allowing organizations to identify and address vulnerabilities in third-party and open-source components more effectively.
SBOM ensures transparency by documenting all components within the software, which is vital for regulatory compliance and meeting industry standards.
In the event of a security incident, an SBOM allows teams to quickly locate and assess affected components, speeding up response and mitigation efforts.

Now, let's start for quick demo.

Before starting with SBOM export, we need to have Customer Managed KMS and S3 bucket created in advance.

I have created Customer Managed KMS with below policy:

{
    "Version": "2012-10-17",
    "Id": "key-consolepolicy-3",
    "Statement": [
        {
            "Sid": "Enable IAM User Permissions",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::111122223333:root"
            },
            "Action": "kms:*",
            "Resource": "*"
        },
        {
            "Sid": "Allow access for Key Administrators",
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam::111122223333:role/<rolename>"
                ]
            },
            "Action": [
                "kms:Create*",
                "kms:Describe*",
                "kms:Enable*",
                "kms:List*",
                "kms:Put*",
                "kms:Update*",
                "kms:Revoke*",
                "kms:Disable*",
                "kms:Get*",
                "kms:Delete*",
                "kms:TagResource",
                "kms:UntagResource",
                "kms:ScheduleKeyDeletion",
                "kms:CancelKeyDeletion",
                "kms:RotateKeyOnDemand"
            ],
            "Resource": "*"
        },
        {
            "Sid": "Allow Amazon Inspector to use the key",
            "Effect": "Allow",
            "Principal": {
                "Service": "inspector2.amazonaws.com"
            },
            "Action": [
                "kms:Decrypt",
                "kms:GenerateDataKey*"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:SourceAccount": "111122223333"
                },
                "ArnLike": {
                    "aws:SourceArn": "arn:aws:inspector2:us-east-1:111122223333:report/*"
                }
            }
        }
    ]
}

Also, create S3 bucket with below policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "allow-inspector",
            "Effect": "Allow",
            "Principal": {
                "Service": "inspector2.amazonaws.com"
            },
            "Action": [
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:AbortMultipartUpload"
            ],
            "Resource": "arn:aws:s3:::<bucketname>/*",
            "Condition": {
                "StringEquals": {
                    "aws:SourceAccount": "111122223333"
                },
                "ArnLike": {
                    "aws:SourceArn": "arn:aws:inspector2:us-east-1:111122223333:report/*"
                }
            }
        },
        {
            "Sid": "allow-role-access",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::111122223333:role/<rolename>"
            },
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:ListBucket",
                "s3:GetObjectAcl",
                "s3:PutObjectAcl",
                "s3:PutBucketPolicy"
            ],
            "Resource": [
                "arn:aws:s3:::<bucketname>",
                "arn:aws:s3:::<bucketname>/*"
            ]
        }
    ]
}

Once we have KMS and S3 bucket in place, let's export SBOM for EC2.

Go Inspector → In Navigation choose Export SBOM.
On the Export SBOMs page, use the Add filter menu to select specific resources for the report. Without filters, Amazon Inspector will export reports for all active resources. For this demo purpose, I'm exporting EC2 which has specific tag.
Select Export format. Choose anyone.
Choose S3 bucket and KMS key we have created in above steps and Start Export.

Let's wait to finish export. Once finished, go to S3 bucket.

For now, we will just download file and try to understand it. Below I have pasted part of downloaded file.

{
   "bomFormat":"CycloneDX",
   "specVersion":"1.4",
   "version":1,
   "metadata":{
      "timestamp":"2024-11-09T14:48:33Z",
      "component":{
         "type":"operating-system",
         "name":"AMAZON_LINUX_2023"
      },
      "properties":[
         {
            "name":"amazon:inspector:ami",
            "value":"ami-063d43db0594b521b"
         },
         {
            "name":"amazon:inspector:arch",
            "value":"x86_64"
         },
         {
            "name":"amazon:inspector:account_id",
            "value":"11112222333"
         },
         {
            "name":"amazon:inspector:resource_type",
            "value":"AWS_EC2_INSTANCE"
         },
         {
            "name":"amazon:inspector:instance_id",
            "value":"i-0a4358565c08e4"
         },
         {
            "name":"amazon:inspector:resource_arn",
            "value":"arn:aws:ec2:us-east-1:11112222333:instance/i-0a4358565c08e4"
         }
      ]
   },
   "components":[
      {
         "type":"application",
         "name":"libedit",
         "purl":"pkg:rpm/libedit@3.1-38.20210714cvs.amzn2023.0.2?arch=X86_64&epoch=0&upstream=libedit-3.1-38.20210714cvs.amzn2023.0.2.src.rpm",
         "version":"3.1",
         "bom-ref":"40853ebb7fa05c9370e08063b4fd6e94"
      },
      {
         "type":"application",
         "name":"python3-libcomps",
         "purl":"pkg:rpm/python3-libcomps@0.1.20-1.amzn2023?arch=X86_64&epoch=0&upstream=python3-libcomps-0.1.20-1.amzn2023.src.rpm",
         "version":"0.1.20",
         "bom-ref":"b85e33c25b9c33135da9c73eb32c429c"
      },

File contains:

Properties for EC2 instance like AMI-ID, architecture.
Under components, you see all packages on EC2 instance, its version.

But there is no fun just to have JSON in S3, download and checking manually, so what we can do:

Connect it to Athena to search for specific package.
Integrate with OpenSearch to build package search engine.
Analyze File with Lambda as soon as SBOM export done for any specific package.

But these points are for other blogs or some other day ;)

In this second part of Inspector series, we explored how Amazon Inspector’s ECR scanning, CIS benchmarks, and SBOM exports strengthen your cloud security. These tools help you detect vulnerabilities, ensure compliance, and gain visibility into your software components. In next part, we will be checking on what services we can integrate with Amazon Inspector.

Appreciate your time and passion for reading blog !!!