Forem: David Krohn

OWASP Top 10 (2025) and AWS WAF: Putting Managed Rules in Context

David Krohn — Wed, 04 Feb 2026 15:55:00 +0000

The OWASP Top 10 is not a tool recommendation, nor is it a product matrix. They explain systemic risks in web and API apps, no matter the platform, the cloud or the vendor. As a matter of fact, however, the same question arises repeatedly:

Which of these risks can actually be addressed using AWS WAF Managed Rule Groups?

The short answer: AWS WAF can limit risk and make attacks visible, but it cannot fundamentally “fix” them.

AWS WAF is not a substitute for secure design. As a guardrail it covers known patterns and anomalies, but not against structural vulnerabilities to the application’s security model.

In this way a tidy mapping is also useful to know where AWS WAF is useful, where it functions only at the HTTP layer, and where additional controls are required.

AWS WAF in security model

The role of AWS WAF is a logical interface with client and application. It will analyze HTTP(S) requests in terms of managed rule groups and optional custom rules. Its strength resides where attacks materialize in the form of requests: injection payloads, previously identified bad inputs, automated abuse, credential stuffing, or unexpected traffic bursts.

Especially in environments where this value extends beyond blocking, such as managed rule groups, it has a signal and evidence functionality: they emit labels, counters, block events, and structured logs that can be processed centrally.

Just as important, though, is a boundary: AWS WAF knows nothing about business logic, has no role models, and makes no cryptographic decisions. Anything that occurs before or after the HTTP request is beyond the scope of the system.

OWASP Top 10 (2025) → AWS WAF Managed Rule Groups

The following table maps the OWASP Top 10 categories to AWS-managed WAF rule groups.

OWASP Top 10 (2025)	AWS WAF Managed Rule Groups	Interpretation
A01 Broken Access Control	AWSManagedRulesCommonRuleSet AWSManagedRulesAdminProtectionRuleSet AWSManagedRulesAmazonIpReputationList AWSManagedRulesBotControlRuleSet	Reduces scanning and automated abuse on exposed endpoints, but does not enforce authorization or prevent insecure direct object references (IDOR).
A02 Security Misconfiguration	AWSManagedRulesCommonRuleSet AWSManagedRulesKnownBadInputsRuleSet AWSManagedRulesAmazonIpReputationList	Blocks simple exploit payloads, but does not fix insecure defaults, missing security headers, or IAM misconfigurations. Such aspects can only be partially addressed through targeted custom WAF rules.
A03 Software Supply Chain Failures	-	Originates before runtime (dependencies, CI/CD, artifacts) and lies entirely outside the WAF scope.
A04 Cryptographic Failures	-	TLS configuration, key management, and cryptographic algorithm selection are decided outside the WAF. Web application firewalls cannot detect or prevent these issues. Cryptographic weaknesses are instead identified through configuration and compliance tools such as Prowler.
A05 Injection	AWSManagedRulesSQLiRuleSet AWSManagedRulesCommonRuleSet AWSManagedRulesKnownBadInputsRuleSet	Using the AWSManagedRulesSQLiRuleSet, typical SQL injection patterns in query parameters, request bodies, and cookies can be reliably detected.
A06 Insecure Design	AWSManagedRulesCommonRuleSet AWSManagedRulesBotControlRuleSet	Managed rule groups such as AWSManagedRulesCommonRuleSet (indirectly) and AWSManagedRulesBotControlRuleSet, combined with rate-based rules, can limit automated abuse and anomalous request patterns, but do not detect business logic flaws.
A07 Authentication Failures	AWSManagedRulesATPRuleSet AWSManagedRulesACFPRuleSet AWSManagedRulesBotControlRuleSet AWSManagedRulesAmazonIpReputationList	Protects login and signup flows from automated abuse, but does not replace MFA or secure session design.
A08 Software & Data Integrity Failures	AWSManagedRulesCommonRuleSet (partial)	Blocks exploit payloads, but does not verify signatures or build integrity.
A09 Security Logging & Monitoring Failures	WAF logs & labels	Provides actionable signals, but does not replace a SIEM or incident response setup.
A10 Mishandling of Exceptional Conditions	AWSManagedRulesCommonRuleSet AWSManagedRulesKnownBadInputsRuleSet AWSManagedRulesBotControlRuleSet AWSManagedRulesAntiDDoSRuleSet	Filters anomalies and exceptional request patterns, but does not fix error handling in application code.

Anti-DDoS to protect availability

The basic structure of the AWS WAF involves pattern matching plus a behavioural part such as AWSManagedRulesAntiDDoSRuleSet. The goal is not to detect specific payloads, but rather to identify traffic anomalies that indicate Layer-7 DDoS attacks or abnormal load spikes. AWS WAF can automatically respond to these events by blocking browsing momentarily or launching silent challenges. These are combined with specific labels being applied to requests made in the event.

Conclusion

When positioned correctly AWS WAF Managed Rule Groups are an effective solution as a Baseline. They allow detection and remediation of injection attacks, known malicious inputs, automated abuse and traffic anomalies. Nevertheless, the central OWASP risks like supply chain issues, cryptography, and insecure design are explicitly excluded from their focus.

IAM Auto-Remediation: Enforcing Least Privilege Automatically

David Krohn — Mon, 02 Feb 2026 16:04:06 +0000

Misconfigured IAM roles and policies is one of the major root causes of serious cloud incidents: permissions (e.g., admin rights) are too many rather than the principle of least privilege. It isn’t often malicious — most of the time it’s just making it work that becomes drift quietly. This hits hard: once a token is compromised an over-privileged role will do widespread damage in the system data access, logging/evidence tampering, privilege escalation, key policy abuse. In healthcare, that’s more than security; it’s an immediately actionable governance and compliance risk. Least privilege is not a preference. NIST frames it as a control (AC-6), while AWS emphasizes it as a core IAM best practice. Source: NIST SP 800-53 Rev. 5 (AC-6 Least Privilege). Read The Three Pillars of Digital Sovereignty for the bigger picture on why we treat this as an operational governance control. There, we demonstrate that sovereignty is not determined by location or certificates but actual control points: identities, keys, data flows, operations—and exactly why "audit-ready" means implementing those controls on-going.

IAM is the “WHO” control point. Auto-remediation does it - that is, guardrails + evidence in near real time and is not stuck in manual review bottleneck.

The risk of attack is increased because of permanent admin privileges.

It isn’t only a blanket rule: It’s the absolute absence of boundaries. In platforms with many teams, pipelines, and roles such as that one infected token can quickly become “everything”.

Blast radius can explode: one credential could equate to an overall platform control.
Evidence becomes weak: admin can tamper with records, policies and a critical path.
Operational drift: “temporary admin” becomes the default, quietly.
Forensics rather than clarity: “who altered what when?” becomes detective work.

A common pattern.

A batch role is assigned admin “just for now” weeks later, it’s still attached. Then a CI token is leaked—and now not just data but your evidence pipeline and other essentials are accessible and in your hands.

Architecture: event-driven IAM guardrails.

Rather than waiting for occasional reviews we address IAM variations in near real-time (CloudTrail gives you API events, EventBridge captures relevant patterns (e.g. “AWS API Call via CloudTrail”), Lambda remediation takes you through guardrails).

CloudTrail: It records IAM API calls (e.g., AttachRolePolicy, PutRolePolicy).
EventBridge matches the most serious events and sends them to remediation.
Lambda checks: an admin policy? wildcard admin? Optional: a machine-check findings via policy analysis.
Remediation: disconnect admin + set up a permissions boundary (quarantine/seatbelt). Permissions boundaries define the maximum permissions to work for IAM principals.
Evidence: tags/logs/trigger details → audit-ready traceability.

Why Access Analyzer helps here.

For IAM access analytic work, the IAM Access Analyzer offers ValidatePolicy to document the IAM policies and the returned structured findings—handy if you want enforcement to generate machine-readable evidence.

Example: AWS IAM Access Analyzer - Policy validation (ValidatePolicy)

Safe by default: rolling out remediation without chaos

Auto-remediation is powerful—which is exactly why rollout must be controlled. In regulated environments, a staged model works well: observe first, steer next, enforce last.

Observe: collect findings only, no enforcement.
Warn: notify + ticket/Slack, add evidence tags.
Quarantine: apply a permissions boundary (block escalation, avoid breaking workloads).
Block: hard remediation (detach admin immediately) if risk is unambiguous.

In this post we default to safe mode: remediate only allowlisted roles or roles tagged foundra:autofix=true. By doing that, there will be fewer surprises and the same security outcome.

Audit-ready operations: evidence & metrics.

There will be no sovereignty if you can’t quantify it. For IAM guardrails, these measures translate ‘policy intent’ to operational state.

Policy coverage: share of workload roles with boundaries / without admin policies.
Risk findings: admin/wildcard events per team/account/service.
MTTR: amount of time from risky change → remediation (seconds, not days).
Evidence tags: foundra:remediated, foundra:trigger, foundra:reason.

Operational mini-check.

Can you explicitly indicate when certain IAM roles are a growing risk?
Can you evidence the timeliness of triggering remediation, by which event, and with what result?
Is the default secure—even when people are operating under time pressure?

Architectural example: analyzer and rule and remediation.

With this example we outline a pragmatic baseline: AdministratorAccess is not allowed on workload roles. We also identify basic “wildcard admin” inline policies (Action: "*" & Resource: "*") and have a permissions boundary as a guardrail.

Permissions boundaries are an AWS application capability to limit permissions to IAM principals, a “seatbelt” for auto-remediation, ideally minimally invasive, reversible and measurable.

Example: AWS IAM - Permissions boundaries

Access Analyzer: policy analysis / ValidatePolicy as well as structured findings.
EventBridge: To filter out dangerous IAM API changes (CloudTrail events).
Remediation Lambda: disengage admin; apply boundary, tag evidence.
Boundary policy: limit “maximum permissions” (seatbelt) without compromising workloads unnecessarily.

Why even more so in healthcare.

IAM is more than “security” in healthcare platforms. IAM is the technical implementation of privacy and governance objectives into actionable reality—and least privilege is the tipping point between “incident” and “incident with massive exposure”.

Least privilege reduces exposure of sensitive data (PII/PHI) as much as possible in an event.
Guardrails safeguard evidence and operations (logs, keys, policies) from tampering.
Automation makes MTTR lower and compliance on a day-to-day basis.

Combined with the sovereignty framework: you govern WHO (IAM), you perform HOW (guardrails/remediation), and you produce EVIDENCE (tags/logs/metrics)—as one unified machine.

An example implementation which shows this pattern is here.

Securing Digital Sovereignty for regulated Industries

David Krohn — Thu, 13 Feb 2025 10:35:08 +0000

As the digital world continues to evolve, the regulated sector is challenged to maintain strong controls over personal data.. The concept of digital sovereignty not only safeguards governments and corporations from losing control over their data on a routine basis, but empowers them to take advantage of what is already technologically available with the intelligence of the cloud. Critical Infrastructure (KRITIS) regulated sectors and domains like finance, healthcare, energy and telecommunication sectors have a high level of data governance for security, compliance and local control of data needs. AWS is such a robust platform to tackle these type of challenges that, with secure, resilient and compliant cloud services, organisations can innovate without boundaries.

Understanding the Challenges of Digital Sovereignty

Digital sovereignty can be divided into 3 pillars, data sovereignty, technological independence and security.

Data sovereignty is an essential component of digital sovereignty. Governments and regulated sectors must protect sensitive information, secure data sovereignty and protect data from unauthorised access. There are various regulations for this, including:

Germany's critical infrastructures (KRITIS) are defined and regulated by their essential services. An independent and reliable source for KRITIS specifications is OpenKRITIS.
The financial sector is regulated by DORA and ensures its digital resilience.

To gain more insights into data sovereignty and effective practices, check out our blog post about Confidence in the cloud with data sovereignty.

Technological independence means freeing yourself from dependence on a single provider. By using AWS technologies, your organisation can gain flexibility, eliminate dependencies and drive innovation. Here you can get a deeper insight into how to maintain technological independence in the AWS cloud.

Security is a key pillar of digital sovereignty, protecting organisations from cyber-attacks while also ensuring compliance, resilience and reliability. Security as the foundation of digital sovereignty helps you to promote trust, protect sensitive data and prevent reputational and financial losses.

Traditional Data Centers vs. Cloud: Cutting Costs and Complexity

Moving to the cloud eliminates the complexities of traditional data center management, offering organizations greater agility, cost savings, and operational efficiency.

✅ Simplified Operations - Cloud services eliminate the need to manage cumbersome physical servers and complex network configurations, freeing your teams to create value and scale the business with speed.

💰 Cost Efficiency: Cloud computing reduces the capital and operational expenses of maintaining, upgrading, and securing on-premise data centers. With a pay-as-you-go-modell, you pay only according to the usage, thus optimizing costs for scalability.

🔄 Flexible Service Models: IaaS, PaaS, SaaS - Scale and tune your cloud model with different approaches that will provide a mix of control versus flexibility, suitable to meet each organization's business needs.

🔖 Built-in Compliance – Take advantage of certifications from AWS and other cloud providers (like C5) to ease the certification process, as many services already comply with industry standards.

Digital Sovereignty within AWS

AWS is committed to improving digital sovereignty, to overcome the challenges of data control, there is Confidential Computing, which encrypts sensitive and protects data from unauthorised access. In addition, ende 2025 is opening a European Sovereign Cloud, an isolated AWS cloud operated by a separate company. It provides a robust solution for organisations with strict data management requirements, especially in critical sectors. It also ensures that all staff operating this cloud or supporting customers are EU citizens and located in the EU.

AWS offers built-in services such as Audit Manager and Config for automated compliance monitoring and real-time reporting that ensure organisations meet ongoing compliance with ease.

With many services already C5 certified, AWS improves security in highly regulated industries by simplifying compliance through automated framework mapping, which lessens the complexity and burden of various compliance checks.

AWS Driving Innovation Across regulated Industries

Leading banks rely on AWS to safeguard financial transactions, ensuring compliance, data integrity, and protection from cyber threats. Insurance companies use AWS to effectively manage and process sensitive health data while upholding the highest security standards. At the same time, Deutsche Bahn counts on AWS to support and maintain its essential transportation infrastructure, guaranteeing reliability and operational excellence.

Enhancing Your AWS CDK Projects with Testing

David Krohn — Thu, 16 Jan 2025 07:33:45 +0000

Automated unit and integration testing validates system components and increases confidence in your infrastructure code, but unfortunately this is a very neglected topic at CDK. But let's start at the beginning. A few weeks ago, we decided to start a new version of a CDK construct library in a project - this meant taking our CDK testing to a new level. We wanted to test each custom construct with multiple unit and integration tests. Here is an explanation of the difference between a unit test and an integration test.

Unit testing in the CDK focuses on testing the logic within CDK constructs, which are reusable components that define and configure cloud resources. These tests simply synthesized CDK code to a CloudFormation template without actually deploying any AWS resources. In short, AWS services are simply mocked up and we compare the generated template against our checks.

Integration testing, on the other hand, deploys the actual resources to an AWS account to ensure that the deployed resources work together correctly. These tests verify the behavior of the entire AWS application, including permissions, networking, and service integrations. Unlike unit testing, integration testing requires a real AWS account and uses tools such as the AWS SDK, AWS CLI, and custom resources to verify functionality in a real-world environment. We use the integ-test module - part of the CDK library - for testing in CDK.

Since we're just starting out with this new topic and wanted to see how other people are testing their infrastructure, we decided to do some research. Unfortunately there's a lack of good examples on the web and the same example is copied across many blog posts on the web. 🫠

This situation is over now. After troubleshooting, testing, failing and gaining experience. We are now in a position to share our experiences and examples with you. 🙌🏻

Unit Testing

No, you don’t need to test every line of your CDK application - Yan Cui

I see it the same way as Yan Cui, if you only declare and use official CDK constructs, we don't necessarily have to test, I would only test to ensure compliance (security and regulatory compliance), because as in the shift left principle, we can prevent this before we deploy insecure resources by CDK testing. However, this should not be used as an alternative just as an addition to check if you comply with security and regulatory frameworks in your AWS environments. You should still use config and/or conformance packs that are rolled out organization-wide to check AWS resources against your compliance requirements. But now lets start with an easy example. Since Security is everyones job we should test if all RDS Instances will be encrypted. This can be easily done like that.

  template.allResourcesProperties("AWS::RDS::DBInstance", {
    StorageEncrypted: true,
  });

TypeScript

In addition, we should use a custom KMS key to encrypt the RDS Intention and this key should be rotated regularly.

// KMS Key rotation Check
  template.allResourcesProperties("AWS::KMS::Key", {
    EnableKeyRotation: true
  });


//RDS Instance Check
template.allResourcesProperties("AWS::RDS::DBInstance", {
  StorageEncrypted: true,
  KmsKeyId: Match.anyValue(),
});

TypeScript

Unit tests like this could be implemented in any CDK project when using tools like projen - or whenever you create your own custom construct library.

Imagine you have a platform team and a lot of application development teams, and you want to give them easily deployable infrastructure code that meets your company's security and regulatory frameworks, and includes resources that are commonly used for almost every application.

Now let's take it a step further, we want to provide our application teams with constructs that expose databases or other resources according to our corporate specifications and legal requirements. this often requires custom resources. to find these in the synthesized stack, we need a little more configuration to specify the resources and check that the references are set correctly. In the following example, we search for a KMS key using the properties in the synthesised template and check that the reference is set correctly in the following secret.

    // eslint-disable-next-line @typescript-eslint/no-explicit-any
    function logicalIdFromResource(resource: any) {
      try {
        // eslint-disable-next-line @typescript-eslint/no-unsafe-argument
        const resKeys = Object.keys(resource);
        if (resKeys.length === 0) {
          throw new Error("No Resource found.");
        }
        if (resKeys.length !== 1) {
          throw new Error("Resource is not unique.");
        }
        const [logicalId] = resKeys;
        return logicalId;
      } catch (err) {
        console.log(resource);
        throw err;
      }
    }

    const testKmsKey = template.findResources("AWS::KMS::Key", {
      Properties: {"Description": "DescriptionTestKey"}
    });


    const testKmsKeyLogicalId = logicalIdFromResource(testKmsKey);

    template.findResources("AWS::SecretsManager::Secret", {
      Properties: {
        "Name": "test",
        "KmsKeyId": {
          "Fn::GetAtt": [
            `${testKmsKeyLogicalId}`,
            "Arn"
          ]
        },}});

TypeScript

Integration Testing

As mentioned at the beginning, integration tests are concerned with deploying resources from a CDK application to a test environment to test the application from start to finish. These tests can take a little more time than a unit test, but they also ensure that the resources are used as intended. Integration tests allow you to use different assertions to check resources for properties or settings.

For example, there is an awsApiCall that can be executed to check whether certain values are in a resource. Here is an example where we check a secret content.

// Check whether the secret was created with the correct content
const assertion = integTest.assertions.awsApiCall("secrets-manager", "GetSecretValue", {
  SecretId: SECRETARN,
}, ["SecretString"]);
assertion.provider.addToRolePolicy({
  Effect: "Allow",
  Action: ["kms:Decrypt*"],
  // reference errors when using the key. 
  Resource: ["*"],
});
assertion.assertAtPath("SecretString.username", ExpectedResult.stringLikeRegexp("test"));
assertion.assertAtPath("SecretString.password", ExpectedResult.stringLikeRegexp(".*?"));

TypeScript

With this assertion we use a nice Feature, instead of checking the whole response object at the end, we use "assertAtPath" to get back only parts of the secret string to do a regex on it.

During the integration test of a CDK app, two stacks are deployed, one stack containing the resources and another stack containing the testing resources. CloudFormation Outputs can be used to pass arns or other variables between the stacks, for example to read a specific secret. This would look as follows in the example.


new cdk.CfnOutput(testStack, "AdminSecretArn", {
  value: adminSecret.secretArn,
  exportName: testStack.stackName+"-AdminSecretArn"
});


const SECRETARN = cdk.Fn.importValue(testStack.stackName+"-AdminSecretArn");

TypeScript

In addition to the assertions for “awsApiCalls”, there is also the “invokeFunction”. This triggers an AWS Lambda Function that you can define yourself within the assertionStack. Using a lambda function you can execute far more complex tests which return a result at the end which can be checked using the CDK testing framework. For example, if you do not know exactly how long the function will take or when an object will end up in an s3 bucket, you can use a "waitForAssertions" method to define at what interval and for how long a lambda function should be triggered.

// Invoke the assertion Lambda function to validate the received payload in s3
integTest.assertions.invokeFunction({
  functionName: assertionLambda.functionName,
}).expect(ExpectedResult.objectLike({
  Payload: "200"
})).waitForAssertions({
  totalTimeout: Duration.minutes(5),
  interval: Duration.seconds(30),
});

TypeScript

References

Conclusion

Automated unit and integration testing is critical for validating CDK constructs and ensuring the security and compliance of infrastructure code. By improving CDK testing-from unit tests that validate synthesized CloudFormation templates to integration tests that deploy and verify real AWS resources-we can build robust and secure cloud architectures. However, there are still challenges to overcome as the test constructs are still in alpha.

We hope that our insights into testing the CDK have helped you and made it easier for you to get started.

Security as the foundation of digital sovereignty

David Krohn — Sat, 04 Jan 2025 10:01:25 +0000

Any organization processing and storing data of any form is potentially a target of cyber-attacks. Therefore you need to protect your data against such attacks. Especially when you are handling sensitive data. That is why security is an important main pillar of digital sovereignty.

Security is not just about protecting against unauthorised access and cyberattacks. It is also about ensuring the resilience, compliance and reliability of your digital assets. This is achieved through strict access controls and robust authentication mechanisms that protect your data from breaches. By implementing robust security measures, you can protect your customers' data and ensure compliance with relevant regulations, such as General Data Protection Regulation (GDPR) or DORA (Digital Operational Resilience Act).

Beyond technical measures, security fosters trust, as customers and partners are more likely to engage with organisations that prioritise data protection, making it a fundamental aspect of credibility and digital sovereignty. On the other hand companies which have security incidents additionally suffer from big losses in their reputation.

In summary, with security you want to:

Protect Data - Implementing least privilege and encryption protect data and counter the rise of threats.
Comply with regulations - Ensure compliance and meet legal requirements for your data, such as GDPR.
Maintain customer confidence and trust - Build trust by protecting privacy and demonstrating a commitment to digital sovereignty.

Security in the Cloud

Securing the cloud is a dynamic and critical process that requires a blend of robust technical controls, well-defined organizational policies, and proactive continuous monitoring. By addressing the unique challenges of cloud environments with a structured approach, you can safeguard data, applications, and infrastructure against evolving threats.

Here's a comprehensive guide to achieving and maintaining cloud security:

Shared Responsibility

AWS shares the responsibility for security and compliance with you. AWS manages and controls infrastructure components such as physical security, the virtualisation layer, and the host operating system.

You as a Customer are responsible for managing the security in the cloud like guest operating system, application software, updates, patches, and security configurations, such as firewalls and access permissions.

Identity and Access Management (IAM)

Identity and access management helps you to securely manage and scale access to your resources, between your resources and for your employees.

Using temporary security credentials and permission sets enhances security by minimizing the risk of long-term credential exposure, enabling controlled, time-bound access to AWS resources.
AWS Identity Center, you can efficiently manage workload and workforce identities by managing user access within a single AWS account or centralising identity management across multiple accounts for seamless and secure authentication and authorisation.
Access Analyzer helps you identify and remove unused or external access to resources, and create least privilege policies to limit access to only what's needed

For a deep dive into centralised permission management at scale with AWS IAM Identity Center, check out this post.

Detection and Response

AWS provides comprehensive detection and response capabilities through services such as Amazon GuardDuty as an intrusion detection system (IDS) for threat detection and AWS Security Hub for a single view of all security alerts. Amazon GuardDuty continuously analyses network traffic and uses multiple workload and account logs to detect threats using anomaly detection, AI, ML, and threat intelligence. The service integrates directly into the AWS landscape and automated response mechanisms can be used to automatically remediate the threat.

Data Protection

To achieve data protection, you need to focus on three important pillars: Data classification, protection of data at rest and protection of data in transit.

Data Classification helps you categorize your data based on its importance and sensitivity, providing a foundation for determining the appropriate protection and retention controls. AWS services like Amazon Macie can help automate this process.
Protecting Data at Rest is a must. In AWS, you can use AWS Key Management Service (KMS) to encrypt data for your workloads and apply least privileges access policies to your encryption Keys. Additionally, you can bring your own key material using a Hardware Security Module (HSM) and integrate it with KMS through the External Key Store (XKS) feature, enabling seamless encryption across AWS services.
Protecting Data in Transit is crucial to ensure secure communication between systems. AWS provides a Service called AWS Certificate Manager (ACM) to manage SSL/TLS certificates, making it easier to enforce secure connections across your workloads.

For handling confidential data, AWS provides the Nitro system – a specialized hardware security solution that ensures complete isolation of virtual machines.

Compliance

AWS helps you to comply with the legal requirements and compliance for your data by providing you tools and services that help you to comply with various laws and regulations. AWS itself is already certified and fulfils numerous standards such as ISO 27001 or C5 (BSI requirements for cloud providers). So that you can monitor the compliance status of your resources, there are services such as AWS Config that monitor resource changes and check them according to applicable regulations. You can also use automated audit processes with the AWS Audit Manager to check how well your regulations are being met.

Network and Application Protection

Network and application protection in AWS is essential for securing your infrastructure and services against unauthorized access and attacks.

AWS offers services to protect applications from common internet attacks. These include DDoS attacks, which can compromise security and availability. If you want to learn more about web application firewalls, check out our Web application Firewalls at Scale Service.

Amazon VPC lets you create isolated network environments to manage traffic with security groups and NACLs. AWS Security Groups function as virtual firewalls for virtual instances like EC2, controlling inbound and outbound traffic.

Conclusion

Robust security is essential for every organization handling data, especially sensitive information, to protect against cyberattacks and ensure compliance. AWS offers various services to safeguard data and manage access effectively. Security helps you build trust, stay compliant and keep your digital assets safe.

Confidence in the cloud with data sovereignty

David Krohn — Sat, 04 Jan 2025 09:59:35 +0000

Data sovereignty refers to the requirement that specific types of data – including intellectual property, financial records, and personal information – must be collected, stored, and processed within defined geographic boundaries, such as within the European Union (EU). Whether your application stores credit card information on an e-commerce website or backs up the electronic patient record (EPR), data sovereignty must be ensured so that this user data is subject to the legal framework of the country or other legal provisions in which these users are citizens. Especially for companies that are subject to certain regulations, understanding and implementing data sovereignty is of crucial importance.

Almost every country has a data protection law that protects the personal data collected from its citizens in some way. This includes, for example, the General Data Protection Regulation (GDPR), and for certain industries there are additional regulations such as DORA (Digital Operational Resilience Act) for the financial sector.

In simple terms, data sovereignty is about WHO - WHERE and HOW.

WHERE is your data stored?
WHO can access and use your data?
HOW is your data controlled by regulations (which laws and regulations apply to your data)?

Data sovereignty in the cloud

It is important to choose a cloud service that enables the storage, processing and management of data according to specific requirements. For example, it can be helpful to use a service that can restrict storage to specific regions or data centres to enable a higher level of data localisation and storage. In addition, it is crucial that the data is encrypted, for instance with an own dedicated key, and that high standards of access rights can be maintained in accordance with all legal requirements of your organization.

Let's go back to the WHERE, WHO and HOW and discover how AWS can help.

WHERE?

AWS gives you access to a global infrastructure with data centres in many countries. This enables you to choose where your data is stored and processed. To ensure that your data does not leave the selected region, you can restrict access to certain regions. Additionally you can even use AWS Outpost as a fully managed solution that allows you to run AWS services on-premises.

WHO?

Cloud storage services like Amazon S3 offer detailed control over data access permissions. For handling confidential data, AWS provides the Nitro system – a specialized hardware security solution that ensures complete isolation of virtual machines. This system prevents access by operators and separates each virtual machine from both the hypervisor and other machines, creating strong hardware-level protection for your data.

HOW?

AWS helps you to comply with the legal requirements for your data by providing you tools and services that help you to comply with various laws and regulations. AWS itself is already certified and fulfils numerous standards such as ISO 27001 or C5 (BSI requirements for cloud providers). So that you can monitor the compliance status of your resources, there are services such as AWS Config that monitor resource changes and check them according to applicable regulations. You can also use automated audit processes with the AWS Audit Manager to check how well your regulations are being met.

European Sovereign Cloud

AWS is investing EUR 7.8 billion in a special European Sovereign Cloud to support data sovereignty by the end of 2025.The European Sovereign Cloud is an isolated instance of the AWS cloud operated by an independent and separate company. It provides a robust solution for organisations with strict data management requirements, especially in critical sectors.It also ensures that all employees operating this cloud or supporting customers are EU citizens and located in the EU.

Conclusion

Data sovereignty is important for digital sovereignty - it focuses on the legal and jurisdictional aspects of data management. Choosing cloud providers such as AWS and implementing strategic practices will help you maintain control over your data, comply with relevant laws and regulations, and control where data is processed.

AWS IAM Identity Center Permission Management at Scale Part 3

David Krohn — Fri, 26 Apr 2024 14:04:08 +0000

Identity management is the easiest when you can manage identities in one place and use them across accounts and applications. AWS IAM Identity Center streamlines identity management by enabling you to connect to your identity provider (IdP), such as Active Directory, and use the IdP's identity information for access and collaboration within applications. You can do this, for example, by using an AD Connector and connecting it to your on-premises or Azure AD. An AD Connector is a directory gateway that allows you to route directory requests to your on-premises Microsoft Active Directory without storing information in the cloud cache.

If you have a large Active Directory with several thousand groups and users, you may not want to explicitly select the identities you want to synchronise with AWS Identity Center.

In this blog post, we will show you a solution that allows you to specify different prefixes for Active Directory groups that will be automatically synced to your AWS Identity Center.

The solution workflow includes the following steps:

Synchronization of Active Directory Groups
Automated Documentation & Notification

Prerequisite:

Identity Center must be enabled in your Organization
Connection to local or Azure Active Directory
Active Directory User who has read access

Solution overview

The following architecture shows the solution of the automated sync of Active Directory Groups to AWS Identity Center .

🔗 Here you can find the Solution on Github

In our CDK based project several resources will be provisioned due deployment.

The Stack contains:

Amazon EventBridge rule that runs on a schedule
CDK Sops Secrets Construct
- Secret which holds the Active Directory Credentials
Lambda Function to create the Synchronisation Filter in AWS IAM Identity Center
all necessary IAM Roles and Permissions for the Lambda Function

Synchronization of Active Directory Groups

With Active Directory group synchronisation, you use IAM Identity Center to assign users and groups from Active Directory access to AWS accounts and to AWS-managed or customer-managed applications. All Active Directory Groups with specified prefixes will be automatically synchronised to your AWS IAM Identity Center.

How the automation with the lambda function works

The Sync LambdaFunction is triggered by Eventbridge Scheduled event to ensure that the Groups you want sync from your active directory are always uptodate in your AWS Identity Center.

The workflow of our Sync LambdaFunction is the following:

Retrieve Active Directory credentials from AWS Secrets Manager
Log on to Active Directory using LDAP(s) using the credentials just retrieved
Search and retrieve Active Directory group names based on specified prefixes
Retrieve current Filters of Groups which are sync scope from AWS Identity Center
Add missing Active Directory groups as synchronisation filters in AWS Identity Center
1. ℹ Synchronisation to AWS Identity Center will start after a short amount of time.
Generating of automated documentation
Sending notification to MS Teams

Deployment

The deployment of the solution is done via a cdk stack which is part of the solution's repository. All the required Parameters will be configured in a typescript file - an example file is also included in the repository.

What you have to configure:

Property	Description
Produkt	will be used to generate StackName
Stage	will be used to generate StackName
Prefix	will be used to generate StackName
s3_DOKUBUCKET	Name of the S3 Bucket for the generated documentation
WebhookUrlTeams	URL of your MS Teams Webhook
DocuWebsite	Link to the Website where the generated HTML snippet has been published used for a button in the Team Nofitication
LambdaSchedule	Duration how often your Lambda should be trigger from Eventbridge. Rates may be defined with any unit of time, but when converted into minutes, the duration must be a positive whole number of minutes.
rootCaCertificateString*	the Root CA Certificate in PEM format which issues the server certificates Active Directory
vpcId	Id of your VPC where the lambda will be deployed to
subnetIds	Ids of subnets where the lambda will be deployed to
ad_DomainName	Name of your Active Directory
BasePath	Path in your Active Directory where the Lambda can find the Users you want to Sync
GroupPrefixes	Array of prefixes of groups you want to synchronise into your AWS Identity Center
SecretFile	Path to your Sops secrets file, which should contain the Active Directory user credentials.
Url	LDAP Url to your Active Directory. The LDAP URL format is ldap:// hostname
Port	Port of your LDAP - LDAP default is 389, LDAPs = 636
IdentityStoreId	Identity Store Id of your AWS Identity Center
Endpoint	AWS hidden API Endpoint- should be identity-sync..amazonaws.com

Automated Documentation & Notification

After each execution of the Synchronisation Lambda, we aim to notify our team about the actions taken during the previous run. Therefore, we have implemented a Teams notification that includes a status update and a link to an automatically generated dashboard.

ℹ️ Notifications are only sent when groups are added or deleted, or when an automation error occurs, to avoid a flood of notifications.

The following screenshot illustrates an example of a Teams notification.

Documentation

Our Active Directory Synchronization Status Dashboard is a simple HTML file which will be generated trough a Lambda Function, saved in S3 and will be distributed trough a CloudFront. You can integrate this Dashboards in your Confluence or any other internal Wiki. This Dashboard is secured via CloudFormation Function - additionally you can also add a Firewall to restrict the access to an specific CIDR or Geographic region and prevent access from third parties. The screenshot below provides an example of a dashboard.

Conclusion

In this blog post, we showed you how to improve your security posture by automatically and regularly synchronising Active Directory groups that match a specific pattern with AWS Identity Center. This simplifies access management and increases security by automatically revoking access in AWS Identity Center when group objects are deleted or created in Active Directory. Furthermore, the automatically generated documentation facilitates an overview of the synchronised group objects.

🔗 Here you can find the Solution on Github

AI TRiSM - Building trust in artificial intelligence

David Krohn — Wed, 14 Feb 2024 12:06:32 +0000

Artificial intelligence (AI) is an important part of our daily lives. Chatbots, coding assistants, and even self-driving cars are becoming popular. According to a report by the McKinsey Global Institute, generative AI is expected to contribute up to $4.4 trillion to the global economy annually. However it's often challenging for people to embrace change, especially when it involves complicated topics like artificial intelligence (AI). To many individuals, AI can seem like a complex puzzle. It incorporates complicated algorithms and learning models, resulting in confusion for those without expert knowledge when trying to comprehend the reasoning behind certain decisions. Building AI systems requires a vast amount of data and clever algorithms in order to detect patterns and predict future trends. Even the creators of these systems sometimes struggle to fully understand their operations. This lack of transparency can pose issues. It can make it difficult to spot and rectify errors or biased results, leading to the perception that AI systems are untrustworthy due to their hard-to-grasp nature.

Companies not handling AI risk rightly can face more negative results, like AI leaks. Sometimes, models may not function as planned, causing safety and privacy troubles and financial or reputational damage. If done badly, AI can lead the companies to make bad business choices.

AI system obscurity presents a big challenge. This has led to increased attention on creating a solution called AI Trust, Risk, and Security Management, or TRiSM. It's a thorough way to address important parts of creating and launching an AI system. It plays a big part in lessening possible problems like security risks, data breaches, and other risks that might affect how well AI systems work and if they're fair and honest.

What is TRiSM?
Gartner defined the AI TRiSM framework to support governance, trustworthiness, fairness, reliability, robustness, effectiveness, and privacy of AI models. The AI TRiSM framework is built around four key pillars:

Explainability
Explainability refers to the ability of a model to provide clear and human-understandable explanations of its decision-making process. This is critical for building trust with users and ensuring fair and ethical outcomes.

ModelOps
Model Operations (short ModelOps) includes the management of the entire model lifecycle, from data preparation and ingestion to model training, deployment and ongoing monitoring. This ensures that the model performs as expected, identifies and resolves problems or biases, and continually optimises its performance.

AI security
AI security is a critical concern for organisations, as evidenced by a growing class of adversarial machine learning attacks. The data and models are often extremely valuable, and robust security measures must be implemented to ensure their integrity and confidentiality.

Privacy and Ethics
Privacy and ethics ensure that data is collected and used in a responsible and ethical manner, respecting the privacy of individuals. Organisations must also consider the downstream socio-economic impacts of using AI in their applications, and work to mitigate negative impacts.

What are Key Actions for AI TRiSM?

Improve your AI models with making them explainable or interpretable (sometimes you can leverage AWS solution or OpenSource)
Setup a dedicated Competency Circle to manage AI TRiSM efforts, broaden perspectives and give you strategic advice
Incorporate risk management into model operations. Use solutions that ensure both model and data integrity.
Make sure that your AI is designed to maintain accountability.
Implement data protection solutions (Encryption is everyone's Job) used by AI models and prepare the use of different data protection methods for different use cases and their components.

Conclusion
⁤TRiSM is essential in our current time for companies to ensure the responsible and safe use of AI. ⁤⁤The proactive approach of this framework ensures the potential of AI is realised while mitigating risks, maintaining security and ethical standards. ⁤

Ensuring Compliance with Custom AWS Config Rules

David Krohn — Mon, 14 Aug 2023 07:23:13 +0000

In today's data-driven world, ensuring compliance with industry regulations and internal policies has become a top priority for organisations. While many organisations implement standard compliance measures, custom Config Rules offer a more tailored approach to meet specific requirements. In this blog post we will discuss the importance of custom Config Rule for compliance and provide examples of custom Config Rules using AWS CloudFormation Guard 2.0 and Lambda.

Why Custom Config Rules are Important for Compliance

Basic security and governance are provided by standard compliance measures, protecting organizations with a minimum level of security. But businesses operate in different environments, each with unique characteristics. One-size-fits-all compliance frameworks may not meet specific requirements, possibly leaving vulnerabilities unattended. This is where personalised Config Rules come into play, offering bespoke solutions that match an organisation's unique requirements.

Accurate Management of Risks: Personalised Config Rules allow organisations to precisely address potential risks and vulnerabilities that are specific to their operations. When adjusting compliance efforts to their specific context, businesses can effectively reduce threats and improve their security status.
Alignment to standards in the industry: Many fields have unique rules and standards that extend beyond standard compliance frameworks. Organisations can adjust custom Config Rules to satisfy these industry-specific demands without violating complying to broader regulations.
Adjusting to Change: The business world changes constantly, and as organisations develop, so do the risks they face. Tailored setup regulations can be modified and improved with ease to incorporate changes in technology, operations, and regulations, thereby guaranteeing consistent enforcement in a flexible setting.

AWS CloudFormation Guard 2.0: Empowering Custom Config Rules

AWS CloudFormation Guard 2.0 is a powerful tool that helps organizations adhere to compliance requirements with custom Config Rules. This open-source project provides a language that facilitates developers and security teams to establish and enforce compliance rules for infrastructure-as-code. With a user-friendly syntax, CloudFormation Guard 2.0 enables organizations to efficiently and clearly define their compliance requirements.

Examples of Custom Config Rules with CloudFormation Guard 2.0

GuardDuty Settings

The following CloudFormation snippet for a custom Config Rule using AWS CloudFormation Guard 2.0, checks whether GuardDuty has S3 protection enabled, Kubernetes protection enabled and Findings are published every 15 minutes.

  CustomGuardDutySettings:

    Type: AWS::Config::ConfigRule

    Properties: 

      ConfigRuleName: GuardDutySettings

      Description: Compliant if GuardDuty has S3 protection enabled, Kubernetes protection enabled AND Findings are published every 15 minutes.

      Scope:

        ComplianceResourceTypes: 

          - "AWS::GuardDuty::Detector"

      Source:

        Owner: CUSTOM_POLICY

        CustomPolicyDetails:

          EnableDebugLogDelivery: "True"

          PolicyRuntime: guard-2.x.x

          PolicyText: |

            let s3protection = true

            let kubernetesprotection = true

            let publishfrequency = 'FIFTEEN_MINUTES'

            rule compliancecheck when 
                resourceType == "AWS::GuardDuty::Detector" {
                    configuration.DataSources.S3Logs.Enable == %s3protection
                    configuration.DataSources.Kubernetes.AuditLogs.Enable == %kubernetesprotection
                    configuration.FindingPublishingFrequency == %publishfrequency
                }
    SourceDetails: 
    - 
      EventSource: "aws.config"
      MessageType: "ConfigurationItemChangeNotification"

Apache Kafka (MSK) Broker Encryption at rest

The following CloudFormation snippet for a custom Config Rule using AWS CloudFormation Guard 2.0, creates a Config Rule to check whether Amazon Managed Streaming for Apache Kafka (MSK) data volume is encrypted.

  CustomMskBrokerEncryptionAtRest:

    Type: AWS::Config::ConfigRule

    Properties:

      ConfigRuleName: CUSTOM-MSK-BROKER-ENCRYPTION-AT-REST

      Description: Checks whether MSK has at the broker volume encryption at rest active

      Scope:

        ComplianceResourceTypes:

          - "AWS::MSK::Cluster"

      Source:

        Owner: "CUSTOM_POLICY"

        CustomPolicyDetails:

          EnableDebugLogDelivery: false

          PolicyRuntime: guard-2.x.x

          PolicyText:

            !Sub |

            rule check_resource_type {

                resourceType == "AWS::MSK::Cluster"

            }

            rule check_broker_encryption_at_rest when check_resource_type {

              let kmsKey = configuration.EncryptionInfo.EncryptionAtRest.DataVolumeKMSKeyId

              %kmsKey == /arn:aws:kms:\w+(?:-\w+)+:\d{12}:key\/(?:[-a-z0-9]+)/

            }

        SourceDetails:

          -

            EventSource: aws.config

            MessageType: ConfigurationItemChangeNotification

Limitations of AWS CloudFormation Guard 2.0 when using it for custom Config Rules:

Simplicity: CloudFormation Guard 2.0 has been developed to be simple and straightforward when defining compliance rules. However, this simplicity can be a disadvantage when handling complex compliance requirements that include complicated logic, multiple conditions or dependencies on resources.
Flexibility of rules: The flexibility of the rule definitions in CloudFormation Guard 2.0 is somewhat limited when compared to more comprehensive programming languages. If you need to implement complex calculations or evaluations for advanced rules, you may find the syntax of CloudFormation Guard 2.0 restrictive.
Dynamic data retrieval is not possible: CloudFormation Guard 2.0 can't fetch data from external sources or make API calls to gather information for rule evaluations. This limitation makes it difficult to create rules that depend on real-time data from APIs or external sources, especially when compliance mandates it.
AWS Config Limitation: It should be noted that while AWS CloudFormation Guard offers support for array and regex functionalities, AWS Config may currently lack the capability to evaluate all of them.

Example of Custom Config Rule using Lambda

The following CloudFormation snippet for a custom Config Rule backed by AWS Lambda, checks whether any resource tag contains an phone number or an email.

  DsgvoConfigRuleFunction:

    Type: AWS::Lambda::Function

    Properties:

      FunctionName:

        Fn::Sub: DsgvoConfigRuleFunction-${AWS::Region}

      Description: Checks for Email and Phone in Ressource Tags

      Code:

        ZipFile: |-

          import json

          import boto3

          import re

          import os

      def find_violation(current_tags):
          violation = ""
          for tag in current_tags:
              if tag['value'] != "":
                  phoneregex = r"(^\+49+[0-9]*|^49+[0-9]*|^01+[0-9]*|^\+01+[0-9]*)"
                  if re.match(phoneregex, tag['value']):
                      violation += f"- found phone number in Tag: {tag['key']} "
                  emailregex = r"^[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+$"
                  if re.match(emailregex, tag['value']):
                      violation += f" - found E-mail address in Tag: {tag['key']} "
          return  violation

      def evaluate_compliance(configuration_item):
          if configuration_item["configurationItemStatus"] == "ResourceDeleted" or os.environ['AWS_DEFAULT_REGION'] != configuration_item["awsRegion"]:
              return {
                  "compliance_type": "NOT_APPLICABLE",
                  "annotation": "The configurationItem was deleted and therefore cannot be validated."
              }
          violation = None
          current_tags = configuration_item["configuration"].get("tags")
          if current_tags != None or current_tags == "":
              violation = find_violation(current_tags)
          if violation:
              return {
                  "compliance_type": "NON_COMPLIANT",
                  "annotation": violation[:255]
              }

          return {
              "compliance_type": "COMPLIANT",
              "annotation": "The tags of this resource are DSGVO compliant."
          }

      def handler(event, context):
          invoking_event = json.loads(event["invokingEvent"])
          configuration_item = invoking_event["configurationItem"]
          result_token = "No token found."
          if "resultToken" in event:
              result_token = event["resultToken"]


          evaluation = evaluate_compliance(configuration_item)

          config = boto3.client("config")
          config.put_evaluations(
              Evaluations=[
                  {
                      "ComplianceResourceType":
                          configuration_item["resourceType"],
                      "ComplianceResourceId":
                          configuration_item["resourceId"],
                      "ComplianceType":
                          evaluation["compliance_type"],
                      "Annotation":
                          evaluation["annotation"],
                      "OrderingTimestamp":
                          configuration_item["configurationItemCaptureTime"]
                  },
              ],
              ResultToken=result_token)

  Handler: index.handler
  Runtime: python3.11
  Role:
    Fn::GetAtt:
      - DsgvoConfigRuleRole
      - Arn
  Timeout: 60



CustomDsgvoTags:

    Type: AWS::Config::ConfigRule

    Properties:

      ConfigRuleName: CUSTOM-DSGVO-TAGS

      Description: Checks whether no sensitive data from employees are added into Tags are set

      Source:

        Owner: "CUSTOM_LAMBDA"

        SourceDetails:

          -

            EventSource: aws.config

            MessageType: ConfigurationItemChangeNotification

        SourceIdentifier: !Sub 'arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:DsgvoConfigRuleFunction-${AWS::Region}'

Conclusion

In the time of data-based operations, it's important to have both compliance and customisation working together effectively. Although standard compliance measures are important, it is better for companies to have custom Config Rules that suit their unique needs and mitigate specific risks. AWS CloudFormation Guard 2.0 shines as a tool that empowers organizations to implement and enforce custom compliance rules seamlessly. However if you want to implement more complex checks you need to use Custom Config Rules backed by Lambda. Organizations can confidently move towards a secure and compliant future in the data-driven world by using custom Config Rules.

Deploying CloudFormation StackSets with AWS CDK

David Krohn — Thu, 20 Jul 2023 12:55:20 +0000

AWS Cloud Development Kit (CDK) is a powerful framework that allows developers to define cloud infrastructure as code using familiar programming languages. With CDK, you can easily provision and manage AWS resources in a consistent and automated manner. In this blog post, we'll walk you through the process of creating a StackSet using AWS CDK.

Before we dive into the details, let's take a quick look at what a StackSet is and how it can help you manage your AWS infrastructure.

StackSets are containers for CloudFormation stacks that enable simultaneous creation, update, and deletion across multiple AWS accounts and regions. With StacksSets, you can ensure that all environments are consistent and compliant with the policies you have in place.

The native support for StackSet in CDK is somewhat rudimentary, due to the fact that it is more common in CDK to use CDK pipelines to roll out stacks to multiple accounts and regions.

Therefore, in order to use StackSet in CDK, a few things need to be considered. In this blogpost, we will show steps on how to deploy StackSets via CDK.

Let’s see how this works.

Prerequisites

To follow this tutorial, make sure you have the following prerequisites

An AWS account with appropriate permissions to create StackSets and associated resources.
AWS Command Line Interface (CLI)
AWS Cloud Development Kit (AWS CDK)
Basic knowledge of TypeScript

Bootstrap AWS Account

Bootstrapping is the process of providing resources for the AWS CDK before you can deploy AWS CDK applications in an AWS environment. Normally you could use the default

cdk bootstrap aws://ACCOUNT-NUMBER-1/REGION-1 command. However, we need to customise the template to make the assets available to the entire AWS Organization, as we want to use this CDK environment to deploy StackSets. To get the latest version of the CDK bootstrap template, do the following:

cdk bootstrap --show-template > bootstrap-template.yaml

ℹ️ The CDK boostrap template contains an S3 bucket for files and an ECR repository for container images. It also creates few IAM roles.

After that, we need to modify two resources in this template. The s3 bucket for Assets and the KMS Key that will be used to encrypt the assets.

Add the following Part to the Parameters Section of the template:

  PrincipalOrgID:
    Description: >-
      The identifier of your AWS organization. Used in the KMS key policy and S3 bucket to
      share the key with all accounts under your organization
    Type: String

We will reference this Parameter in the Resource section.

Add this CodeSnippet to the FileAssetsBucketEncryptionKey Resource in to the Key Policy Section. This will

        KeyPolicy:
        Statement:
          - Action:
          ....
          - Action:
              - kms:Decrypt
              - kms:DescribeKey
            Effect: Allow
            Principal:
              AWS: "*"
            Resource: "*"
            Condition:
              StringEquals:
                kms:ViaService:
                  - Fn::Sub: s3.${AWS::Region}.amazonaws.com
              ForAnyValue:StringLike:
                aws:PrincipalOrgID:
                - !Ref PrincipalOrgID

Extend the PolicyDocument of the StagingBucketPolicy with the following CodeSnippet. This will ensure that all Accounts of the Organization get access to the objects in the Asset Bucket.

      PolicyDocument:
        Id: AccessControl
        Version: "2012-10-17"
        Statement:
        ...
          - Sid: ''
            Effect: Allow
            Principal: '*'
            Action:
            - s3:Get*

            Resource: !Sub '${StagingBucket.Arn}/*'
            Condition:
              ForAnyValue:StringLike:
                aws:PrincipalOrgID:
                - !Ref PrincipalOrgID
          - Sid: ''
            Effect: Allow
            Principal: '*'
            Action: s3:ListBucket
            Resource: !Sub '${StagingBucket.Arn}'
            Condition:
              ForAnyValue:StringLike:
                aws:PrincipalOrgID:
                - !Ref PrincipalOrgID

After all the adjustments we need to deploy the template.

aws cloudformation create-stack \
  --stack-name CDKToolkit \
  --template-body file://bootstrap-template.yaml

Set Up Your CDK Project

After bootstrapping our Account we are ready to Initialize a new CDK project. We will do that in a new directory. The initialisation creates a new CDK project structure with a sample lib/stackset-cdk-demo-stack.ts file, which we will modify to create our StackSet.

#Create new directory
mkdir stackset-cdk-demo
cd stackset-cdk-demo

#Init new CDK Project
cdk init app --language typescript

Define the StackSet

Open lib/stackset-cdk-demo-stack.ts and remove the example stack definition. We'll define our stackset instead:

import * as cdk from 'aws-cdk-lib';
import * as servicecatalog from 'aws-cdk-lib/aws-servicecatalog';
import { StackSetTemplate } from "./stackSetTemplate";
import * as s3 from "aws-cdk-lib/aws-s3";

export class StackSetCdkDemoStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);


    const stackSetTemplateSandbox = new StackSetTemplate(this, "stacksettemplate", {
      Config: props.stacksetProps,
      assetBucket: s3.Bucket.fromBucketName(this, "assetbucket", "myCDKAssetBucket")
    });

    new cdk.CfnStackSet(this, "TESTSTACKSET", {
      permissionModel: "SELF_MANAGED",
      stackSetName: "TEST-STACKSET",
      description:
        "example of StackSet with CDK",
      capabilities: ["CAPABILITY_NAMED_IAM"],
      templateUrl: servicecatalog.CloudFormationTemplate.fromProductStack(stackSetTemplateSandbox).bind(this).httpUrl,
      operationPreferences: {
        failureToleranceCount: 30,
        maxConcurrentCount: 30,
      }
    });
  }
}

⚠️ Ensure to adjust the myCDKAssetBucket to your AWS Account Assets Bucket.

🚨 Using the servicecatalog ProductStack construct we get rid of the PseudoParameter for the assets bucket in lambdas in the template.

❌ Without using the servicatalog ProductStack:

!sub cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}

✅ Using the servicatalog ProductStack:

cdk-hnb659fds-assets-123456789012-eu-central-1

Undefined

This workaround will ensure that all AWS accounts can access the assets in our CDK app.

Create StackSet template

In the lib directory create a new file called stackSetTemplate.ts and add the following code to the file:

import * as cdk from "aws-cdk-lib";

import { Construct } from "constructs";

import * as servicecatalog from "aws-cdk-lib/aws-servicecatalog";

import { NodejsFunction } from "aws-cdk-lib/aws-lambda-nodejs";

import * as lambda from "aws-cdk-lib/aws-lambda";

//Standardstackdefinition

export class StackSetTemplate extends servicecatalog.ProductStack {

// export class CdkStack extends cdk.Stack {

  constructor(scope: Construct, id: string, props: cdk.StackProps) {

    super(scope, id, props);

/**
 * Dummy Node JS Lambda Function
 */
  const lambdaFunction = new NodejsFunction(this, "testFunction", {
    memorySize: 128,
    timeout: cdk.Duration.seconds(60),
    runtime: lambda.Runtime.NODEJS_18_X,
    handler: "handler",
    entry: path.join(__dirname, "lambda/index.ts"),
    bundling: {
      minify: true,
      externalModules: ["aws-sdk"]
    }
  });
  lambdaFunction.applyRemovalPolicy(cdk.RemovalPolicy.RETAIN);
}



}

}

Example Lambda Code

Create a new directory lambda in the lib directory. In the new lambda directory create a new file called index.ts and add the following code:

import { Handler } from 'aws-lambda';

export const handler: Handler = async (event, context) => {

    console.log('EVENT: \n' + JSON.stringify(event, null, 2));

    return context.logStreamName;

};

Deploy the StackSet

Run cdk deploy

in a terminal to deploy the StackSet and associated CloudFormation stacks.

ℹ️ CDK will ask you to confirm that you want to deploy the changes. Type y and press Enter to continue.

Conclusion

In this blog post, we've explored how to create a StackSet using AWS CDK. We also learned how to share the CDK Assets to the whole AWS Organization. StackSets are an essential tool for managing infrastructure at scale across multiple AWS accounts and regions. Using CDK, you can easily define and deploy complex cloud infrastructure as code, and leverage the full power of AWS CloudFormation to ensure consistency, compliance, and efficiency across your organisation's cloud resources.

Enterprise-scaled Self-Healing StackSets

David Krohn — Mon, 29 May 2023 14:40:06 +0000

With more than 5 million articles from over 7,000 brands, OTTO is one of the leading German online shopping platforms. In the future, it will open up to even more brands and partners as part of its transformation. OTTO is part of the internationally active Otto Group, with headquarters in Hamburg, and employs 6,100 people throughout Germany. In the 2020/21 financial year, OTTO generated revenues of 4.5 billion euros.

At OTTO, we faced several challenges to operate AWS CloudFormation StackSets at Scale. We must govern several hundred AWS accounts for our product teams, all while balancing the need for agility and control.

At this scale, operations can take a lot of time, because there are multiple operational tasks that we need to do when AWS accounts are leaving the AWS Organization or Teams are nuking the AWS account, StackSets Instances get drifted, because not all required resources for compliance can be secured ( SCP Limitations ), existing AWS accounts are joining the AWS Organization and all mandatory StackSets needs to be deployed, and manual steps should be reduced to a minimum. Furthermore, there is no feature from the Service itself to gain an overview of the status of drifted Instances and the general health of your StackSet health and compliance.

The cloud competence center at OTTO IT, also known as the Governance at Scale (GAS) team, developed a solution for self-healing on StackSets, that is integrated into the OTTO tooling ecosystem with Confluence and Microsoft Teams.

OTTO worked with globaldatanet to set up its Landing Zone. globaldatanet is an award-winning AWS Advanced Consulting Partner and longtime Cloud Solution Provider for OTTO, supporting the team in cloud security and GAS. Their focus on building cloud-native solutions using Serverless supported over 100 companies within 5 years to develop and innovate products and services in the cloud.

In this post, we’ll demonstrate how to implement fully automated enterprise-scaled self-healing on StackSets using AWS StepFunctions and create a Dashboard to get an overview of your StackSet health and compliance and reduce operational time.

The solution workflow includes the following steps:

The tagging concept for StackSets
Automatically create StackSets configuration in SSM Parameter Store
Implementing StepFunction for StackSet Self-Healing

Let’s see how this works.

Prerequisites

The following prerequisites are necessary for following along with the contents of this post:

Two existing AWS Accounts
Few AWS StackSets

Solution overview

The following architecture shows the whole solution of the Self Healing StackSets.

Architecture of fully-automated Self Healing Solution with integration to Confluence.

Tagging concept for StackSets

The solution requires a JSON file in the AWS parameter store, the easiest way is to create it automatically based on the StackSet configurations and the tags assigned there. We'll go into more detail about this in the next section of the Automatically create StackSets configuration Parameter Store article. In the following, we describe which tags we introduced to our StackSet and what we need these tags for.

⚠️ AWS tags do not allow commas in value, so ":" as divider for arrays

Key	Value	Result	Example
antidependson	StackSet Name	antidependson marks stacksets which collide with each other.	MYSTACKSET
dependson	[List of StackSet Names]	List of Stacksets that need to be rolled out before deploying this stackset (e.g. Enable Config before Activate Config Rules). NOTE : Please reduce to only one dependson-stackset for now. Form "chains" for multi-dependencies.	MY-STACKSET1:MYSTACKSET2
mandatory	true or false	The stackset instances must be present on all AWS accounts	true
selfhealing	true or false	StackSet can be healed via Delete & Redeploy (exception e.g. IDP roles) - Parameter Overwrites will be cached.	true
region	[Regions]	List of Regions in which the stackset instances are to be deployed	eu-west-1:eu-central-1:us-east-1

Automatically create StackSets configuration Parameter Store

The automated generation of the Stackset-configuration via JSON inside the ParameterStore is a multi-purpose-utility:

Removing the chore to configure manually a JSON-document
Ensure the Account vending-machines knows what to deploy in which order
Supporting the self-healing StepFunction about the expected setup of the member-accounts

The Lambda responsible for the task is invoked via a Events-Rule:

Every time a Stackset-Operation has been finished with status "succeeded".

This is due the tags on a Stackset are part of the stackset, not Additional items describing a Stackset, therefore a change to the tags always will result in a Stackset-Update-operation.

In terms of computerscience the Lambda is quite interesting, as the primary problem was to build a nonweighted tree based on the "dependson" and "antidependson" tags and then compile an ordered one-dimensional list, like in the good old "travelling salesmen"-problem.

Implementing StepFunction for StackSet Self-Healing

AWS Step Functions is a cloud service that enables you to coordinate the components of distributed applications and microservices using visual workflows. It allows you to build and automate the execution of complex processes and tasks across multiple AWS services, using a visual interface to define and execute your workflows. Since the Self Healing Solutions needs a complex workflow we decided to use Step Functions for this Usecase. Following we will explain you the workflow of the Self Healing.

StepFunction Workflow

Functionality

ƛ Serverless Functions

StackSetInitCleanupLambda: Performs a search to identify StackSet instances of AWS Accounts that are either not present within the AWS Organization or deployed to AWS accounts that are suspended. Once identified, proceed with the deletion of these instances from all associated StackSets.
MandatoryStackSetDeploymentLambda: Search missing StackSets Instances (which are tagged with mandatory = true) and deploy those Instances
StackSetDriftDetectionLambda: Trigger Drift Detection on all StackSets
TriggerDriftStatusLambda: Check if Drift Detection is completed on all StackSets
SearchStackSetInstanceToHealLambda: Searches for drifted StackSet Instances from StackSets which are tagged with Selfhealing = true
StackSetCleanupLambda: Removes unhealthy StackSet Instances and redeploys them. Parameter Overrides will be cached so the new deployed instance will have the same setting as before.
StatusPrepareHTMLLambda: Prepare the HTML output Dashboard for Confluence and Json log file of the current StackSet Healthiness State
TeamsNotificationLambda: Send Teams Notification which summary to notify the GAS Team after each execution

？！Decisions

InitCleanup Complete: Check whether all unnecessary instances have been removed. If not, StepFunction is triggering the StackSetInitCleanupLambda function again.
MandatoryStackSetDeployment Complete: Checks whether all mandatory instances have been deployed. If not, StepFunction is triggering the MandatoryStackSetDeploymentLambda function again.
StackSetDriftDetection Complete: Wait until StackSet Drift Detection has been finished on all StackSets
Healing Complete: Check if all unhealthy Instances are healed otherwise invoke StackSetCleanupLambda again

Limitations

While developing the solution we faced several limitations. Here are our findings and solutions for that.

🚨 StackSets instance operations: Maximum number of stack instances, across all stack sets, that you can run operations on in each Region at the same time, per administrator account is limited to 10.000 operations.

✅ We implemented a counter to count the current StackSets operations which are in progress, in addition we also catching the Exception from CloudFormation and waiting few seconds to try the operation again.
🚨 Parameter Overwrites Caching: Whenever removing a drifted StackSet Instance which has Parameter Overwrite you will lose the individually parameters of the Instance.

✅ Before deleting the drifted StackSet Instance we cache the Parameter Overwrites and deploy the StackSet Instance after successful deletion again with the cached Parameter Overwrites again.
🚨AWS Step Functions Payload size: AWS Step Functions supports payload sizes up to 256KB. For our solution we need more Payloads between the States especially when we want to pass our log or the concurrent Parameter Overwrites per StackSet.

✅ We are storing our states in an S3 bucket to pass the state. At the end of the execution we are deleting the state from S3 to not to influence the next Step Function execution with the wrong state.

Documentation

After each execution of the StackSet Health StepFunction, we aim to notify our GAS team about the actions taken during the previous run. Therefore, we have implemented a Teams notification that includes a status update, a link to the generated dashboard, and a link to the log file.

The following screenshot illustrates an example of a Teams notification. It provides a summary report and directs you to the dashboard and log file for further details.

Dashboard

Our StackSet Health Dashboard is a simple HTML file which will be generated trough a Lambda Function, saved in S3 and will be distributed trough a CloudFrount. You can integrate this Dashboards in your Confluence or any other internal Wiki. This Dashboard is secured via CloudFormation Function - additionally you can also add a Firewall to restrict the access to an specific CIDR or Geographic region and prevent access from third parties. The screenshot below provides an example of the overall StackSet Health status information for an entire AWS Organization.

Conclusion

In this post, we demonstrated a solution to automatically heal AWS CloudFormation StackSets at scale. By implementing this Solution Organisations we reduced manual effort for StackSet cleanup operations by 4 hours per week, improved the overall reliability of our StackSets, increased our compliance in the organisation, and managed to get a daily updated overview for all StackSet Instances using the dashboards. In summary, the self-healing CloudFormation StackSets solution combines automation, monitoring, and self-recovery capabilities to deliver a robust and resilient system for StackSets.

Serverless querying and evaluating of Logs using Athena Part 2

David Krohn — Tue, 21 Mar 2023 09:08:22 +0000

In part one of this blog series, "Serverless Querying and Evaluating of Logs Using Athena," we discussed the importance of logging and monitoring in your organization and how to implement it in a serverless way for the most critical AWS services. We also showed you how to use partition projection in Athena to fully automate partition management. These partition projections are configured through Glue parameters, with each table having an account parameter that includes all AWS Account IDs required as partitions to query for.

However, the question arises of how to keep these parameters automated and up-to-date with the current status of the AWS Organization. New accounts can be created at any time, or accounts can join the AWS Organization via invitation. To avoid manually maintaining all parameters of the Glue tables for these events, we have an automation solution for you.

ℹ️ We actually decided to not remove AWS Account Ids from the partition parameter whenever an AWS Account leaves the AWS Organizations or will be SUSPENDED, because the logs are still valid and maybe needed whenever an request for this account from the past is made in the future.

Architecture Overview

Our automation creates an EventBridge where the Management Account of our AWS Organization has access to perform a "PutEvent" action. This PutEvent action to our central EventBridge will be performed each time a new AWS Account is successfully created in our AWS Organization (CreateAccountResult) or an existing AWS Account accepts the AWS Organization invitation (AcceptHandshake). This event will trigger a Lambda function through our central EventBridge, which will update our existing Glue tables for the new AWS Account ID for the specific Glue table parameter required as input for the AWS Accounts partition. After this automated update, you will be able to query your logs immediately after your new AWS Accounts have been created or your AWS Account has joined your AWS Organization.

Open Source

We have published the complete code of the solution freely on Github as we want to give something back to the community. We hope you find this solution helpful in automating your indexes for Glue / Athena tables! If you have any feedback about the solution, please feel free to reach out to us or open a github issue.

Conclusion

Automating the Glue partitions will help you with your daily work so that no one needs to worry about adding new AWS Accounts to the partition parameters. This ensures that decisions due to missing information in log queries are prevented, and your queries for the information sources are kept up to date.