Forem: Markus Loupeen

Managing multiple Application and Deployments in AWS CDK

Markus Loupeen — Sun, 26 Mar 2023 11:18:43 +0000

In my previous post I talked about how we manage our AWS accounts in CDK, now it is time to see how we manage applications first, and after that, we have a look at the deployments.
Just AwsAccounts class we use a class Application. One Application is one Stack in one Git Repository.
Here is an example for four applications.

interface ApplicationProps {
  name: string;
  repositoryName: string;
  stackName: string;
}

interface ApplicationObject {
  readonly name: string;
  readonly repositoryName: string;
  readonly stackName: string;
}

export default class Application {
  private static readonly applicationsArray: ApplicationObject[] = [];

  /* Account setup */
  public static readonly DNS = new Application({ name: 'dns', repositoryName: 'aws-dns', stackName: 'AwsDnsStack' });
  public static readonly SES = new Application({ name: 'ses', repositoryName: 'aws-ses', stackName: 'AwsSesPipelineStack' });

  /* Project */
  public static readonly COGNITO = new Application({ name: 'cognito', repositoryName: 'projectA-cognito-admin', stackName: 'CognitoAdminStack' });
  public static readonly API_GATEWAY = new Application({ name: 'apigateway', repositoryName: 'projectB-gateway', stackName: 'GatewayStack' });

  readonly name: string;
  readonly repositoryName: string;
  readonly stackName: string;

  private constructor({ name, repositoryName, stackName }: ApplicationProps) {
    this.name = name;
    this.repositoryName = repositoryName;
    this.stackName = stackName;

    Application.applicationsArray.push(this);
  }

  public static get applications(): ApplicationObject[] {
    return [...Application.applicationsArray];
  }
}

Here you can see the four applications; DNS, SES, COGNITO and API_GATEWAY.
Now it is time to put them into our Deployment class.

import Application from '../applicaitons/application';
import { AwsAccount } from '../aws-accounts/awsAccounts';

interface DeploymentProps {
  readonly name: string;
  readonly accounts: AwsAccount[];
  readonly applications: Application[];
}

interface DeploymentObject {
  readonly name: string;
  readonly accounts: AwsAccount[];
  readonly applications: Application[];
}

export default class Deployment {
  private static readonly deploymentsArray: DeploymentObject[] = [];

  public static readonly ACCOUNTS_SETUP = new Deployment({
    name: 'AccountSetup',
    accounts: [AwsAccount.PROJECT_A_TEST, AwsAccount.PROJECT_A_QA, AwsAccount.PROJECT_B_TEST, AwsAccount.PROJECT_B_QA, AwsAccount.PROJECT_B_PROD],
    applications: [Application.DNS, Application.SES],
  });

  public static readonly PROJECT_A = new Deployment({
    name: 'TheGame',
    accounts: [AwsAccount.PROJECT_A_TEST, AwsAccount.PROJECT_A_QA],
    applications: [Application.COGNITO],
  });

  public static readonly PROJECT_B = new Deployment({
    name: 'TheGame',
    accounts: [AwsAccount.PROJECT_B_TEST, AwsAccount.PROJECT_B_QA, AwsAccount.PROJECT_B_PROD],
    applications: [Application.API_GATEWAY],
  });

  readonly name: string;

  readonly accounts: AwsAccount[];

  readonly applications: Application[];

  private constructor({ name, accounts, applications }: DeploymentProps) {
    this.name = name;
    this.accounts = accounts;
    this.applications = applications;

    Deployment.deploymentsArray.push(this);
  }

  public static get deployments(): DeploymentObject[] {
    return [...Deployment.deploymentsArray];
  }
}

Still no strange stuff.
DNS and SES should be deployed to all five accounts, while COGNITO goes to Project A´s TEST and QA accounts and API_GATEWAY will be deployed to Project B´s TEST, QA and PROD accounts.
Now, how does our pipeline for this look like?
This is our stack:

const app = new cdk.App();

new PipelinesStack(app, 'PipelinesSetup', {
  env: { account: AwsAccount.CICD.accountId, region: 'eu-north-1' },
});

export default class PipelinesStack extends Stack {
  constructor(scope: Construct, id: string, props: StackProps) {
    super(scope, id, props);

    const deploymentsSetup = Deployment.ACCOUNT_SETUP;
    const deploymentsProjA = Deployment.PROJECT_A;
    const deploymentsProjB = Deployment.PROJECT_B;

    new DeploymentsStack(this, `Pipeline-${deploymentsSetup.name}-Pipelines`, {
      deployments: deploymentsSetup,
    });

    new DeploymentsStack(this, `Pipeline-${deploymentsGame.name}-Pipelines`, {
      deployments: deploymentsProjA,
    });

    new DeploymentsStack(this, `Pipeline-${deploymentsGame.name}-Pipelines`, {
      deployments: deploymentsProjB,
    });
  }
}

And for the DeployStack, it can be found here:
https://gist.github.com/kronis/de876999f725275f0ab92ed9f0ef5bf6

Managing multiple AWS Accounts and Environments with CDK.

Markus Loupeen — Sun, 26 Feb 2023 22:19:40 +0000

This is just a small post that will be followed up later with pipeline examples.

We do not want our CDK applications to know much about how and where they are deployed, instead it is all defined in a small private npm package that contains all account information.

Here is how we have it, so we can import it in multiple projects and pipelines.

import { Annotations, Stack } from 'aws-cdk-lib';
import { Construct } from 'constructs';

export const DeploymentRegion = {
  EU_NORTH_1: 'eu-north-1',
  EU_CENTRAL_1: 'eu-central-1',
};

export const ApplicationStack = {
  DEV: 'dev',
  TEST: 'test',
  QA: 'qa',
  PRODUCTION: 'production',
};

type ApplicationStackType = (typeof ApplicationStack)[keyof typeof ApplicationStack];
type DeploymentRegionType = (typeof DeploymentRegion)[keyof typeof DeploymentRegion];

export class AwsAccount {
  private static accounts: AwsAccount[] = [];

  // -- Applications --
  // - Production -
  static readonly APP1_PRODUCTION = new AwsAccount('APP1_Production', '100000000001', ApplicationStack.PRODUCTION, [DeploymentRegion.EU_NORTH_1, DeploymentRegion.EU_CENTRAL_1], true);

  // - QA -
  static readonly APP1_QA = new AwsAccount('APP1_QA', '100000000002', ApplicationStack.QA, [DeploymentRegion.EU_NORTH_1, DeploymentRegion.EU_CENTRAL_1], true,);

  // - Test -
  static readonly APP1_TEST = new AwsAccount('APP1_Test', '100000000003', ApplicationStack.TEST);

  // - Dev -
  static readonly APP1_DEV = new AwsAccount('APP1_Dev', '100000000004', ApplicationStack.TEST);
  static readonly APP2_DEV = new AwsAccount('APP2_Dev', '200000000001', ApplicationStack.TEST);

  // -- Aws Core --
  static readonly AUDIT = new AwsAccount('Audit', '900000000010', ApplicationStack.PRODUCTION);
  static readonly LOG_ARCHIVE = new AwsAccount('Log archive', '900000000011', ApplicationStack.PRODUCTION);

  // -- Infrastructure --
  static readonly OPS = new AwsAccount('OPS', '000000000001', ApplicationStack.PRODUCTION);

  private constructor(
    public readonly name: string,
    public readonly accountId: string,
    public readonly stack: ApplicationStackType,
    public readonly deploymentRegions: DeploymentRegionType[] = [DeploymentRegion.EU_NORTH_1],
    public readonly approveBeforeDeploy: boolean = false,
  ) {
    AwsAccount.accounts.push(this);
  }

  static fromScope(scope?: Construct): AwsAccount | undefined {
    if (!scope) {
      return undefined;
    }
    const accountNumber = Stack.of(scope).account;
    const account = this.accounts.find((acc) => acc.accountId === accountNumber);
    if (account) return account;
    Annotations.of(scope).addError(`Couldn't find account [${account}] in AwsAccount array!`);
    return undefined;
  }
}

In next post I will show you how we utilize this in pipelines to make life esier.

Using Private GitHub NPM repositories in CodeBuild

Markus Loupeen — Tue, 13 Sep 2022 19:32:56 +0000

I use GitHub instead of CodeCommit for many reasons, and I also use GitHub Npm Registry, for private packages. It took me some time to understand how I can use them in CodeBuild, so I want to share it with you.

Disclaimer, this is maybe not the best (or even correct way) to do this, but this how I managed to solve the problem for me.

To do this, the secure way I will have to use some magic :) I am using SecretsManager to store a GitHub Personal Token securely, for later using it in the pipelines as an environment variable. We will accomplish this in four easy steps.

1. GitHub - Create token

First, we have to to create a private personal token in Github, you will do that under your settings and Developer Settings. The key needs only access to read:packages.

Save it and make sure to copy the key that is generated for next step.

2. AWS Secrets Manager - Add token

Now we have to add the key to Secrets Manager in AWS.

Review the details and save. It will result in a key looking like this:

3. Configure NPM

This was the tricky part for me, because I want to use npm login when I am developing on my personal machine, and do not want to share the key with other people. so i had to create two .npmrc files, on for local development (.npmrc) and one for CodeBuild (.npmrc_ci).

Make sure to change @loupeen to your account/organisation.

4. Pipeline

Last thing is to configure our pipelines synth and build step.
First we are getting the secret at line 1, after that we are using it in synth (line 10–17) and build (line 20–27) steps, and last thing we have to do is to use the correct .npmrc file, and that is on line 32.

CodePipeline example

And that is it, hoping it makes some sense.

AWS CDK Frontend Pipeline (Cross Account)

Markus Loupeen — Tue, 16 Aug 2022 21:51:10 +0000

For those who just want to see the source and get into it:
https://github.com/loupeen/aws-cdk-frontend-pipeline

Ok, I am going to keep this post short. To be honest, frontend pipelines can be crazy easy or stupidly hard to achieve in AWS. One easy way is Amplify, that is easy, or CodeStar, but if you know your AWS, that is not the best way when it comes to separating your applications in AWS referring to https://aws.amazon.com/organizations/getting-started/best-practices/

Architecture

So I made this little cdk-helper to easily setup and create new pipelines for multi-account setups. It is based on CodeCommit (for now), CodePipeline, S3, Lambda and CloudFront.

Architecture is something like

As you can see there are two CodePipelines, one is for setting up the AWS services in all accounts you want the frontends to be deployed, and the other CodePipeline is just for building the frontend and deploying it to a S3 bucket.

Infrastructure CodePipeline

A pipeline that creates the resources in all specified frontend accounts (eg. Test, QA and Production)

Some of the resources

The main resources the infrastructure CodePipeline creates is following;

S3 Bucket Frontend — the bucket where the frontend pipeline publish the built frontend.

Lambda Copy — a lambda that listens on changes in the S3 Bucket Frontend, and when something is changed (the pipeline publish new version) it copies it to the S3 Bucket CloudFront.

S3 Bucket CloudFront — a bucket where CloudFront serves files from.

Lambda Cache Invalidation — a lambda that listens on changes in S3 Bucket CloudFront and is for invalidating the cache in CloudFront so we can get the updates asap.

CloudFront — So we can have a secure frontend with HTTPS.

Optional resources

If route53 configuration is specified in the configuration.

ACM — Just a SSL certificate for CloudFront.

Route53 Alias — Alias pointing to the CloudFront specified.

But why two buckets? I had some problems with owner of objects in buckets when only using one bucket. Tried to use a lambda to just update the permissions of all files, but it failed. So using two buckets was the fastest way to fix the permissions problem.

Frontend pipeline

A pipeline that just builds the frontend for every environment and publish it to a bucket in that environments account. But why do we build the frontend multiple times? So we can configure specific environment settings per account, for example different AWS Cognito user pools.

Frontend app

For this frontend I have just used Create React App to fast and easily create a frontend.

npx create-react-app my-app — template typescript
And the build pipeline is also using the stack as parameter to REACT_APP_STAGE in the build step. There will be another blogpost about this later on.

Don’t forget
Bootstrap all accounts as well that you wish to deploy your application to.

cdk bootstrap aws://2222222222222/eu-north-1 — trust 111111111111
cdk bootstrap aws://3333333333333/eu-north-1 — trust 111111111111

Testing nested Stacks in AWS CDK 2

Markus Loupeen — Tue, 16 Aug 2022 21:44:00 +0000

I just wanted to mention that testing nested stacks in AWS CDK v2 is easier than you think.

Check out the source over here https://github.com/loupeen/aws-cdk-nested-stacks-test

Just write a small stack wrapper for your nested stack.

You find more code and tests in the GitHub repository.