Forem: Joseph

[Boost]

Joseph — Wed, 26 Feb 2025 13:22:19 +0000

Agentic AI Assistants

Pablo Orozco for ImagineX ・ Feb 25

Intro to Cloud: An API Gateway

Joseph — Mon, 02 Dec 2024 14:38:16 +0000

I tried to think of the best way to describe building an app from scratch in the cloud, but there are so many different ways you can easily get lost in all the options. So I decided I would start by showcasing what is arguably the easiest app you can make from a computing perspective. It offers the least amount of operational overhead, but at the same time, it's the most expensive option on paper.

AWS API Gateway

When I think of simple I think API. That might just be because I'm so used to them at this point, but there are also some direct comparisons to other compute solutions which we'll get into later in other posts as we get into more complex cloud compute ideas. Right now, though, we're going to use AWS API Gateway to deploy the first version of the API we're going to make for the rest of this series. This is a great choice if you want to spend as little time maintaining your gateway as possible. There are no load balancers to deal with. You can optionally put an edge cache (CloudFront) in front of it easily, and it connects to other serverless technology in AWS seamlessly.

Setting up the gateway

For the example I'm building, I'm going to actually use a terraform module that already has all the pieces we need. The first thing we need to do is add the module to our project by editing the cdktf.json file to include the module.

"terraformModules": [
 {
    "name": "apigateway",
    "source": "terraform-aws-modules/apigateway-v2/aws",
    "version": "~> 5.2.0"
 }
],

Later on, we're going to add another module but for now, that's all we need to build the gateway.

Defining the gateway

I'm going to make a serverless folder and put an api.ts to hold all the work we'll need to get the app up and working. Inside our class, we have to import the module we made and then create the gateway.

import { Apigateway } from "../.gen/modules/apigateway";

let gw = new Apigateway(scope, `${name}-gw`, {
  name: `${name}-gw`,
  protocolType: "HTTP",
  corsConfiguration: {
    allowHeaders: [
      "content-type",
      "x-amz-date",
      "authorization",
      "x-api-key",
      "x-amz-security-token",
      "x-amz-user-agent",
    ],
    allowMethods: ["*"],
    allowOrigins: ["*"],
  },

  createDomainName: true,
  createDomainRecords: true,
  createCertificate: true,

  domainName: config.domain,
  hostedZoneName: config.hostedZone,

  routes: {
    "ANY /": {
      integration: {
        uri: lambdaArn,
        payload_format_version: "2.0",
        timeout_milliseconds: 3000,
      },
    },
    $default: {
      integration: {
        uri: lambdaArn,
        payload_format_version: "2.0",
        timeout_milliseconds: 3000,
      },
    },
  },
});

In my case, I already had a hosted zone set up in my AWS account for a subdomain aws.josephbulger.com. I decided to go ahead and add this gateway to that so the hosted zone in this example is aws.josephbulger.com and the API's domain is apigw.examples.how2cloud.aws.josephbulger.com.

You don't have to set this up with a domain. If you don't AWS will generate a random URL to host your gateway instead, and you can use that for making your API calls.

CORS

I have the API's cors configuration set up to allow traffic from anywhere because, with this kind of setup, I'm assuming the front end that would potentially use this is not hosted by the same app. This is a common approach I use for my systems. I don't like to have the API backend be served by the same system that serves the front end. I like to keep them separate.

Routes

Finally, you need to define the routes that the gateway will serve. In my case, I'm going to send all my traffic to a single lambda, which we'll talk about in a second. For now, we just need to know to send all the traffic over to the lambda, and let it figure out what to do with it.

Stages

Additionally, we have to define the default stage. Stages in API Gateway are kind of a weird concept. It's beyond the scope of what I want to cover in this post, but for this purpose let's just say you have to define a default one, so the gateway knows which stage gets the traffic. In our case, we're just going to set it up the same way we set up the route we made.

Lambda

The lambda is the part that makes this computing solution simple and easy. It does this because it's truly serverless. We don't have to worry about running a server, neither as a virtual machine an EC2 instance somewhere, nor as compute sitting on top of an orchestration plan like ECS or k8s. We just create the app, and give it to the lambda. The largest amount of operational overhead with the lambda boils down to dealing with how much CPU and memory to allocate to it, and these days you only specify the memory because the CPU is adjusted based on how much memory you choose. The price we pay for this ease of use is, well, the actual sticker price. All things considered, you can expect to pay anywhere from double to up to 6 times the price of other compute solutions in AWS for the equivalent compute time, but that's on paper. In reality, there are some very compelling reasons to use Lambda, including financially.

Setting up your Lambda

So to get using the lambda we first need to add the module into cdktf the same way we did the gateway.

"terraformModules": [
 {
    "name": "lambda",
    "source": "terraform-aws-modules/lambda/aws",
    "version": "~> 7.14.0"
 }
],

Defining the Lambda

Now we can add the lambda to our API.

let lambda = new Lambda(scope, `${name}-function`, {
  functionName: `${name}-function`,
  createPackage: false,
  packageType: "Image",
  architectures: ["x86_64"],
  imageUri: `${config.ecrUri}:${config.tag}`,
  publish: true,
});

const lambdaArn = lambda.lambdaFunctionArnOutput;

We used the lambdaArn earlier where we defined the gateway. This is actually where it came from, as an output from the lambda module. Now the last thing we need to do to connect the two is set up the lambda's triggers. The trick to them is that the lambda needs to know about the gateway, but we have to define the lambda first. What we do to get around this is just define the triggers after we make both the lambda and the gateway and then add the triggers back into the lambda.

It looks something like this:

let apiExecArn = gw.apiExecutionArnOutput;

let triggers = {
  APIGatewayAny: {
    service: "apigateway",
    source_arn: `${apiExecArn}/*/*`,
  },
};

lambda.allowedTriggers = triggers;

Now that the lambda is connected to the gateway we need to create the app that the lambda is going to run.

Docker

I created a small server in examples that creates a simple go web server built using docker. What's nice about Go is that you can actually deploy it from a scratch image so the overall size of the image is really small (~8 MB). Perfect for running on a lambda.

The docker file looks something like this:

# syntax=docker/dockerfile:1

FROM golang:1.23 AS builder

WORKDIR /app

COPY go.mod go.sum ./
RUN go mod download

COPY *.go ./

ARG TARGETARCH=amd64
ENV GOARCH=$TARGETARCH

ARG TARGETOS=linux
ENV GOOS=$TARGETOS

RUN CGO_ENABLED=0 go build -o /api

FROM scratch

COPY --from=builder /api /api

EXPOSE 8080

ENTRYPOINT ["/api"]

We'll be building it using docker buildx bake so we need to set up a bake file too:

group "default" {
 targets = ["api"]
}

variable "IMAGE_URI" {
 default = "api"
}
variable "IMAGE_TAG" {
 default = "local"
}

target "api" {
 dockerfile = "Dockerfile"
 context = "."
 tags = ["${IMAGE_URI}:${IMAGE_TAG}"]
 platforms = ["linux/amd64"]
 args = {
 TARGETARCH = "amd64"
 TARGETOS = "linux"
 }
 attest = []
 provenance = false
}

Provenance

One issue with deploying to lambdas, at least for now, is that they don't support multi-architecture docker images. So when we deploy our image to ECR we have to disable provenance so docker knows to only build the image in the format we specify with multi-mode turned off.

That's why in our GitHub action we turn provenance off when we do the docker build and push:

- name: Build and push
  uses: docker/bake-action@v5
  with:
    push: true
    workdir: examples/api
    provenance: false
  env:
    IMAGE_URI: [[ECR_URL]]/api
    IMAGE_TAG: ${{ github.ref_name }}

That's It

Now you have a fully functioning API. You can see the one I've deployed at https://apigw.examples.how2cloud.aws.josephbulger.com/. There's also a route you can play around with the changes in the response based on what you give it: https://apigw.examples.how2cloud.aws.josephbulger.com/some/kind/of/trick.

Show me the code

If you want to follow along with the code that I used to build this, or any other part of the "from scratch" series, check out my how to cloud repo on github.

How to Cloud: Fencing

Joseph — Mon, 01 Jul 2024 13:00:00 +0000

Before you can dive into deploying your application you need to create a safe environment for your application to be deployed in. This starts with understanding some basics around making good security decisions, so this next topic is about how to make a good “fence”.

First Things First

Ok, in all fairness, if you are just getting started on your cloud journey, you probably only need one vpc and one account. Keep it simple. When you get to the point where you are going to be responsible for multiple products, then you need to potentially consider different options. Also, the “you” here is not just yourself as a leader of your team, it’s the overall landscape of your organization or company. Is your company planning on migrating multiple platforms or systems to the cloud? Then you need to read on. If they are a start up that’s literally just starting out on a single product idea, you can probably read this later, but it’s still useful to know. Either way, that’s why we need to talk about VPCs.

VPCs

In the cloud world, VPCs act as a fence. They handle a number of key technical issues related to security and network pathing that you need to be able to make quick decisions on. I’m not going to do a deep dive on all the ins and outs of VPCs, but you need to learn how they work so that you can easily choose which way to go in your organization, and with the app you are building. I plan to cover the kinds of decisions that you could make, and why you’d go one way or the other. This decision is kind of in between a hard the right call situation vs preferences. I say that because depending on decisions that your company or organization has made that is outside of your control, you will want to align your decision to the things that were already chosen for you. With that being said, since VPCs act as a fence in a lot of ways, let’s talk about the different ways that you can build that fence in your account.

One VPC per Account

This strategy will make sense for a variety of reasons. You might be leading a single team working on a single product but you own an entire account. What’s the point in having more than one VPC when you are actually only building one thing? There really isn’t a good reason to have more than one. Things get trickier when you are responsible for more than one product or platform. I say it that way because in order to build the right fence you have to start thinking about what you build as something consumed by a customer, or client. The second that your answer to that question becomes, “well I have more than one”, then I argue you have another decision to make.

Multiple Accounts or Multiple VPCs

The bottom line is that every product or platform that you own should have a fence around it. The thing is, however, that while VPCs can and do act as a good fence in a lot of situations, there are some drawbacks to having multiple products and/or platforms in the same account.

Noisy Neighbors

A really good fence isolates you from noisy neighbors. In most cases, VPCs do a pretty good job of this, especially from a networking perspective. They prevent unwanted access through the routing tables you will make coupled with the subnets that are associated to them. That’s all great. What’s not so great is what happens when one system is hogging all the allocated resources you have on your account. Oh yeah, you heard me correctly. Your cloud account has limits. AWS calls them “quotas” now, actually. You can look up the specifics of what your account has and get a report on what they currently are, too. The point, though, is that when your account has reached it’s quota on something you will be denied from allocating more of that thing for your entire account, everywhere. For example, if you have two products on your account, and one of them auto scales during a spike and uses up your quota on auto scaling groups, and then your other product sees a spike as well, guess what, that second product will bottleneck and start crashing. Nothing you can do with your VPC will prevent that from happening. If this is a significant concern for your team, and you operate in an environment, organization, or company with a lot of products and/or platforms in this manner, you will probably be better off strategically operating in a multiple account scenario.

Having said all that, there are some good reasons to own a single account and maintain good boundaries by leveraging multiple VPCs. If your company only sells one product, that’s a good sign you might be better off with one account, even if you have to deploy multiple systems. If you are in small company, odds are it’s better to have one account. What this means is you have to be aware of those service limitations, and how to deal with them correctly. You need to set up monitors to alert you when you are getting close. We have monitors set up to alert us when we get to 80% of our quota on anything. Once we get the alert we send a support ticket to our cloud vendor to increase the quotas. If you have a mature process for dealing with those situations you can deal with noisy neighbors pretty easily.

VPC for each product

If you do operate in a single account with multiple products, then you should deploy each product in their own VPC. This prevents systems from accidentally colliding with one another and becoming noisy neighbors, or teams creating security issues by not collaborating together properly. Each VPC has their own setup of networking and traffic rules, which can be managed by the team that owns the product that lives in it. Having multiple products inside one VPC probably also means you have multiple teams operating in that VPC, and that means they might step on each other’s toes.

VPC for each environment

You should also split up your environments into different VPCs. You do have multiple environments, right? We’re not going to talk through that right now, that could be it’s own blog post. For now, I’m assuming you at least have a non prod and a prod environment, which means you need to have at least one VPC for prod and another for non prod. One common pattern we’ve used it actually making an account for production and another account for all non production systems, so we’re guaranteed that no non production system could ever take down a production system for any reason. That actually segues nicely into the next thing we need to talk about.

Accounts

I guess logically it would make sense to start off talking about accounts and then get into VPCs, but I think this actually makes more sense when you are trying to think about the decisions you need to be making. It becomes a lot more clear why you need multiple accounts or not when you understand the reason why you needed the VPCs to begin with, and how their “fencing” works. The example I gave before is a good one when you want to isolate production from anything going wrong, but you still need to consider the reasons why you might want to have a single account solution vs a multi account solution. Let’s get into it.

Single Account Model

I know I just got finished talking about how important it is to isolate production from non production systems (and it is), but there are some situations where it makes sense to go single account that are worth discussing.

Innovating

If you are innovating and at the very beginning stages of whatever idea you are cooking up, there’s no reason to have multiple accounts, because you have no production ready product anyway. Don’t spend the extra money on a prod account just to prove that you can do it when you are in research mode.

That doesn’t mean you are going to be able to “flip a switch” and go from proof of concept straight to prod and making money. That’s not what I’m saying. What I’m saying is, don’t worry about tomorrow’s problems today when you have no good reason to plan that out yet, worry about that tomorrow when you have a better idea of what you are going to try to do.

Mature DevOps Culture

This is going to seem like complete ends of a spectrum, mostly because it is, but another reason why it’s feasible to only have one account is because your overall DevOps culture is extremely mature. Everyone knows how to execute well, and it’s boring. You’ll know your teams are heading down the right road towards maturity because the same teams that probably can handle operating a single account will be the same ones that beg you to isolate production. It’s a good litmus test. They understand the risks it poses having everything in one account. The decision you will have to make in this situation is largely a financial one at this point. Does it save you a significant amount of cost to merge your accounts into one? Odds are it won’t, because of the way you are billed for the services you use in the cloud. Either way, once you do a cost analysis then you’ll know which way you should go.

Multiple Accounts Model

So if your company didn’t fall into of those the situations above, then you need to go multi account. Deciding to go multi account is one thing, but knowing how to split them up is another. Let’s talk about considerations you need to make.

Separate Environments

We’ve talked at length about the production, non production split, so that should be a given at this point, but let’s also talk about some other varieties here. It’s very common to do a prod/non-prod mix, but something else to consider is how well established are your non production systems in your company? Meaning, do all of your dev systems talk to each other? All of your integration systems work well together,? Are all of your test environments playing nicely together? If you have strong synergy in your company across environments, then it makes sense to actually separate them out by accounts too. Why? because it makes establishing communications among them a lot easier. It makes the permissions model simpler, the networking easier, etc. You don’t have to worry about a dev system from one team accidentally reaching out to integration or testing and blowing something up.

Multi Tenant Platforms

When your company is large enough you are bound to have teams creating systems that are leveraged across multiple products, which invariably means they will be servicing multiple teams. If you happen to engineer as system in that manner, congratulations you’ve just become multi tenant. In this situation multi tenant platforms should have their own accounts. It creates the proper level of isolation from their “customers” so that they can properly operate in a SaaS engagement model. This is particularly important when the technology office is large enough to have multiple organizations inside it. Speaking of…

Multiple Organizations

If your technology office is large enough, you’ll have multiple organizations inside it. To keep things clean, it makes a lot of sense to have each organization own and operate their own accounts. There are exceptions to this (remember what I said about a mature devops culture), but in reality the larger the company is the more sense it makes to break out accounts to match the organizational structure of the technology office.

Allocation

As with all things cloud, once you’ve decided which way you are going to go, you need to script it. So let’s talk about that.

In the example I’m doing, I’m going to build out the repository in a multi account, single vpc per account model. I’m choosing this because I wanted to do something that is slightly more difficult than a “startup situation” but not so complicated that you might get lost in the weeds.

So what does this look like? Inside our iac folder, I’ve created an account folder just for account allocation, so we’ll be making everything for the VPC in there. In the future when we need to make more than one account we can refactor this folder to accomodate. Later the systems we made will be allocated in totally different folders so we’ll have a clear separation from what we needed on the account and what we needed for each system we’re building.

class MyStack extends TerraformStack {
  constructor(scope: Construct, name: string) {
    super(scope, name);

    const roleId = "devOps";

    new AwsProvider(this, "AWS", {
      region: "us-east-1",
    });

    new Vpc(this, "htc-vpc", {
      name: "htc-vpc",
      cidr: "10.0.0.0/16",
      azs: ["us-east-1a", "us-east-1b", "us-east-1c"],
      privateSubnets: ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"],
      publicSubnets: ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"],
      enableNatGateway: true,
    });

    const ecr = new EcrRepository(this, "htc-ecr", {
      name: "htc-ecr",
      imageTagMutability: "MUTABLE",
    });
  }
}

const app = new App();
new MyStack(app, "how-to-cloud");
app.synth();

I’ve left out a number of other things I’ve added to the repo for brevity. None of the other code in the account is related to allocating the VPC so it’s not relevant to this article, but if you want you can always head over there and dive in.

How to Cloud: IaC

Joseph — Mon, 24 Jun 2024 13:00:00 +0000

In my last post I intentionally jumped the gun a little bit when it came to deploying containers into your cloud account. It should be your first priority to get your team to learn containerization, but deploying to the cloud requires allocating container registry infrastructure. That brings us to our next topic which is Infrastructure as Code, and it’s a very close second in priority.

Don’t do it manually

Do not, under any circumstances, try to allocate infrastructure in your account manually through the console. I have tried to think of any situation where it makes sense to just use the console, but there really just isn’t one unless you are truly just toying around. Any product you intend to build and make money on, you can’t afford the windfall of problems you will encounter if you go down that path. Allow me to highlight a couple of these just to give you an idea.

Your team will forget what they were doing

It may seem like it’s faster to just jump into the console and allocate something, especially like a container registry, really quickly and just get it over with. After all, it’s so easy. 6 months later, your team has forgotten how many things they allocated manually, and they don’t remember where to find them, and all of the sudden you are going to have to build it all over again because it’s actually lost in your account and there’s too many other things in your account to find the one thing you need to find. Having your infrastructure written in code completely solves this problem.

Fear of making changes

Even if your team can find the infrastructure they made 6 months ago, they will be terrified to change it, because they won’t be able to easily tell what depends on it, and how it depends on it. Their solution will just be to make another one while keeping the original around for legacy reasons, and your cloud bill cost will suffer the consequences.

Speed of Delivery

The reality is that, if you are using the console, you think you are going faster, but you are actually going at a minimum 4 times slower than the way I do it. I know this from experience. I’ve watched other teams that did it manually because they were convinced they could get it done quicker. They took a month and had nothing to show for it because everything kept breaking, and my team came in and got their entire account working from scratch in 1 week using scripts. Allocating an entire compute cluster stack for my teams today takes less than an hour, from start to finish. I’ve never seen a team allocate a web service on any compute cluster cloud product in the console in that time frame. You think you’re going faster in the console because of an illusion. That illusion is the self gratifying feedback loop that the console gives you from seeing a little bit of progress every 10 minutes or so. It is a complete falsehood.

Knowing what’s necessary and what’s flavor

Before I get into the next steps for IaC, I want to take a second and point something out that’s really important for decision making as a leader. You need to be able to tell the difference between what is the right call vs what’s just your own preference or the team’s preference. What I said earlier about manual vs scripting? That’s the right call. You want to make sure that’s something you handle at your level as a leader. Which tech stack you choose to do the scripting? That’s preference. You want to help facilitate your team into making the best decision here, but I like to leave this area up to my teams, collectively. Empower them to do this for themselves and work together on this decision, not in silos for those of you that manage multiple teams. There are, of course, pros and cons to what you choose, and some situations will call for one solution more than another, but you can be successful using a lot of different tech stacks.

In the realm of IaC you have a few options. I’ll cover the main points how I see them today and explain why I choose what I choose, but that doesn’t mean it’s going to be the right choice for your team. This is the preference part. You need to choose what’s right for your situation, and you need to get good at making that choice quickly.

AWS CLI

The OG for AWS IaC scripting. It was the best and most comprehensive solution for allocating AWS infrastructure for a long time. It isn’t hosted inside AWS, however, because it’s just a cli you run yourself, but it can be extremely versatile. It used to be that AWS would roll out commands to the cli before any other IaC solution so you had to leverage it for the newly released tech on your account.

AWS CloudFormation

The first true IaC language developed by AWS. It’s declarative, runs inside your AWS account which is great for tracking and maintainability. CF was always lagging with features in the past because the cli would get them first, but AWS has done some work to try to close that gap so it may not be that much of an issue anymore. Being declarative has it’s drawbacks, though.

Azure ARM

For those of you that use Azure, ARM is the equivalent tech for that cloud vendor.

AWS CDK

Which is why AWS made a new language for IaC called AWS CDK. This is the next evolution in AWS’s offering for IaC. It is programmatic, getting away from being declarative, which allows for much more flexibility in defining your infrastructure. It is, however, still vendor locked into only leveraging AWS cloud technology. If you are 100% using only AWS Cloud offerings, then any of AWS’s options should be fine, including CDK, but before you make that choice let’s talk about what I mean by 100%.

Terraform

Terraform offers two varieties of IaC, one declarative and one programmatic, but the main distinction that Terraform has over the AWS offerings above is that it’s built to be multi-cloud, multi-provider. Let’s talk about what multi-provider actually means.

Most people getting into the cloud and learning about IaC will latch onto the idea of scripting out whatever they are building in their cloud account of choice pretty quickly. It’s a pretty natural process. What will happen to a lot of teams, however, is that you will come to realize that you actually purchase or own licenses for all kinds of other services you need to build the product your team is focused on. Things like…. maybe you use fastly as a cdn provider, MongoDB Atlas for a database solution, or datadog as your observability platform. Now, you may be asking yourself, why do any of those things matter? Well, with terraform, all of those products are connected in the terraform ecosystem as providers. So you can script one IaC solution together that ties all of those things, including your cloud account infrastructure, all in one place. So when I was talking earlier about 100%, I was talking about everything your team uses.

Point of note, I didn’t really bring up GCP anywhere in this discussion, but that’s because they leverage APIs for a lot of their allocation, and those APIs feed into terraform providers so if you are in GCP land you are probably looking to adopt terraform. In a lot of ways, the same may be true for folks looking to adopt Azure, even though they have their own templating stack. Terraform is a choice my teams tend to make these days because of all the flexibility I mentioned before. It tends to be the selection of choice, although I have managed teams in the past that used AWS CloudFormation extremely successfully.

Getting back

Ok so getting back to the example we started, let’s now talk about how you actually create that infrastructure you need for you container registry. My teams use Terraform CDK so my examples will be with that, but our approach translates well to whatever choice you make. And again, I’m not going to get into the details of how to do all things terraform, rather I want to point out some key decisions that you want to make when you start building out your IaC.

Organizing your stack

When we start this build out you are going to want to start organizing where you put your scripts, and from my experience you want to immediately split up infrastructure you make for your account, and what you make for the product you are deploying, and their subsequent environments. Things you need to do in your account would be things like allocating your VPC, networking, domain management, stuff that you don’t normally handle on a product by product basis. For this example I’m going to actually put my ECR allocation at the account level because I’m going into this with the mindset that the overall stack here is going to be rather small, so allocating an ECR for each product is probably overkill. Think the difference between a start-up vs an enterprise corporate team. This stack is more of a start-up scene right now, whereas in a more enterprise environment you might allocate an ECR for each product, or maybe each team.

This is what allocating the ECR looks like right now in my repo, it’s nothing fancy:

import { Construct } from "constructs";
import { App, TerraformStack, TerraformOutput } from "cdktf";
import { AwsProvider } from "@cdktf/provider-aws/lib/provider";
import { EcrRepository } from "@cdktf/provider-aws/lib/ecr-repository";

class MyStack extends TerraformStack {
  constructor(scope: Construct, name: string) {
    super(scope, name);

    new AwsProvider(this, "AWS", {
      region: "us-east-1",
    });

    new EcrRepository(this, "how-to-cloud", {
      name: "how-to-cloud-ecr",
    });

    new TerraformOutput(this, "testing", {
      value: "hello world",
    });
  }
}

const app = new App();
new MyStack(app, "how-to-cloud");
app.synth();

The code is actually way less important here than the folder structure I’ve started. We started with setting up docker and we have that over in a docker folder, and now that we’re allocating infrastructure we put that over in an iac folder, with an account folder inside that just for the account stuff. Later we’ll make more scripts in product folders that are separate from the account, so building out new infrastructure will be really easy.

But that’s for a later post.

From Scratch: OIDC Providers

Joseph — Mon, 20 May 2024 12:00:00 +0000

It's time to divert our attention to what will soon become our infrastructure deployment process. This will feel a lot like application deployments, but there will be some stark differences we'll be going over. Our applications will incorporate infrastructure in the future, but not everything we do with our IaC will be related to an application.

Github Actions

I'll be deploying my infrastructure through Github Actions because it's where I keep my repositories anyway, and keeping the deployment scripts there is really convenient and makes tracking things really easy too.

The Github and AWS Dance

If we're going to be deploying infra from within github actions, it stands to reason we'll have to give our workflows permission to deploy infrastructure in our AWS account. That's where OIDC comes in. AWS allows you to build a trust relationship with an OpenID connected provider (in our case github), and then define a permissions policy to a role in your account. Once the role is defined, you then have complete control over what permissions you let that role have within your AWS account.

Github has some documentation on how the whole process works.

Creating the Provider

We'll start off by essentially following AWS guidance on connecting with Github. In order to do that, we're going to need to grab a certificate from github to put into our provider definition.

Certificate

Grab the certificate from github by using

openssl s_client -servername token.actions.githubusercontent.com -showcerts -connect token.actions.githubusercontent.com:443

The cert you want is the last one you see in the terminal. You want to create a file named certificate.crt and put all the cert information from the terminal which includes the -----BEGIN CERTIFICATE----- and -----END CERTIFICATE----- parts too.

Thumbprint

Now with the certificate, we'll need to run through a process to grab the thumbprint. Open up a terminal in whatever path you made put the cert, and run the following command:

openssl x509 -in certificate.crt -fingerprint -sha1 -noout

The output should look something like this:

SHA1 Fingerprint=99:0F:41:93:97:2F:2B:EC:F1:2D:DE:DA:52:37:F9:C9:52:F2:0D:9E

You only care about the alphanumerics in the fingerprint itself, so remove the colons and you're left with something like this:

990F4193972F2BECF12DDEDA5237F9C952F20D9E

Depending on where your provider is, you're fingerprint will be different.

Provider Definition

Now we're finally ready to script the provider itself.

const oidcProvider = new IamOpenidConnectProvider(scope, `${name}-provider`, {
  url: "https://token.actions.githubusercontent.com",
  clientIdList: ["sts.amazonaws.com"],
  thumbprintList: ["1b511abead59c6ce207077c0bf0e0043b1382612"],
});

In the thumbprint list you'll put the fingerprint you got from earlier. You need to make sure your thumbprint is in all lowercase when you do this, because if you don't then terraform will always detect drift from the AWS provider that is provisioned since AWS will tell it it's in all lowercase.

Trust Relationship

After we've defined the provider we have to describe the trust relationship between it and our AWS Account. This basicaly boils down to defining a policy that allows role assumption with a web identity.

The policy looks like this:

const assumePolicy = new DataAwsIamPolicyDocument(
  scope,
  `${name}-assume-policy`,
  {
    statement: [
      {
        actions: ["sts:AssumeRoleWithWebIdentity"],
        principals: [
          {
            type: "Federated",
            identifiers: [oidcProvider.arn],
          },
        ],
        condition: [
          {
            test: "StringEquals",
            variable: "token.actions.githubusercontent.com:aud",
            values: ["sts.amazonaws.com"],
          },
          {
            test: "StringLike",
            variable: "token.actions.githubusercontent.com:sub",
            values: [`${config.githubOrg}/*`],
          },
        ],
      },
    ],
  }
);

The provider ARN is the provider we made earlier, and the value in token.actions.githubusercontent.com:sub is your github org you want to allow to provision in your AWS account. You can limit this in various ways, but in my case I just want to give the whole org the same set of permissions. Alternatively you can limit this down to specific repos, or github envs, etc etc.

Permissions

Now that we have the provider scripted and the trust relationship defined we can turn our attention to what permissions we want to give the role. To keep things simple, I'm going to give the deployment process PowerUserAccess. This is a managed policy from Amazon, and I have no security concerns around the permissions it allows. If you work in a space where the security concerns need to be more limiting, this is where you could make a more restrictive policy yourself, which is also fine.

For this situation, I just need to grab the power user policy like this:

const powerUserPolicy = new DataAwsIamPolicy(scope, `${name}-trust-policy`, {
  name: "PowerUserAccess",
});

Role

None of this matters if we don't create a role that is going to use all this stuff:

const role = new IamRole(scope, `${name}-trust-role`, {
  namePrefix: `${name}-trust-role`,
  assumeRolePolicy: assumePolicy.json,
});

The assume role policy here is the trust relationship we defined. Now we need to give the role power user permissions.

new IamRolePolicyAttachment(scope, `${name}-rpa`, {
  role: role.name,
  policyArn: powerUserPolicy.arn,
});

Try is Out

After we've cdktf deploy '*' the stack we should try this out and see if it's working. I made a really simple workflow that just configures the aws cli and then prints out all the roles in the account to make sure it has the permissions we gave it earlier and everything is wired up properly.

The workflow file looks like this:

name: Deploy Infrastructure
on: push
permissions:
  id-token: write
  contents: read
jobs:
  Deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Git clone the repository
        uses: actions/checkout@v4
      - name: configure aws credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: [arn:aws:iam::therolearn-i-just-made]
          role-session-name: githubsession
          aws-region: us-east-2
      - name: Proof of Life
        run: |
          aws iam list-roles

After that it's just a matter of cleaning things up a little bit and push main up to github, which triggers the workflow to run, and I'm able to see the github action produce a list of the roles in my account.

Next Steps

We're now ready to start letting Github Actions handle our infrastructure deployments. That's going to involve changing a few things we already did and then making some more workflows in our repo, but that's for next time.

From Scratch: Permissions

Joseph — Wed, 15 May 2024 23:55:12 +0000

Once we have our groups and users set up from our external identity provider we can move on to defining what permissions they should have. This is mostly straightforward, but we'll have to take a few detours along the way because of how groups work with the Google Workspace IDP.

Permission Sets

The first thing we have to do is script our permission sets. We will use these to connect IAM policies to our IDP groups. This will all be done in CDKTF.

const set = new SsoadminPermissionSet(scope, `${name}-admin`, {
  name: `${name}-admin`,
  instanceArn: arnId,
});

There's not a lot to this. The ARN we are using here for the instanceArn actually comes from the same DataAwsSsoadminInstances we used to get the storeId before.

Policies

Now that we have our admin permission set, we have to actually give it an admin policy.

new SsoadminManagedPolicyAttachment(scope, `${name}-admin-policy`, {
  instanceArn: arnId,
  managedPolicyArn: "arn:aws:iam::aws:policy/AdministratorAccess",
  permissionSetArn: set.arn,
});

The policy here is just the AWS managed policy that comes with every account. We're ready to deploy at this point, so we need to go back into our AWS console, and reactivate the root user's access key. Remember, the last time we used it we deactivated it as soon as we were done, and we're going to do the same thing here.

With the access key active again, we can go ahead and cdktf deploy our stack, and we should see two more resources get created. Once it's done we go back into the console and turn off the access keys again.

Group Assignment

With these new permissions we should be all set now to assign our groups those permissions and be on our way, but this is where we hit our first snag. We need to go into the identity center console as root and assign the group to our AWS account along with the admin permission set we just made, which we can do, but there's a problem. Because of how we had to make the groups in identity center, we aren't able to assign users to the group in the console. It has to be scripted. The root user shouldn't do this, though, because managing permissions at this level would mean people would have to be logging into root all the time, which is a big problem. Instead, what I'm going to do instead is assign myself to the AWS account. My user was brought over automatically through the identity provider, and I can set all that up in the console easily.

Switching Users

With that set up, I now have my user assigned as an admin in the AWS account. Now it's time to configure AWS CLI with my sso. With that done, we're going to make a new stack that uses the admin profile to allocate infrastructure.

Account Assignment

Now we're ready to assign the group to the AWS account, and add any users we want along the way. Since we'll be doing this as an admin user, we'll need to make a new CDKTF stack, and deploy it using the admin profile after logging in using aws sso login. In my case, I made an admin profile and configured the aws cli with it.

The first thing we'll add to the new stack is going to be the account assignment that adds the group to the AWS account with admin permissions.

new SsoadminAccountAssignment(scope, `${name}-ga-admin`, {
  provider: config.provider,
  principalType: "GROUP",
  targetType: "AWS_ACCOUNT",
  targetId: config.caller.accountId,
  permissionSetArn: config.identityCenter.permissions.admin.arn,
  instanceArn: config.storeArn,
  principalId: config.identityCenter.groups.admin.groupId,
});

At this point the group is set up and ready to use. We can now go ahead and add some users to it.

const juser = new DataAwsIdentitystoreUser(scope, `${name}-iu-joseph`, {
  identityStoreId: config.storeId,
  alternateIdentifier: {
    uniqueAttribute: {
      attributePath: "UserName",
      attributeValue: "joseph@awsdemo.com",
    },
  },
});

new IdentitystoreGroupMembership(scope, `${name}-gm-joseph`, {
  identityStoreId: config.storeId,
  groupId: config.identityCenter.groups.admin.groupId,
  memberId: juser.userId,
});

Remember the users are automatically syncronized from our external identity provider, so we won't be making a user. Instead, we just need to grab the one we already have. In my case I found my own account by username. There are other options too, but this one is pretty straightforward. From this point it's just a matter of creating a membership for the user to belong to the group.

Separate Stacks

We started off by building some resources using root and then made a separate stack for things an admin can then do. However, the interesting thing about this approach is that once you have an admin that can log into the system, they also have enough permissions to do everything in the first stack we made. We used the root user to get us just far enough in our IaC build out that our admin users could take over. From this point on, we won't be needing the root user anymore, so we can go ahead and delete the access key it had.

As always, if you are interested in how everything ties together, you can take a look at the how to cloud repository.

From Scratch: IaC

Joseph — Fri, 10 May 2024 01:33:23 +0000

It occurred to me recently that it might not be obvious when someone should start using IaC. This can be daunting because there are things you can't do, and some things you shouldn't do with IaC. There are also situations where you could use it, but it doesn't lead to any benefit.

Sign up for AWS Cloud Account

There will some some things you have to do that are probably common sense you can't leverage IaC for. Signing up for your AWS cloud account is one of them. In my case I'm completely starting from scratch, so let's go through everythign involved in that. During this series, when we actually use IaC, I will be very explicit about it, so you'll know the difference between something I had to do by hand / manually vs something I scripted.

Root Access

So you've signed up for your account, and you have your root account. For the majority of the work you'll be doing, you will not be using your root account. It's dangerous to rely too heavily on your root account, but there are a couple things you have to use it for, especially at the beginning.

MFA your Root Account

I've had the privelege of working with some really talented security engineers around cloud, like Chris Farris for example. When it comes to cloud governance you really can't do wrong by just doing what they recommend. So it should come to no surprise the next few steps I'll be doing are going to essentially be doing that.

Your Root Account will be the only account that can be logged in outside your identity provider, and the account that can do the most damage if compromised, so it's critical you keep it as secure as possible.

Setup External Identity Provider

You really need to connect your organization with an identity provider. Before you can do this you have to enable Identity Center. While you go through this process AWS is going to ask you to enable Organizations as well, which you'll want to do. In my case I'm going to be doing this with google workspaces, so I just followed this documentation and got up and running.

Customize the access portal url

This isn't necessary but it's a nice to have. Once you have your identity provider active, the portal to sign in can be a little obscure, but you can change it to use a subdomain of your liking, assuming it's available.

Groups

When working with Google Workspaces, one of the limitations with the IDP integration is that workspace groups don't automatically come over like users do. To make matters even more complicated, you can't add groups using the AWS console, you have to add them through the CLI or the API. That being said, it's worth going through the trouble to make some groups anyway, because they will be useful later when we want to start giving permissions to our teams.

Enter IaC

Now, most of the time folks will just make the groups using the CLI and call it a day, but I think this is actually a good starting point to creating your IaC footprint. This does come with some caveats though, because this isn't going to follow the normal flow of scripting we will get into later after we're done using the root user.

Scripting against the Root Account

We are going to be creating identity groups using terraform, but in order to do that we need some access keys, and they have to come from root (because we don't have any other users yet). This is something that is usually recommended never to do, and while I largely do agree with this practice, in this situation I argue it's actually a better practice to do it this way than to do it manually. Why? Scripting something makes it repeatable, which is less prone to error. Additionally, there are some safeguards we'll be putting in place so that the root user doesn't get compromised. I can't tell you how many times I've set something up without IaC, and then a month later I forgot what I did, which caused me or my team a lot of headaches. It also creates a less secure situation because you now have to circle back on what you did and rethink everything, which can introduce errors and misteps.

So we're going to make some access keys, but we're going to immediately deactivate them. Why? Because we haven't written our scripts yet, and we're not going to leave anything to chance when it comes to the root account.

Show me the code

I'm going to be building a solution around this using CDKTF but you could solve this with other scripting tools as well.

To remedy this group situation is actually pretty easy and doesn't require a lot of code. We're going to define a construct to hold onto all our org's groups. We'll deploy this once and shouldn't need to revisit this again, which will be a common pattern for anything we script using the root user.

export class IdentityCenterGroups extends Construct {
  constructor(
    scope: Construct,
    name: string,
    config: {
      provider: AwsProvider;
      org: string;
    }
  ) {
    super(scope, name);

    const stores = new DataAwsSsoadminInstances(
      scope,
      `${name}-stores`,
      config
    );
    const storeId = Fn.element(stores.identityStoreIds, 0);

    new IdentitystoreGroup(scope, `${name}-admins`, {
      identityStoreId: storeId,
      displayName: `${config.org}-admins`,
      provider: config.provider,
    });
  }
}

Here I'm just making an admins group, but you would also add others like developers or security based on whatever needs you have. This is important because later we can give these groups specific permissions.

Finally we need to add this construct to our stack and deploy it.

class RootStack extends TerraformStack {
  constructor(scope: Construct, id: string) {
    super(scope, id);

    const providerAsRoot = new AwsProvider(this, `${id}-provider`, {
      profile: "root",
    });

    new IdentityCenterGroups(this, `${id}-identity-center`, {
      provider: providerAsRoot,
      org: organization,
    });
  }
}

const app = new App();
new RootStack(app, `${organization}`);
app.synth();

Before we can deploy it, we need to go back into our AWS console and activate the access keys for our root account. Now we can deploy this with cdktfy deploy and we should immediately see the groups show up in Identity Center. After we're verified the group was created, immediately go back to the access keys for root and deactivate them.

What's Next

We have our AWS account created and connected to our identity provider of choice. That Identity Provider isn't configured to provide access into our AWS account yet, however, because we haven't configured permissions yet. That's what we'll get into next. It's also worth pointing out that, even though we've been doing everything using the root user up until this point, our goal is to do as little as possible with this user, so once we've finished the bare minimum with root, we'll be switching over to an identity user for everything else. We're just not to that point yet.

Also one last thing for reference because I get asked this a lot: it took me about 3 hours to do everything in this article and write the blog article. If you are wondering how long it takes to do IaC properly, it really doesn't take that long. If you're curious exactly what I built for this set up, you can check it out in my how to cloud repository in the aws section.

How to Cloud: Virtual Machines

Joseph — Wed, 08 May 2024 01:08:27 +0000

Ok at this point we are now ready to get into deploying your app into some kind of compute solution in your cloud provider. In order to make the best decisions when it comes to choosing said compute solution, we need to start with the foundations of almost all of them: virtual machines.

Cornerstone of Compute

Virtual machines will make up the cornerstone of all compute solutions you will build on top of, whether you have control over them or not. In fact, that’s one of the decision criteria you should consider when you get to choosing one, but that’s for another post. For now, what’s important is understanding the ins and outs of how your cloud provider allocates and operates their VMs, and how those aspects of your cloud provider effect the higher level compute solutions they all offer.

Bare Metal

Virtual Machines (VMs) are probably the closest to the metal kind of solution that you can leverage with a cloud provider. AWS calls this offering AWS EC2, Azure just calls them Azure Virtual Machines, and GCP calls them Compute Engine. They all end up doing the same thing in the end. You request for a certain configuration of machine, and after some time (usually a few minutes), you can have a machine spun up that closely mimics an on prem hypervisor. You can almost, for all intents and purposes, do the same thing to this machine that you would do with one mounted in a data center. That means you have the highest degree of control on this machine that you can possibly have in your cloud provider. You can mount disks to them for storage, you can configure them with GPU, SSD state storage, basically set up any kind of compute or data storage solution you want.

This also means, however, that you have the least amount of operational efficiency by leveraging a virtual machine. You have to keep the OS up to date from a security perspective. You have to maintain the overall health and state of the machine. You have to connect that virtual machine to all the other offerings of your cloud provider to create an overall solution. There is a lot of extra work you have to put in to make that virtual machine do what you want it to do. The cloud provider is really only giving you one thing you didn’t have before, and that is rapid availability.

Cost Model

This is where things might get a little tricky. On paper, VMs look to be the cheapest of all compute solutions. However, if you are using VMs as the only element in your compute solution, you are going to have significant labor costs. In fact, if you don’t add other aspects to your compute solution, like serverless functions, or some type of orchestration plane, you will end up operating the most expensive solution you could have built once you consider the total all in cost. VMs are still crucial to understand, however, because they feed into all the other compute solutions you can use. This will ultimately boil down to trades off between labor costs or operational efficiency, and operational flexibility or control. We can get into those specifics more in other posts when we talk about the specific solutions that are offered and how they effect those trade offs.

To keep things simple for now, let’s just do a basic run down of the cost structure for VMs by themselves. It will serve as a fraction of the overall cost to build your solution, and in a lot of ways it makes up the foundation of your cost model.

Examples

AWS

Amazon offers a variety of ways to rent their VMs. The most straight forward (and expensive) one is just using their On Demand allocation. Prices vary depending on your configuration of CPU, Memory, GPU support, storage, etc, and I won’t be going into the details of how they effect cost, but it’s important to note that On Demand pricing effectively influences all other pricing.

There is also Reserved pricing, which allows you to actually reserve a certain level of allocation in 1 or 3 year agreements. That may sound confusing, but it’s important to note in this pricing model that you aren’t renting a specific VM for a specified time. It’s more like you are renting the time for that configuration, and that agreement nets you some cost savings. Having said that, there are some limitations. There are two ways to purchase a Reserved instance, Standard and Convertible. Standard rental allows you to modify certain aspects of the reservation like availability zone, networking type, instance size, but you can’t change other things such as the instance type. You can, however, sell that rental license on the AWS marketplace. Then there is convertible. Convertible agreements allow you to exchange one instance for another. You can do this as many times as you like but the value of the overall exchange has to be equal or greater than the agreement you put in place to begin with. Convertible rentals can not be sold on the AWS marketplace. If you choose either of these styles of pricing, you can get upwards of a 75% reduction in price from AWS for the equivalent machine with on demand pricing.

Finally we have Spot pricing. Spot pricing doesn’t require a termed agreement like reserved pricing, but it can still afford you similar savings. It achieves this through one major stipulation: AWS may choose at any time to terminate your instance with only a two minute warning. This can sound devastating if your cloud solution relies solely on VMs, but, as we will discuss in later posts, layering an orchestration platform on top of your rented VMs can make this an extremely appealing option. With this type of set up you can achieve savings up to 90% of what you would have spent on an on demand instance.

GCP

You’re going to start to see a lot of repetition amongst cloud providers. What GCP offers is extremely similar to AWS, and largely the only differences are in names.

GCP’s main offering is largely referred as on demand or “pay as you go” pricing. It’s akin to AWS’ on demand pricing, and the structure of offering has a lot of similarities. From there, GCP offers “commitment” pricing that is similar in principle to AWS’ reserved pricing. In addition to this offering, GCP will also give discounts for what they call “sustained use” discounts (or SUDs). SUDs can give you a 30% discount if you haven’t received some other form of discount and your compute has been running for at least 25% of the billable month. GCP also offers Spot or Preemptible VMs, and as you’ve probably guessed they function in a very similar way to what AWS Spot instances offer. With GCP spot instances you can achieve savings upwards of 90%.

Azure

We’re going to do the big three and then we’re going to stop. I wanted to get all three laid out so you can see the pattern. This will effect your decision process later when choosing your solutions, and it’s important to understand that all cloud providers should (most do) offer you the same cost savings models.

Azure’s VMs can be rented under a Pay as you Go model, a Savings Plan, Reserved Instances, and finally Spot models. Pay as you Go is the same thing as On Demand. Reserved is the same as before. Spot as well. Savings Plans are worth pointing out. They allow you to lock in an agreed to hourly price for a variety of services Azure offers for a period ranging from 1 to 3 years. In order to get those reduced prices, however, you have to agree to a certain level of consumption during that time period.

Get Containerized

Developing the right compute solution is a series of related decisions. It starts with understanding how to choose the right VM structure. We’ve already talked about containerization, so the next thing we’ll go over will be how to tie together containerized applications with the foundation of virtual machines we just went over.

How to Cloud: Containerization

Joseph — Tue, 19 Mar 2024 18:11:10 +0000

I have kind of a standard approach on building out a cloud presence that I’ve developed over the last decade or so. I’ve used it extensively with multiple teams to create a highly mature DevOps culture within groups again and again. Today I’m beginning a series that explains some of the basics. Hopefully it can be of some use to those trying to understand good ways to leverage cloud technologies.

Any Cloud Will Work

I’m going to start this series off with examples of how to deploy the concepts I’m talking about in AWS, but the approach I use works with a lot of cloud solutions out there, including Azure and GCP. I will do my best to call out how the approach would work in the Big 3 where it’s appropriate, but the topic here is not about showing you a detailed solution of a tech stack, but rather the strategy behind the build out and how to make the best decisions when it comes to rolling things out. And speaking of decisions, when it comes to adopting a cloud approach, the very first thing your teams need to understand is containerization. The need to learn Docker.

Learn Docker

Docker is the gateway to understanding what containerization is all about. Not only is it a containerization solution itself, but the concepts your team learns from understanding it will help them navigate the massive forest of different compute options they will have across any cloud vendor. Without starting here you will never be able to see the forest through all the trees being thrown at you. Options range from AWS ECS, to Fargate, or EKS, or Lambdas, and that’s just one cloud vendor, so trust me and start here. I’m not going to get into the specifics of all things Docker, but I do want to highlight some of the learnings I’ve made over the years.

Organize your images

When you are just beginning to learn what containers are this won’t make a lot of sense, but after you have your first image working and you actually have a system working inside a container, you should stop and think about organizing your image builds. What this means is not only keeping them logically organized so your team can understand how to use them efficiently, but also layering them together so you can reduce your build times significantly where you have a lot of commonality across systems. Let me show you a simple example.

The Starting Point

Once the team knows how to build docker images and can reliably run them, I start introducing a pattern that allows them to create layered images that use each other instead of creating separate duplicate builds that make your CI/CD process take forever to finish. This begins by identifying your base starting point and branching off from there. Let’s say your team needs to build a web app and a background service, but they plan to use nodejs for both. Start by making a docker compose yml that will simply define the common base both of them will use. For example:

docker-compose.yml

version: "2"
services:
  how-to-cloud-base:
    build: .
    image: how-to-cloud:base

Dockerfile

FROM node:alpine

The base image right now is mind numbingly simple. The docker compose yml just builds the Dockerfile below and then tags the image with a base tag. The Dockerfile doesn’t really do anything other than grab the alpine distro of node. That’s how it starts. Later your team will realize there are commonalities that both systems share and they can “bubble up” those things into the base image when they realize it. Now that you have the base the team can use it to create the web app and the background service.

Let’s build out the web version first…

docker-compose.web.yml

version: "2"
services:
  how-to-cloud-web:
    build: 
      context: ../
      dockerfile: ./docker/Dockerfile.web
    image: how-to-cloud-web:latest

Dockerfile.web

FROM how-to-cloud:base

COPY ./web /app

WORKDIR /app

CMD npm start

and now the background service, which for now will look very similar to the web…

docker-compose.background.yml

version: "2"
services:
  how-to-cloud-background:
    build: 
      context: ../
      dockerfile: ./docker/Dockerfile.background
    image: how-to-cloud-background:latest

Dockerfile.background

FROM how-to-cloud:base

COPY ./background /app

WORKDIR /app

CMD npm start

Clarifying Points

One thing that’s worth pointing out here is that the web app and the background service function completely independently. Meaning one system does not in any way depend on the other from a deployment perspective. If they did you there are other ways to deal with that.

Another call out I want to make is that when I use this organizational method I really like to make an LTS build that has kind of a default set of what my team normally uses in the cases where they just wanna use “the normal” build and not have to have something specialized. It looks something like this.

docker-compose.lts.yml

version: "2"
services:
  how-to-cloud-lts:
    build: 
      context: ../
      dockerfile: ./docker/Dockerfile.lts
    image: how-to-cloud:lts

Dockerfile.lts

FROM how-to-cloud:base

COPY ./lts /app

WORKDIR /app

CMD npm start

Deployment

So with all this set up we can talk about how this all ties together in a CI/CD pipeline and why you can get some pretty significant performance gains for your build when you follow this approach. The process is separated into three main stages: build, deploy, clean. It looks something like this.

Build

#!/bin/bash
set -e

# build image
docker compose build
docker compose -f docker-compose.web.yml build
docker compose -f docker-compose.background.yml build
docker compose -f docker-compose.lts.yml build

The important thing here is that you build the base docker image first, and then build the images that depend on that base image. You can get much more elaborate with the layering of the images if you want to, but the main point here is that the more commonalities that you bring up into higher layers the faster you’ll make your build, and the easier it will be to maintain your systems across all your teams because they’ll be sharing and collaborating together instead of duplicating the same concepts in silos.

Deploy

#!/bin/bash
set -e

# login to ECR
version=$(aws --version | awk -F'[/.]' '{print $2}')
if [ $version -eq "1" ]; then
  login=$(aws ecr get-login --no-include-email) && eval "$login"
else
  aws ecr get-login-password | docker login --username AWS --password-stdin $ecr
fi

# push image to ECR repo
docker compose -f docker-compose.lts.yml push
docker compose -f docker-compose.web.yml push
docker compose -f docker-compose.background.yml push

After the images are built you can deploy them to your container registry. In AWS it looks basically like this. The AWS CLI has two versions, and both of them are pretty widely used from what I’ve seen, so I still have a check for which version the team has installed so it uses the right command, but if you know your team is going to only use version 2, you can clean that up a little bit and only use the second command to log into the ECR.

Aside from that once you have logged into your ECR it’s a simple matter of pushing all the images up. You don’t need to publish the base image because you should be building that every time your CI/CD pipeline runs, unless you have a really good reason to need some kind of available redundancy for it. From my experience that is usually over kill.

Also, in the example above we’re deploying to AWS ECR, but every cloud vendor has their own container registry. The point here is how you design the process, not the specifics of how you get the image into the registry. The process works for most if not all cloud vendors.

Clean

At the end of your build process you might need to do some cleaning up, so we have a third stage to handle that, but a lot of the time it’s empty because the CI/CD process is ephemeral anyway. However, if you are using external dependencies during your build and you need to clean them up after the fact, then you can do that here, or if your CI/CD process is not ephemeral then you’ll probably want to do some clean up before you call the build done.

In this set up our clean up process is empty because there’s nothing really to clean, but I’ve used this in the past under certain situations. For example, we’ve built github self hosted runner farms that required us to pre-emptively download and extract binaries because the docker build was actually failing to sign certificates, so the host had binaries we needed to clean up after we were finished with the build.

Show me the Code

I went ahead and made a repository to showcase all the lessons I talked about here, as well as any future topics surrounding How to Cloud. If you want to see the full set up I described here head over to https://github.com/josephbulger/how-to-cloud