Forem: Josh Pollara

Building in Public: Pricing is our scariest decision

Josh Pollara — Mon, 31 Jul 2023 19:35:10 +0000

While building Terrateam, pricing have been some of the scariest decisions we’ve made. We have a lot of pride in what we’ve built. It doesn’t matter how many people say they like your product when you show it to them, the rubber meets the road when they have to give you their credit card.

At first, we just tried to copy the pricing of companies that we thought were our competitors. We didn’t know what we were doing but they probably did. Take what they have and make a slight tweak to showcase what we think makes us special. We also asked for advice from a lot of people but it was hard to get an answer to “why” they recommended a price to us. A lot of the advice just seemed like “sounds like something that should cost $X”.

To get our first paying customers, we went low. We looked at the cheapest non-free plan of our competitors and undercut it by 30%. And it worked! We started getting paying customers. But defining the value of your product purely based on the value of another product leaves an uneasy feeling. We didn’t want our product to just be a cheap copy of another. We had our own vision.

We knew that we didn’t want to create a billion dollar company. We just wanted to solve our customers’ problems and make enough money to enjoy life. Early in our formation we read Company of One by Paul Jarvis. Company of One is for entrepreneurs that are happy creating a small, steadily growing, company. In other words: us. One of the biggest take-aways for us was about finding your market. A large company needs to appeal to a lot of people but a small company has more freedom to have a personality. We can be really appealing to a smaller group of people at the expense of being a turn off to others.

We believe there are users that want a really good experience planning and applying their Terraform changes and will pay for support and SLAs. Our vision for Terrateam is a product for people who are happy never leaving GitHub for Terraform changes. We want very few touch-points with the product. Once it is setup, it should become invisible. In their day-to-day usage our customers see almost no branding. All Terrateam configuration and operations are done inside the repository or through pull requests. We don’t have a UI. Users create a pull request and see their Terraform plan in the comments and then they apply it in the pull request. There is no reason to leave. And we have a ton of features! So how much should that cost?

We aren’t flashy. But we are feature rich. We are focused on a specific use case: Terraform plan and apply on GitHub. Being bootstrapped, we are very lean. We’ve implemented 70%-80% of the enterprise features of the competition but on a shoestring budget. We’re a small company that doesn’t have investors to repay. We want to stay small while maintaining our high quality of support, so we can’t have too many customers. Finally, we don’t even need to make a million dollars in revenue per year to be a success.

That’s when we realized that we didn’t know who our competition was. There are the popular names in Terraform CI/CDs and we just assumed they were our competition. But they are rocket ships. They are after those enterprise contracts that are thousands of dollars per month. We don’t think your Terraform CI/CD should be so sophisticated that it should cost that much.

With all that, the latest iteration on pricing is: $496/mo. We provide most of the enterprise features our users want at 1/10th the cost of the other options. With no GUI, the experience is more utilitarian, but you get the information you need in the place where you already are. We’ve only recently rolled out this change so we are still evaluating it. If history is any indicator of the future, we’ll be iterating on pricing again.

The Future of Terraform: ClickOps

Josh Pollara — Thu, 22 Jun 2023 09:14:46 +0000

Quarter questions

Every now and then it's important to step back from what we're doing and think about the future. At Terrateam, we like to ask a question each quarter to get our gears turning. This quarter we asked:

What will Infrastructure as Code (IaC) look like in five years?

Infrastructure is slow to change

Software infrastructure does not move quickly. While greenfield projects may be able to take advantage of the latest trends, once software is made and in production, the lower bits don't change that much.

It is difficult to migrate an existing project from DynamoDB to PostgreSQL or from JavaScript to Rust. The number of banks still running on mainframes with COBOL might make you shudder.

If the Oxide podcasts are any indication, things get more ossified further down the stack. In many cases, we prefer to add a layer to existing infrastructure to give us the new functionality we want rather than rewriting something from scratch.

More people creating and modifying infrastructure is good

More and more people are encoding their infrastructure in some way that can be tracked, audited, destroyed, and recreated. Terraform is the biggest contender along with Pulumi and the AWS CDK. These tools provide a way to provision and manage infrastructure. And while they are very powerful, they can be daunting to learn. There is a lot of desire to break the traditional silos of development and operations, empowering software engineers to manage their own infrastructure, but in many cases it's too much to ask them to become proficient in a whole new ecosystem.

It's not uncommon for organizations to use templates in order to simplify the process. Existing TACOS tools support this and tools like Backstage are also being used for this. Want to create a database and some compute? Fill out a form and the code will be generated. The learning curve is minimal. The downside of templates is that they only work for creation. They are not much use for modifying existing code. As the project grows, development will slow down as the team will have to learn Terraform, AWS CDK, or Pulumi. ClickOps is the idea of providing GUIs for manipulating infrastructure where the output is code, such as Terraform HCL.

There are a lot of ways that ClickOps can be achieved, but we think the any ClickOps solution should use HCL as its source and destination language. ClickOps tools can integrate into the existing rich ecosystem of Terraform tooling and advanced users can read and write Terraform code by hand if they wish. With ClickOps, we can maximize the number of people who can comfortably manage infrastructure.

CDKTF, AWS CDK, and Pulumi are steps in the wrong direction

According to Hashicorp:

HCL is a toolkit for creating structured configuration languages that are both human- and machine-friendly, for use with command-line tools. Although intended to be generally useful, it is primarily targeted towards devops tools, servers, etc.

Whether or not one thinks Hashicorp's HCL is good at expressing infrastructure, it is in the least, a very limited language, which makes it hard to write convoluted code. HCL is closer to other configuration languages like YAML or INI than a programming language.

On the other hand, the AWS CDK and Pulumi allow one to write infrastructure in general purpose languages, such as TypeScript, Go or Python, This gives a lot more power to the user but also means the code representing their infrastructure can be quite complex. HashiCorp is working on CDKTF to compete with AWS CDK and Pulumi.

One perceived benefit of CDKs is that they allow multiple languages to be used in managing infrastructure, such as TypeScript, Go, Python or Java. This has downsides as well. Documentation explodes as each language needs to be covered. APIs for each language can be auto generated, but auto generated APIs can feel awkward, so native APIs need to be maintained. There is a lot of magic going on to make these general purpose languages generate declarative infrastructure which makes writing code for a CDK quite a different experience. While it might be TypeScript one is writing, how they can use it is going to be different from what they are familiar with.

The number of places that a bug can be encountered goes from just the Terraform CLI to now all the machinery between the CDK and Terraform. Finding help becomes a challenge: is one trying to resolve a Python issue or a CDK issue? What if you have a bug in your TypeScript CDK and the answer found online is in response to a person asking in Go? Can you even understand the answer if it requires details of that language?

HashiCorp has tried to keep up with AWS CDK and Pulumi with CDKTF. The way CDKTF works is the CDK code is compiled to a program which, when run, generates HCL, in JSON format. CDKTF then runs the Terraform CLI on the generated HCL. For those that worked with JavaScript before source maps were a thing: compiling one language to another is a great way to solve some problems but it makes debugging difficult without a lot of tooling that can translate an issue in the
compiled code to the source code.

The multi-language approach makes creating tooling more challenging. There has been a lot of tooling for Terraform built that consumes .tf files, like linters, security scanning, cost estimation, etc. The CDKs make this a lot more difficult. Not all of them can read HCL in JSON format, and even if they can, the developer experience of looking at an issue in generated code is not ideal.

Platform engineering is trying to deliver the self-service tools teams want to consume to rapidly deploy all components of software. While it may sound like a TypeScript developer would feel more empowered by writing their infrastructure in TypeScript, the reality is that it's a significant undertaking to learn to use these tools properly when all one wants to do is create or modify a few resources for their project. This is also a common source of technical debt and fragility. Most users will probably learn the minimal amount they need to in order to make progress in their project, and oftentimes this may not be the best solution for the longevity of a codebase.

These tools are straddling an awkward line that is optimized for no-one. Traditional DevOps are not software engineers and software engineers are not DevOps. By making infrastructure a software engineering problem, it puts all parties in an unfamiliar position.

I am not saying no-one is capable of using these tools well. The DevOps and software engineers I've worked with are more than capable. This is a matter of attention. If you look at what a DevOps engineer has to deal with day-in and day-out, the nuances of TypeScript or Go will take a backseat. And conversely, the nuances of, for example, a VPC will take a backseat to a software engineer delivering a new feature. The gap that the AWS CDK and Pulumi try to bridge is not optimized for anyone and this is how we get bugs, and more dangerously, security holes.

On the surface, the AWS CDK, Pulumi, and CDKTF seem like a step forward, but they bring so much complexity without really solving the issue of making infrastructure easier to create, modify, and maintain, that they are anything but progress.

The limits of HCL are also its strengths. In this way, Hashicorp HCL shines. Those limitations mean it is easy for humans and computers to read, write, and understand. With HCL, we allow humans to write Terraform code when it suits them but also allow computers to provide different views of that code as needed.

Keeping IaC as code is vital. We do not want to rewrite source control, losing the rich ecosystem. But we do want to allow the Terraform code to be manipulated in the way that suits that user best.

ClickOps enables more users

ClickOps is the idea of providing GUIs for manipulating infrastructure which produce code. There are a lot of ways that ClickOps can be achieved, but we think that any ClickOps solution should use HCL as its source and destination language.

ClickOps tools can integrate into the existing rich ecosystem of Terraform tooling and advanced users can read and write Terraform code by hand if they wish. The goal here is to increase the number of users that can comfortably modify infrastructure while not alienating those who are already comfortable. We think HCL is a language that can cross that chasm, but not because it's easy for everyone to use but because it is a good balance of being consumable by humans and computers. HCL is a great language for ClickOps.

We want ClickOps that allows for the creation and modification of infrastructure without losing any of the value we get of IaC. We still want to have pull requests before the change is applied. We want to utilize the powerful ecosystem of Terraform tooling. We want advanced users to be able to write Terraform code.

Imagine a world where you have a service that knows how to read and write Terraform code. It presents that code to the user as if it were an AWS console. They click through the GUI in order to make their changes and it writes the change out as Terraform code. The user can then make a pull request from the change. A reviewer then reads the code, as they would any other change. Or maybe they prefer to use the service to visualize the pull request.

Many users are familiar with the AWS console and would be able to make and review changes quickly, never having seen the Terraform code. Users that are more comfortable in Terraform could look at the generated code. We could create more than just write-once templates. An organization could have a template for creating their infrastructure in a testing environment. Once they are ready, they want to transition those changes to a production environment. A service could understand the difference between these two environments, take the testing infrastructure code and produce new code that has the changes for production. This doesn't have to be a one-time change, but every time an infrastructure change is validated in testing it can transform it into production, making the appropriate changes.

With ClickOps, we can have GUIs that have rich interfaces, preferably self documenting, enabling more users to work with infrastructure. By providing powerful templates, wizards, and intuitive interfaces, users are more likely to make the right changes. The end result is speed and stability. Software engineers can make infrastructure changes faster, not having to wait for support from operations, and they can also make them safer, the ClickOps platform providing guardrails on their changes. By using HCL as the underlying language for all of this, we get the benefits of a GUI without losing the benefits of IaC.

The future of ClickOps

We are curious what the community thinks.

Do you think we are way off?
Is the AWS CDK, Pulumi, and CDKTF actually the future?
What is your experience with colleagues when they help manage their own infrastructure?

We are hoping this blog post stirs up discussion. Even if you hate the idea, please let us know.

Terraform. GitOps. Lock Policies.

Josh Pollara — Fri, 02 Jun 2023 09:49:09 +0000

What is Terrateam?

Terrateam is Terraform continuous delivery. Purpose-built for GitHub with an expressive configuration file, popular third-party integrations, advanced features like OIDC, access controls, drift detection, and more. Self-Hosted and Cloud versions available.

⭐ Star us on GitHub: https://github.com/terrateamio/terrateam

Announcing Lock Policies

Strict locking by default

In a collaborative environment, it can be easy to forget to apply a change that has been merged or to merge a change that has been applied. Guaranteeing your infrastructure matches your code is one of the benefits of using Terrateam.

When a change is applied in a pull request, Terrateam acquires a lock on the directory that changed and requires it be merged into the main branch.
Similarly, if the change is merged, Terrateam acquires a lock on the directory and requires that it be applied. A change is either merged or applied to acquire a lock, and the other operation must be done to release the lock.

Safety guarantees can get in the way

Some directories are different. For example, it is common for Terraform repositories to have both development environments and production environments described in them. Using our access control feature, Terrateam can be configured such that anyone can modify development but production is locked down. Sometimes, to iterate faster, it makes sense to plan and apply development changes locally before making a pull request.

Because the development environment is sometimes run via Terrateam and sometimes managed outside of Terrateam, the safety guarantees can get in the way. Terrateam is too strict in these scenarios.

Lock Policies

To support this workflow, we've introduced a new
workflow configuration called lock_policy. The lock_policy option tells Terrateam under what situations it should acquire a
lock.

It has four modes:

strict - This is the default and matches the current behavior. If a user comments terrateam apply in the pull request or the change is merged, Terrateam acquires a lock on the directory until the complimentary operation is performed. We recommend all production directories keep this setting.
apply - This instructs Terrateam to only acquire a lock if the directory has been applied in Terrateam (terrateam apply). The lock will be released once the change is merged. If the change is just merged, Terrateam will not acquire a lock. This is what should be used in the scenario described above. The development directories should be set to lock_policy: apply, that way if they are applied outside of Terrateam, no lock is acquired.
merge - This instructs Terrateam to only acquire a lock if the directory has been merged. The lock will be released when the change is applied (terrateam apply). This is useful if a pull request is used as a playground in development and then closed when done, rather than merging.
none - Never acquire a lock.

Of course, there are other situations where these settings make sense other than the ones described here. But, be careful! Locking is fundamental to how Terrateam keeps code and infrastructure synchronized.

Configuration

The locking policy is defined in the workflows section. To set lock_policy to apply for all directories dev directories:

workflows:
  - tag_query: dev in dir
    lock_policy: apply

Docs

Check out the Terrateam Docs for more examples.

Terrateam Self-Hosted: The Enterprise Atlantis Alternative

Josh Pollara — Fri, 02 Jun 2023 09:43:17 +0000

What is Terrateam?

⭐ Star us on GitHub: https://github.com/terrateamio/terrateam

Announcing Terrateam Self-Hosted

I'm excited to introduce the self-hosted version of Terrateam designed for customers that need to meet strict security and compliance requirements or just prefer to host their own deployment.

Terrateam Self-Hosted provides the flexibility to deploy all of Terrateam to your own infrastructure giving you full control of your data.

Feature parity with Cloud

Terrateam Self-Hosted is 100% identical to Terrateam Cloud. Full feature parity.

Pricing

We're also releasing new pricing today. Here's the breakdown:

Self-Hosted

Free forever
Access to all features
Unlimited users
Unlimited usage
Same-day business support

Terrateam Cloud

Free for teams of two
$145/month (paid annually) or $175/month (paid monthly)
Access to all features
Unlimited users
Unlimited usage
Same-day business support

Same-day business support

Do we really have same-day business support? Even for Terrateam Self-Hosted?

Yes, we really have same-day business support for all of our plans. When a customer reaches out to us for support, we see this as a golden opportunity to improve Terrateam. Whether that's by automating a process, updating our documentation, or creating a new feature request in our backlog, it's a way to improve the developer experience.

Deploying Terrateam Self-Hosted

Deploying Terrateam is really easy. There are two components of Terrateam:

PostgreSQL Database
Terrateam Server

The easiest way to kick the tires is to use our Docker Compose
instructions. This gets you going in minutes. More production-grade deployment instructions (Kubernetes Helm Chart) are available.

Get started

The easiest way to get started is by signing up for Terrateam Cloud. If you need help onboarding please ping us on Slack.

CDKTF Frequently Asked Questions

Josh Pollara — Thu, 23 Feb 2023 11:27:52 +0000

CDKTF Frequently Asked Questions

What does CDKTF mean?

CDKTF stands for Cloud Development Kit for Terraform. The term CDK is used to describe tools which allow managing cloud infrastructure using common programming languages.

What problem does CDKTF solve?

Developers can manage their infrastructure using familiar programming languages in place of the Terraform HCL language. As well as using familiar programming languages, developers benefit from being able to use the ecosystem of their language of choice when developing infrastructure components.

CDKTF was initially developed in
collaboration with AWS which had released AWS CDK. AWS CDK turned out to be popular and developers really liked being able to work in their language of choice.

AWS CDK and CDKTF are similar to Pulumi, which was developed with the idea of managing infrastructure using familiar programming languages.

Key differences between the three offerings are as follows:

Pulumi has its own engine for planning and applying changes
AWS CDK generates AWS Cloud Assembly for planning and applying changes
CDKTF generates Terraform code (HCL) for planning and applying changes

What languages does CDKTF support?

CDKTF can be written in the following languages:

Is CDKTF production ready?

Yes. On Aug 1
2022, HashiCorp announced that CDKTF is Generally Available and ready for production usage. CDKTF is still a maturing technology, however, you should feel confident using it in a production environment.

What are the advantages of CDKTF?

Developers who are familiar with programming languages but not HCL can contribute to infrastructure changes.
Being able to express infrastructure in TypeScript, Python, Java, C#, or Go allows for leveraging the ecosystem of those languages.
The development environments (editors, IDEs, language servers, testing frameworks) are very mature.
Programming languages are more powerful than HCL.
More people are familiar with programming languages than HCL.

What are the disadvantages of CDKTF?

CDKTF is still maturing. The underlying engine which executes CDKTF is robust. One needs to have implementations of providers and resources in the CDKTF ecosystem, which is still growing.
Documentation, especially around providers, is lacking. One might have to read code in order to understand how to use a provider.
CDKTF is complicated. It requires converting code written in a programming language to HCL, which you could have written yourself.
CDKTF also adds new concepts (for example "stacks"), making it not a direct one-to-one translation of Terraform. This means those developers who are familiar with Terraform will need to learn CDKTF before they are effective.
With great power comes great responsibility. Programming languages are much more expressive than HCL. While HCL has its limitations, these limitations also mean the code has a ceiling of complexity. Programming languages do not have the same constraints.

Should I use CDKTF?

Like most technical decisions: it depends.

Whether or not CDKTF is the right decision for you is something you need to determine.

Here are some factors you should consider:

Is using CDKTF going to make it easier for developers in my organization to contribute infrastructure changes?
Do I have the innovation tokens to use a maturing technology?

CDKTF will continue to be developed and improved by HashiCorp, so if you like where it is now, it is probably a good decision.

Does CDTKTF require Terraform Cloud?

No. You can use CDKTF anywhere.

What language is CDKTF implemented in?

The CDKTF CLI is implemented in TypeScript. Providers and resources are implemented in TypeScript as well. jsii is used to compile the providers and resources to the supported languages.

How does CDKTF work?

The high-level process of how CDKTF works is as follows:

Developer writes code in their preferred programming language that CDKTF supports.
The code must create an instance of an "app".
The developer creates "stacks", "providers", and "resources" and adds them to the "app". This creates an internal representation of the infrastructure which can be turned into something else using the synth function.
After constructing the infrastructure objects, at the end the developer must call the synth function.
The user then executes cdktf diff, cdktf deploy, or cdktf synth.
The CDKTF CLI runs the code the developer wrote and the call to synth knows how to convert the app (which has the stack and resources and providers added to it) to the JSON representation of HCL.
The CDKTF CLI can then run terraform plan or terraform apply on the HCL.

It is possible that cdktf diff and cdktf deploy could perform the translation to Terraform HCL and execution in-memory, however cdktf synth will always produce Terraform HCL in a directory that you can manually run the Terraform CLI against.

Considering the following CDKTF code in TypeScript:

import { Construct } from "constructs";
import { App, TerraformStack } from "cdktf";
import * as Null from './.gen/providers/null';

class MyStack extends TerraformStack {
  constructor(scope: Construct, id: string) {
    super(scope, id);

    new Null.provider.NullProvider(this, 'test-provider');
    new Null.resource.Resource(this, 'test');
  }
}

const app = new App();
new MyStack(app, "cdktf-stack1");
app.synth();

On the last line, app.synth(), the entire app has been created and all providers and resources have been added to the stack. The synth call can then
produce Terraform HCL files and the CDKTF CLI can execute terraform plan or terraform apply.

How can I convert Terraform code to CDKTF?

The CDKTF CLI can convert Terraform files using cdktf
convert.

What is an app?

CDKTF code constructs an object called "app". This represents all of the infrastructure that will be generated in CDKTF. Stacks will be added to the app. And providers and resources will be added to the stack.

What is a stack?

Stacks represent a collection of infrastructure configuration that will be managed. You can think of them as somewhere between Terraform modules and workspaces. You might use a stack to represent the infrastructure in an AWS region and then create an instance of that stack per region. For example:

import { Construct } from "constructs";
import { App, TerraformStack } from "cdktf";

class MyStack extends TerraformStack {
  constructor(scope: Construct, id: string, region: string) {
    super(scope, id);

    // define resources here
  }
}

const app = new App();
new MyStack(app, "cdktf-stack1", "us-east-1");
new MyStack(app, "cdtff-stack2", "us-west-1");
app.synth();

Where can I learn more about CDKTF?

Using Multiple AWS IAM Roles

Josh Pollara — Fri, 17 Feb 2023 14:48:42 +0000

Access Control with IAM

One of the most important, and oftentimes most confusing, aspects of cloud infrastructure is access control. AWS IAM provides a way to secure access to AWS resources.

IAM roles are a key concept in AWS IAM that allow users to delegate access to AWS resources to other AWS accounts, AWS services, and external identities.

Using multiple AWS IAM roles in combination with Terraform allows you to manage access to resources in a more secure and scalable manner.

IAM Roles

AWS IAM roles that allow you to grant permissions to one or more AWS resources to a specific entity. An entity is just a way to reference another AWS account, AWS service, or third-party.

It's important to understand that IAM roles are not associated with specific users or groups. IAM roles are only associated with specific policies that grant access to AWS resources.

When an entity assumes an IAM role, it receives the permissions that are defined in the policy attached to that IAM role.

Benefits of IAM Roles

There are many advantages when using IAM roles vs. other access controls.

Static credentials are not required
Ability to grant permissions to third-parties without the need of a dedicated IAM user
Granular permissions for a specific use case
Leverage ephemeral resources like EC2 instances and Lambda functions

Using Multiple IAM Roles with Terraform

There are a few different ways to leverage AWS IAM roles with the Terraform AWS provider.

One common way is to use the assume_role parameter to specify which IAM role should be assumed
when running the Terraform CLI. This can be defined on the provider or resource level.

A real-world example is to use separate IAM roles for a production and qa environment.

provider "aws" {
  assume_role {
    role_arn = "arn:aws:iam::123456789012:role/production-role"
  }
}

provider "aws" {
  assume_role {
    role_arn = "arn:aws:iam::123456789012:role/qa-role"
  }
}

Once defined, you can specify which provider block to use for each resource.

resource "aws_instance" "example" {
  ami           = "ami-4c55b159cbfafe1fe"
  instance_type = "t2.micro"
  provider      = aws.qa
}

Another common way to leverage AWS IAM roles with the Terraform AWS provider is by using the AWS CLI to assume a role before executing Terraform commands. The aws sts assume-role command provides a very easy way to return a set of temporary security credentials for a specific IAM role.

aws sts assume-role \
--role-arn "arn:aws:iam::123456789012:role/qa-role" \
--role-session-name qa-role

AWS will return a set of temporary credentials that can then be used to create AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables which the Terraform AWS provider consumes. This is how
the Terrateam GitHub Action assumes a user-defined IAM role. This method doesn't require you to specify role_arn in your Terraform code.

By using separate IAM roles in each environment, you have the ability to define each role with its own set of permissions. The qa environment IAM role would not need access to the production environment
and vice versa.

Benefits of Using Multiple IAM Roles with Terraform

As you can already start to see, there are many benefits of using multiple IAM roles with Terraform.

Security: Separate IAM roles improves your security posture by limiting access to your AWS resources only to entities that require access.
Simplicity: It's easier to understand which entities have access to AWS resources by using separate IAM roles for different environments. Having the ability to easily manage access to resources based on
internal needs helps keep you more organized and secure.
Compliance: Industry standards, best practices, and various other regulations like HIPAA and GDPR often require having the ability to enforce least privilege access to infrastructure. Separate IAM roles
give you the building blocks to satisfy these requirements.
Flexibility: Having the ability to create granular access rules based on the needs of an organization creates flexibility and scalability for engineering teams.
Auditing: Monitoring account activity by separate IAM roles for each environment, entity, and third-party gives you the ability to easily audit access to resources.

Using Multiple IAM Roles with Terrateam

Using multiple IAM roles in a Terraform repository with Terrateam is easy to accomplish by creating simple configuration in your .terrateam/config.yml.

Take the following directory structure as an example:

josh@elmer:~/terraform $ tree
.
├── modules
│   └── ec2
│       └── v1
│           └── main.tf
├── production
│   └── main.tf
└── qa
    └── main.tf

5 directories, 3 files
josh@elmer:~/terraform $

In this Terraform repository, we have three directories:

modules: Custom-written Terraform modules that's referenced throughout the repository
production: Defined Terraform resources for the Production environment
qa: Defined Terraform resources for the QA environment

An ideal workflow for this repository layout is to use separate IAM roles for each environment. By creating a .terrateam/config.yml at the root of the repository, one can safely assume separate IAM roles
before any Terraform CLI commands are issued.

workflows:
  - tag_query: dir:production
    plan:
      - type: oidc
        provider: aws
        role_arn: "arn:aws:iam::123456789012:role/terrateam-production-role"
      - type: init
      - type: plan
    apply:
      - type: oidc
        provider: aws
        role_arn: "arn:aws:iam::123456789012:role/terrateam-production-role"
      - type: init
      - type: apply
  - tag_query: dir:qa
    plan:
      - type: oidc
        provider: aws
        role_arn: "arn:aws:iam::123456789012:role/terrateam-qa-role"
      - type: init
      - type: plan
    apply:
      - type: oidc
        provider: aws
        role_arn: "arn:aws:iam::123456789012:role/terrateam-qa-role"
      - type: init
      - type: apply

The above configuration will execute the following steps in your isolated Terrateam GitHub Actions runtime environment

Issue a secure token using the official GitHub OIDC Identity Provider. This token is used to authenticate against your AWS account. After a successful authentication, the Terrateam action will
automatically assume the IAM role defined in the role_arn configuration.
After successful authentication to AWS, Terrateam will execute an aws sts command in the GitHub Actions runtime environment to generate short-lived credentials for the IAM role defined
in the role_arn depending on which directory contains Terraform-related changes in the pull request.
Execute the necessary Terraform CLI commands in the directory where the Terraform-code change lies
- terraform init
- terraform plan or terraform apply depending on the operation

Separate IAM roles are defined and used by leveraging Terrateam workflows. With custom workflows, the user can define many customizations to the steps
Terrateam will take depending on what file, directory, or workspace is modified in a pull request.

Conclusion

Multiple AWS IAM roles with Terraform allows your team to properly manage access to your AWS resources in many environments. This is instrumental in achieving a proper security posture and creates a strong
foundation for scalable IAM. With the help of Terrateam, you can create and enforce specific IAM roles to be used against resources laid out in your Terraform repository.

Give Terrateam a try and start leveraging multiple IAM roles with OIDC today.

Creating an AWS Lambda Function with Terraform

Josh Pollara — Thu, 16 Feb 2023 12:18:06 +0000

Creating an AWS Lambda Function with Terraform

Writing the proper Terraform code for an AWS Lambda function may seem like a daunting task but it doesn't have to be.

While there are a few moving parts to the process, I'll explain how to deploy a Lambda function to a VPC that can access other resources within your VPC using
appropriate IAM permissions.

Full code example here: https://github.com/terrateamio/terraform-aws-lambda-example

`main.tf`

The below resources are examples and should be adjusted to suit your needs. The code can be added to your Terraform repository in a file named main.tf.

Provider

Add the aws provider

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.0"
    }
  }
}

provider "aws" {
  region = "us-west-2"
}

VPC

Create a new VPC

resource "aws_vpc" "example" {
  cidr_block = "10.0.0.0/16"
  enable_dns_hostnames = true
  enable_dns_support = true

  tags = {
    Name = "example-vpc"
  }
}

Subnets

Create at least one subnet

resource "aws_subnet" "example" {
  cidr_block = "10.0.1.0/24"
  vpc_id = aws_vpc.example.id
  availability_zone = "us-west-2b"

  tags = {
    Name = "example-subnet"
  }
}

Security Group

Create a security group

resource "aws_security_group" "example" {
  name_prefix = "example-sg"
  vpc_id = aws_vpc.example.id

  ingress {
    from_port = 0
    to_port = 65535
    protocol = "tcp"
    cidr_blocks = ["10.0.1.0/24"]
  }

  egress {
    from_port = 0
    to_port = 65535
    protocol = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

IAM Role and Policy

Create an IAM role and policy

resource "aws_iam_role" "example" {
  name = "example-role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "lambda.amazonaws.com"
      }
    }]
  })
}

resource "aws_iam_policy" "example" {
  name        = "example-policy"
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect = "Allow"
      Action = [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ]
      Resource = ["arn:aws:logs:*:*:*"]
    },{
      Effect = "Allow"
      Action = [
        "ec2:CreateNetworkInterface",
        "ec2:DescribeNetworkInterfaces",
        "ec2:DeleteNetworkInterface"
      ]
      Resource = ["*"]
    }]
  })
}

resource "aws_iam_role_policy_attachment" "example" {
  policy_arn = aws_iam_policy.example.arn
  role = aws_iam_role.example.name
}

This IAM role and policy allows your Lambda function to create, describe, and delete network interfaces, and create and push logs to CloudWatch.

Lambda Function

The last step is to create your actual Lambda function

resource "aws_lambda_function" "example" {
  function_name    = "example-lambda"
  filename         = "lambda_function_payload.zip"
  source_code_hash = filebase64sha256("lambda_function_payload.zip")
  handler          = "index.handler"
  role             = aws_iam_role.example.arn
  runtime          = "nodejs14.x"
  vpc_config {
    subnet_ids = [aws_subnet.example.id]
    security_group_ids = [aws_security_group.example.id]
  }
}

Verify Changes

View the changes to be deployed to your AWS account

$ terraform init
$ terraform validate
$ terraform plan

Apply Changes

Create the AWS resources defined in your main.tf

$ terraform apply

Conclusion

By following these steps, you can deploy an AWS Lambda function to a VPC with all the proper IAM permissions. Check out Terrateam to learn how to safely plan and apply infrastructure changes alongside other code and with your teammates.