Forem: Matheus Cunha

Terraform Modules: Atomic Design

Matheus Cunha — Sat, 28 Aug 2021 10:10:48 +0000

Intro

Following The Pragmatic Programmer mantra, I do my best to ...

Learn at least one new language every year. Different languages solve the same problems in different ways. By learning several different approaches, you can help broaden your thinking and avoid getting stuck in a rut.

Not necessarily to show it off or to be capable of talking about random technologies, but to expand and train my problem-solving skills, to get new perspectives when approaching a challenge.

We might not notice it but when we learn (or have learned) to code we aren't just learning to type some characters that a compiler/interpreter can understand, it is a new way of thinking, a new way of breaking down solutions (into sequential steps).

It doesn't matter whether you ever use any of these technologies on a project, or even whether you put them on your resume. The process of learning will expand your thinking, opening you to new possibilities and new ways of doing things.
The cross-pollination of ideas is important;

As someone who works intensively with infrastructure components (servers, databases, Kubernetes, CI/CD, etc) I aimed for something completely different this year. Something that stands on a whole different spectrum of the system, this year I decided to learn Flutter.

In-a-nutshell, Flutter is a better React Native. A framework that enables implementation of GUI applications for multiple platforms with a single code base.

Then it reminded me a discussion I had with a friend in the past about React components and the Atomic Design methodology, which helps to structure web components into modules.

In the Atomic Design methodology, the granularity of modules is distinguished by using chemistry inspired names: atoms, molecules and organisms.

Then the connection of the ideas from

Pragmatic Programmer's cross-pollination to
Atomic Design (on Flutter components) to
Terraform modules

came almost like a thunderbolt, striking me with this insight when I was working with a huge legacy Terraform code base refactoring with lots of code duplication (read: copy+paste, "we fix it later", then the author quits the company and
never fix anything).

Although initially proposed as a Web UI methodology, Infrastructure as Code tools such as Terraform that makes heavy usage of modules can benefit from Atomic Design to improve its code reusability and massively reduce duplication.

Details

The Atomic Design methodology proposes five distinct levels, listed from the finest to the thickest:

Atom;
Molecules;
Organisms;
Templates;
Pages.

However, to extract the gist, we'll only be focusing on Atoms, Molecules, and Organisms (from 1. to 3.). Templates and Pages are too domain-specific focused on Web UI development.

Atoms

Atoms represent the finest grain in terms of granularity in the design. When referring specifically to its implementation in Terraform a resource and a small scoped single-purpose module could be used interchangeably.

Sometimes the idea of turning a simple resource into a module makes sense to ease parameterization and reusability, especially when it is necessary to parse inputs. Although, due to its extreme limited scope it might not look attractive
to convert the resource into a module at first sight, on the long run it pays off to do so in order to achieve scalability and reproducibility.

e.g.:

data "aws_route53_zone" "default" {
  zone_id = var.zone_id
  name    = var.zone_name
}

resource "aws_route53_record" "default" {
  zone_id = data.aws_route53_zone.default.zone_id
  name    = var.name

  ttl  = var.ttl
  type = var.record_type

  records = var.records

  dynamic "alias" {
    for_each = [var.alias]

    content {
      name = each.value.name
      zone_id = try(each.value.zone_id, data.aws_route53_zone.default.zone_id)

      evaluate_target_health = lookup(
        each.value,
        "evaluate_target_health",
        false,
      )
    }
  }
}

In this case, even though aws_route53_record is a simple resource that might feel too narrow in scope to write a module, the implementation of the module allows to bundle the AWS Route53 Zone data source together, which helps to:

provide a simpler contract by allowing the usage of zone_name alone;
validate the zone_name input, ensuring that a given zone_name corresponds to an actual existing and valid AWS resource;
same goes to zone_id, which will feel (and oftentimes, be) redundant, when specified as an input Terraform will read the data from AWS API ensuring consistency.

e.g.:

module "awesome_dns_fqdn" {
  source = "path/to/modules/atoms/aws_route53_record"
  version = "~> 1.0"

  name      = "record.example.com"
  zone_name = "example.com."

  record_type = "A"
  records     = ["1.2.3.4"]
}

Hence, resources and modules are sometimes interchangeable as they deliver the same outcome for the finest resources' granularity.

Molecules

When groups of atoms are bounded together, they create a molecule which is the smallest fundamental unit of a compound.

Contrary to the original Atomic Design for Web UI, in Terraform, Atoms are useful on their own. However, the usage of atoms comes with a high price on scalability: code duplication. Actually, duplication is an understatement, it is more like code exponentiation (more on this later).

Implementation example

Suppose we are creating a public facing API Gateway that needs a DNS record.

Let's compose it with the previous example:

data "aws_route53_zone" "default" {
  name = var.zone_name
}

module "awesome_api_gateway_certificate" {
  source  = "terraform-aws-modules/acm/aws"
  version = "~> v3.0"

  domain_name = var.domain_name
  zone_id     = data.aws_route53_zone.default.zone_id

  wait_for_validation = true
}

module "awesome_api_gateway" {
  source = "terraform-aws-modules/apigateway-v2/aws"
  version = "~> 1.0"

  name          = var.api_gateway_name
  description   = var.api_gateway_description
  protocol_type = "HTTP"

  cors_configuration = {
    allow_headers = [
      "content-type",
      "x-amz-date",
      "authorization",
      "x-api-key",
      "x-amz-security-token",
      "x-amz-user-agent",
    ]
    allow_methods = ["*"]
    allow_origins = ["*"]
  }

  # Custom domain
  domain_name                 = var.domain_name
  domain_name_certificate_arn = module.awesome_api_gateway_certificate.acm_certificate_arn

  # Routes and integrations
  integrations = var.api_gateway_integrations
}

module "awesome_dns_fqdn" {
  source  = "path/to/modules/atoms/aws_route53_record"
  version = "~> 1.0"

  name    = var.domain_name
  zone_id = data.aws_route53_zone.default.zone_id

  record_type = "CNAME"
  alias     = {
    name    = module.awesome_api_gateway.apigatewayv2_domain_name_configuration[0].target_domain_name
    zone_id = module.awesome_api_gateway.apigatewayv2_domain_name_configuration[0].hosted_zone_id
  }
}

This helps illustrating an example in which the aws_route53_record atom could be easily replaced with its equivalent resource and it would still provide the same outcome.

Commonly it is possible to use module and resource interchangeably as Atoms, the decision of whether or not to implement a module is ultimately defined by the need of parsing and/or validating the inputs (variables).

Usage example

module "awesome_lambda" {
  source  = "path/to/modules/molecules/aws_lambda_function"
  version = "~> 1.0"

  function_name = "awesome"
  description   = "An Awesome lambda function for the Awesome API Gateway"
  handler       = "index.lambda_handler"
  runtime       = "python3.8"

  # Incomplete implementation, don't use this on production
}

module "another_awesome_lambda" {
  source  = "path/to/modules/molecules/aws_lambda_function"
  version = "~> 1.0"

  function_name = "awesome"
  description   = "An Awesome lambda function for the Awesome API Gateway"
  handler       = "index.lambda_handler"
  runtime       = "python3.8"

  # Incomplete implementation, don't use this on production
}

module "awesome_api_gateway" {
  source = "path/to/modules/molecules/aws_api_gateway"
  version = "~> 1.0"

  domain_name = "record.example.com"
  zone_name   = "example.com."

  api_gateway_name = "awesome-api-gateway"
  api_gateway_description = "An Awesome API Gateway"

  api_gateway_integrations = {
    "POST /" = {
      lambda_arn             = module.awesome_lambda.function_arn
      payload_format_version = "2.0"
    }

    "$default" = {
      lambda_arn = module.another_awesome_lambda.function_arn
    }
  }
}

As you probably have already realized, when the level of abstraction goes up (e.g. from atom to molecule) the module implementation is in itself a good implementation example (i.e. as in community modules examples).

They help to self-document the usage and implementation of a given module and through generic implementations it allows us to have multiple molecules implementing multiple distinct use-cases. e.g.:

Public API Gateway with DNS record + TLS certificate;
Public API Gateway v1, no DNS record;
Private API Gateway.

Why would we chose to implement multiple times the Atom modules in order to create multiple distinct use-cases? We are getting closer to the code exponentiation problem and solution proposal. Can you feel it?

Organisms

Going further, the example of composition for molecules can have its hard-coded values turned into variables in order to compose an Organism, which can facilitate the implementation of the same definition across different environments. Thus, achieving reproducibility as well as the Factor X. of the Twelve Factor App.

However, it is important to note that the level of abstraction between Organisms and Molecules can be easily confused or misunderstood. Generally speaking, as a
rule of thumb an Organism is the composition of Molecules that allow parameterization for business or domain-specific logic (e.g. the actual awesome_api configuration).
Therefore, in comparison with the previous, Organisms (usually) have a lower level of generalization since they are business-specialized modules.

Iterating over our implementation example, the Organism would implement the awesome_api, creating the following resources:

AWS Lambda function;
AWS API Gateway;
TLS Certificate on AWS ACM;
DNS record on AWS Route53.

By implementing the previous examples as organisms we:

reduce the amount of boilerplate code;
foster reusability of modules;
provide a simple interface for non-operators to manage TF code.

When you sum it all up, you will notice that it is all about autonomy and "DevOps" through encouragement of self-service Ops. One wouldn't need to know a lot about Terraform to grab a module and pass some parameters to it, followed by
a code review process Operators and Software Developers can manage the Infrastructure in harmony, together. (:

Code Exponentiation? What?

Read that as a dramatization of the "code duplication" term.

When it comes to Infrastructure as Code, there is no easy way around the jungle of resources that grows over time. Fast pacing tech companies are "moving fast and breaking things", oftentimes the Operators are worried about a massive
amount of challenges at once: keep the servers up and running, with a consistent response time, low error rate, and all that playbook from Google's SRE wisdom.

All things considered, a good Infrastructure as Code design is generally a first-world problem. However, as the time passes it evolves into a real issue that slows down the implementation of resources as code. Either that or there
will be a huge ton of copy+paste to keep up with the pace, followed by a routine of find+replace when changes are applied, then harder to track pull requests and slower code reviews.

Lets take our awesome_api example and scale it up to multiple environments followed by a second awesome_api:

.
├── development
│   ├── an-awesome-api
│   │   └── main.tf
│   └── another-awesome-api
│       └── main.tf
├── staging
│   ├── an-awesome-api
│   │   └── main.tf
│   └── another-awesome-api
│       └── main.tf
└── production
    ├── an-awesome-api
    │   └── main.tf
    └── another-awesome-api
        └── main.tf

In order to replicate the configuration and ensure consistency, the following is way simpler to implement (and review) than copy+paste huge chunks of Terraform definitions

module "awesome_api" {
  source = "path/to/modules/organisms/aws_lambda_with_api_gateway"
  version = "~> 1.0"

  domain_name = "record.example.com"
  zone_name   = "example.com."

  lambda_functions = [
    # Index 0 -- An Awesome Lambda Function, used for POST
    {
      name        = "an-awesome"
      description = "An Awesome lambda function for the Awesome API Gateway"
      handler     = "an_awesome.lambda_handler"
      runtime     = "python3.8"
    },
    # Index 1 -- Another Awesome Lambda Function, used as $default
    {
      name        = "another-awesome"
      description = "Another Awesome lambda function for the Awesome API Gateway"
      handler     = "another_awesome.lambda_handler"
      runtime     = "python3.8"
    },
  ]

  api_gateway_name = "awesome-api-gateway"
  api_gateway_description = "An Awesome API Gateway"

  api_gateway_integrations = {
    "POST /" = {
      lambda_function_index  = 0
      payload_format_version = "2.0"
    }

    "$default" = {
      lambda_function_index = 1
    }
  }
}

Conclusion

At the end of the day we get an ugly Terraform state containing many

module.something.module.something_else.module.yet_another_thing...

But the productivity boost gained by merging modules based on context is a worth investment. Especially for huge Terraform repositories with multiple teams collaborating and managing a lot of resources.

Cross-team collaboration is fostered by applying the Atomic Design methodology for Terraform modules, code reusability becomes an important factor over copy+paste and the repository gravitates towards the DRY principle.

Same post, different places

Reddit r/Terraform post: Terraform Modules: Atomic Design

Real-life Terraform Refactoring Guide

Matheus Cunha — Mon, 16 Aug 2021 18:06:35 +0000

Intro
How to break a big ball of mud? STRANGLE IT
The mono-repository (monorepo) approach to Legacy
1. Splitting the modules sub-path to its own repository
2. Let’s start strangling the repository
  1. Import state? Remove state and code from what? Where?

Intro

As reality hits, the unavoidable fact of dealing with a hard-to-manage Terraform Big ball of mud code base comes in. There is no way around natural growth and evolution of code bases and the design flaws that come with it. Our Agile mindset is to “move fast and break things”, implement something as simple as possible and let the design decisions for the next iterations (if any).

Refactoring Terraform code is actually as natural as developing it, time and time again you will be face situation where a better structure or organization can be achieved, maybe you want to upgrade from a home-made module to an open-source/community alternative, maybe you just want to segregate your resources into different states to speed-up development. Regardless of the goal, once you get into it, you will realize that Terraform code refactoring is actually a basic missing step on the development process that no one told you before.

As the Suffering-Oriented Programming mantra dictates:

“First make it possible. Then make it beautiful. Then make it fast.”

So, time to make the Terraform code beautiful!

How to break a big ball of mud? STRANGLE IT

<joke>Martin Fowler has already written everything there is to write about (early 2000s) DevOps, Agile, and Software Development. Therefore, we could reference Martin Fowler for virtually anything Software related</joke>, but really, the Refactoring book is THE reference on this subject.

Martin Fowler shared the Stangler (Fig) Pattern, which describes a strategy to refactor a legacy code base by re-implementing the same features (sometimes even the bugs) on another application.

[…] the huge strangler figs. They seed in the upper branches of a tree and gradually work their way down the tree until they root in the soil. Over many years they grow into fantastic and beautiful shapes, meanwhile strangling and killing the tree that was their host.

This metaphor struck me as a way of describing a way of doing a rewrite of an important system.

In this document we are going to follow the same idea:

implement the same feature on a different Terraform composition;
migrate the Terraform state;
delete (kill) the previous implementation.

The mono-repository (monorepo) approach to Legacy

Let’s suppose that your Terraform code base is versioned in a single repository (a.k.a. monorepo), following the random structure displayed below (just to help illustrate)

.
├── modules/    # Definition of TF modules used by underlying compositions
├── global/     # Resources that aren't restricted to one environment
│   ├── aws/
├── production/ # Production environment resources
│   └── aws/
└── staging/    # Staging environment resources
    └── aws/

In this example, each directory corresponds to a Terraform state. In order to apply changes, you have to walk to a path and execute terraform.

The structure on this example repository was created a few hypothetical years ago when the number of existing microservices and resources (DB, message queues, etc) was significantly smaller. At the time, it was feasible to keep Terraform definitions together because it was easier to maintain, Cloud resources were managed with one-shot!

As time went by, the number of Products and the team grew, and engineers started facing concurrency issues: Terraform lock executions on shared storage when someone else is running terraform apply as well as a general slowness on every execution since the number of data sources to sync is frightening.

A mono-repository approach is not necessarily bad, versioning is actually simpler when performed in one single repository. Ideally, there won’t be many changes on the scale of GiB meaning that it is safe to proceed on this one as long as the Terraform remote states are divided.

Splitting the `modules` sub-path to its own repository

One thing to mention though is the modules sub-path, this one could be stored in a different git repository to leverage its own versioning. Since Terraform modules and their implementations don’t always evolve at the same pace, keeping two distinct version trees is beneficial. Additionally, a separated repository for Terraform modules allows the specification of “pinned versions”, e.g.:

module "aws_main_vpc" {
    source = "git::https://github.com/terraform-aws-modules/terraform-aws-vpc.git?ref=2ca733d"
    # Note the ref=${GIT_REVISION_DIGEST}
}

That reference for a module’s version should always be specified, regardless if it comes from an internal/private repository or public. When you specify the version, you are ensuring reproducibility.

Therefore, let’s move the modules sub-path to another git repository, following instructions from this StackOverflow answer so that the git commit history is preserved:

0.

Walk to the monorepo path and create a branch from the commits at monorepo/modules path

MAIN_BIGGER_REPO=/path/to/the/monorepo
cd "${MAIN_BIGGER_REPO}"
git subtree split -P modules -b refact-modules

1.

Create the new repository

mkdir /path/to/the/terraform-modules && cd $_
git init
git pull "${MAIN_BIGGER_REPO}" refact-modules

2.

Link the new repository to your remote Git (server) and push the commits

git remote add origin <git@git.com:user/terraform-modules.git>
git push -u origin master

3.

Cleanup the history related to modules from $MAIN_BIGGER_REPO [OPTIONAL]

cd ${MAIN_BIGGER_REPO}
git rm -rf modules
git filter-branch --prune-empty \
    --tree-filter "rm -rf modules" -f HEAD

Let’s start strangling the repository

Now that a substantial piece of code was moved somewhere else, it is time to put the Stangler (Fig) Pattern in practice.

Move all the existing content as-is to the legacy sub-path, keeping the same repository and change history (commits). It also allows applying the legacy code as it used to be from one of those paths.

.
└── legacy
    ├── global
    │   └── aws
    ├── production
    │   └── aws
    └── staging
        └── aws

Once the content is moved to legacy, the idea is to follow the Boy Scout rule in order to strangle the legacy content little by little (unless you are
really committed to migrating it all at once, which is going to be exhaustive).

The Boy Scout rule goes like this:

every time a task that involves deprecated code appears, we implement it on the new structure;
import the Terraform state to keep the Cloud resources that a given code represents/describes;
remove the state and the code from legacy.

Until there is nothing left inside legacy (or there are only unused resources/left-behinds that could be destroyed/garbage collected either way).

Import state? Remove state and code from what? Where?

That will depend on the kind of resource we are migrating from the remote state, on the bottom of each resource on Terraform’s provider documentation you can find a reference command to import existing resources into your Terraform code specification. e.g.: AWS RDS DB instance.

Suppose we want to replace the code of the AWS RDS Aurora defined in production/aws and then re-implement the same using the community module. After creating the corresponding sub-path to the monorepo according to your preference, provisioning the bucket and initializing the Terraform backend:

1.

Implement the definition of the community module github.com/terraform-aws-modules/terraform-aws-rds-aurora with the closest parameters from the existing one; e.g.:

module "aws_aurora_main_cluster" {
 source  = "terraform-aws-modules/rds-aurora/aws"
 version = "~> 5.2"

 # ...
}

2.

Import the Terraform states from the previous (existing) cluster

terraform import 'aws_aurora_main_cluster.aws_rds_cluster.this[0]' main-database-name
terraform import 'aws_aurora_main_cluster.aws_rds_cluster_instance.this[0]' main-database-instance-name-01
terraform import 'aws_aurora_main_cluster.aws_rds_cluster_instance.this[1]' main-database-instance-name-02

# ...

then if you haven’t yet and would like to “match reality” between the existing and the specified resource, run terraform plan a few times and adjust the parameters until Terraform reports:

No changes. Your infrastructure matches the configuration.

3.

Last but not least, remove the corresponding resources from the legacy Terraform state so that it doesn’t try to keep track of the changes and also don’t try to destroy once the resource definition is no longer in that code base:

# Hypothetical name of the resource inside production/aws/main.tf
terraform state rm aws_rds_cluster.default \
    'aws_rds_cluster_instance.default[0]' 'aws_rds_cluster_instance.default[1]'

# ...

once that is performed, feel free to remove the corresponding resource’s definition from the legacy code.

resource "aws_rds_cluster" "default" {
    # ...
}

resource "aws_rds_cluster_instance" "default" {
    count = var.number_of_database_instances

    # ...
}

Terraform Design Best Practices

Matheus Cunha — Sat, 07 Aug 2021 10:10:55 +0000

Intro
1. “Bad Idea” capitalized!
Design
1. Shallow “tree” of shared resources
2. Product areas (a.k.a. Business capabilities) structure and ownership
3. Shared resources, organized around technologies
4. Files inside the composition?
5. What about Terraform modules?

Intro

As someone who believes in empowering people and distributing power in order to achieve higher outcomes I always felt that the best existing best-practices proposals don’t touch some key aspects (IMHO) on code evolution and business structures.

Therefore, this design document shall compose on the previous including some self-service Ops and micro-services spice to the mix.

On Terraform best practices great insights on how to write code inside a module is provided, e.g. naming conventions, Terraform file naming.

We can’t leave Terragrunt epic blog post unmentioned:

5 Lessons Learned From Writing Over 300,000 Lines of Infrastructure Code;

As well as the Terragrunt documentation pointing “one of the most important lessons” is that:

large modules should be considered harmful. That is, it is a Bad Idea to define all of your environments (dev, stage, prod, etc), or even a large amount of infrastructure (servers, databases, load balancers, DNS, etc), in a single Terraform module. Large modules are slow, insecure, hard to update, hard to code review, hard to test, and brittle (i.e., you have all your eggs in one basket).

“Bad Idea” capitalized!

Which is totally true, as this “Bad Idea” usually coming from a lack of care towards Terraform code design tend to be harmful in the long run, with a tendency towards making the implementation a big ball of mud.

A Big Ball of Mud is a haphazardly structured, sprawling, sloppy, duct-tape-and-baling-wire, spaghetti-code jungle. These systems show unmistakable signs of unregulated growth, and repeated, expedient repair. Information is shared promiscuously among distant elements of the system, often to the point where nearly all the important information becomes global or duplicated.

The overall structure of the system may never have been well defined.

Oftentimes, Terraform code implementation fluctuate towards mono-repositories (a.k.a. monorepos) containing all the specification in a single place. In order to tame the chaos, the Terraform state needs to be at least sub-divided into logical sections.

Design

Shallow “tree” of shared resources

Following the recommendations for structuring code one of the proposals is to keep a shallow “tree” of resources and modules. This tree produces a small and clear distribution of Terraform code.

Why a shallow “tree” of resources? It helps achieving a short amount of resources and modules that result in a small remote state file. With a small remote state we speed-up the development process and reduce waste (Muda in the Toyota 3M model), as the shallow tree enables faster executions of Terraform (less data to sync and compare).

The granularity level will be defined for each specific case (no silver bullet) balancing the smallest and most feasible composition possible.

Product areas (a.k.a. Business capabilities) structure and ownership

Ideally, the composition level would be organized around Product Areas (either squads/crews or guilds) with a fallback to shared technologies (e.g. vpc, databases). Therefore, Terraform compositions are designed around what Martin Fowler calls “Business capabilities” in micro-services terminology, ideally the Terraform composition will follow the organizational structure so that each team “owns” (in both senses: ownership and freedom) its own state.

The main goal here is to structure the Terraform code as a reflection of the organization so that is fosters self-service Ops. If the Infrastructure as Code is mature enough to the point of having well-described Terraform modules, everyone should be empowered to define these modules by setting the parameters according to their needs, without centralizing power on a Operations team.

The resource composition must gravitate towards the following (ordered by priority from higher to lower):

Product Areas (ownership) directory structure:
1. squad/crew OR guild;
2. product.
Shared resources, around technologies.

Looking on the structure from bottom-up it starts from the product and then attributes the product to a crew through the directory tree.

e.g.:

# Squad or Crew
red-team
└── payment     # Product (i.e. micro-service) name
    └── main.tf # Any resource used by the payment product

# Guild (organized around technology)
back-end
└─ monolith    # Shared application in terms of ownership
   └── main.tf # Cloud resources used by the monolith

On the example above, we can’t ignore that monolith is a product with shared ownership among back-end developers and therefore it is organized to follow the business structure.

The structure is inspired on Terragrunt’s best-practices to some extend. However, it distinct from Terragrunt proposal in the way resources are divided, rather than organizing resources exclusively around technologies.

Shared resources, organized around technologies

Oftentimes in organizations we will face shared resources among products, there is no way around reality. e.g. a shared VPC or SQL database.

However, these situations should be the exception and not the norm. Dealt similar to the organization of Terraform compositions around guilds/technologies.

platform # as in Platform Engineering
└── vpc
    └── main.tf

back-end
└── database
    └── main.tf

Files inside the composition?

Ideally the files in the sub-directory (which specify the composition) are going to partially follow this spec and include data.tf, terraform.tf and providers.tf on top of that.

main.tf: contains locals, module and resource definitions;
variables.tf: contains declarations of variables (i.e. inputs/parameters) used in main.tf;
data.tf: contains data-resources for input data used in main.tf;
outputs.tf: contains outputs from the resources created in main.tf;
providers.tf: contains provider and provider’s versions definitions;
terraform.tf: contains the terraform back-end (e.g. remote state) definition;

What about Terraform modules?

Terraform modules are containers for multiple resources that are used together to achieve a shared goal. Modules can be used to create lightweight abstractions, facilitating reusability and distribution of Terraform code.

Therefore, we assume that the following are anti-patterns that make Terraform modules’ reusability difficult:

Configuration of Terraform Providers inside a module;
Implementation of Business logic and/or hard-coded parameters in a module;
Default values are specified in optional variables instead of hard-coding;
Modules should be self-contained and provide a clear contract. Dependencies (pre-existing resources) must be specified through required variables.
Modules must serve to a singular purpose. Multiple purpose must be achieved through composability of modules and not by “monolithic” modules.

Modules are abstractions that should be used to reduce the amount of code duplication, implementing the DRY (don’t repeat yourself) principle.

On top of that, modules are an important factor to reduce the parity among environments, which helps to better address the Twelve-Factor App model in regards to Factor X (ten).

Forem: Matheus Cunha

Terraform Modules: Atomic Design

Intro

Details

Atoms

Molecules

Implementation example

Usage example

Organisms

Code Exponentiation? What?

Conclusion

Same post, different places

Real-life Terraform Refactoring Guide

Table of Contents

Intro

How to break a big ball of mud? STRANGLE IT

The mono-repository (monorepo) approach to Legacy

Splitting the `modules` sub-path to its own repository

0.

1.

2.

3.

Let’s start strangling the repository

Import state? Remove state and code from what? Where?

1.

2.

3.

Terraform Design Best Practices

Table of Contents

Intro

“Bad Idea” capitalized!

Design

Shallow “tree” of shared resources

Product areas (a.k.a. Business capabilities) structure and ownership

Shared resources, organized around technologies

Files inside the composition?

What about Terraform modules?

Forem: Matheus Cunha

Terraform Modules: Atomic Design

Intro

Details

Atoms

Molecules

Implementation example

Usage example

Organisms

Code Exponentiation? What?

Conclusion

Same post, different places

Real-life Terraform Refactoring Guide

Table of Contents

Intro

How to break a big ball of mud? STRANGLE IT

The mono-repository (monorepo) approach to Legacy

Splitting the modules sub-path to its own repository

0.

1.

2.

3.

Let’s start strangling the repository

Import state? Remove state and code from what? Where?

1.

2.

3.

Terraform Design Best Practices

Table of Contents

Intro

“Bad Idea” capitalized!

Design

Shallow “tree” of shared resources

Product areas (a.k.a. Business capabilities) structure and ownership

Shared resources, organized around technologies

Files inside the composition?

What about Terraform modules?

Splitting the `modules` sub-path to its own repository