Forem: Jorge Tovar

How to set up Single Sign-On in AWS (IAM Identity Center)

Jorge Tovar — Mon, 08 Sep 2025 16:44:59 +0000

When I was at university, I met some programmers who were finishing their software engineering degrees. Some of them were already working, and they gave me what is still one of the best pieces of advice I’ve ever received about this craft:

“If you want to be good, stick with university knowledge. If you want to be great, become a lifelong learner.”

When I started learning AWS, having a personal account was the single best investment I made. It felt risky at first—you need to add your credit card, and you might accidentally leave an EC2 instance running or overspend on a machine. (That’s why budgets, policies, and guardrails are so important—but that’s a topic for another post.)

Once I got past that initial fear, I realized it was a great time to start using multiple accounts. Not only could I access more free credits, but I also unlocked the opportunity to practice real-world multi-account use cases, such as:

Shared Data Lake
RDS Cross-Account Database Access
VPC Peering
Centralized Logging

At first, I managed my AWS accounts with just some personal notes. But as the number of accounts grew, it became harder to keep track, and I found myself juggling multiple browsers just to run experiments. Eventually, I moved to a better strategy: using AWS Organizations and, later, IAM Identity Center to enable Single Sign-On across all my accounts.

That’s exactly what you’re going to learn in this post. Hopefully, this will unlock tons of learning opportunities for you too. Having multiple accounts gives you the freedom to:

Launch pet projects without risking production
Monitor and experiment in isolation
Explore advanced AWS use cases

And most importantly: it helps you avoid doing experiments in your company’s account. Even though it might seem cheaper, it’s far riskier—I’ve heard plenty of horror stories about that.

The first requirement to implement AWS SSO is having a valid AWS Organization.

AWS Organizations

AWS Organizations is a service that helps you centrally manage and govern multiple AWS accounts, with benefits such as:

Consolidated Billing
Service Control Policies (SCPs)
AWS SSO Integration (one of the main advantages of having organizations in place)

You can either invita a AWS existing account or create new one in the organization console.

AWS SSO

AWS Single Sign-On (SSO) enables you to centrally manage SSO access and user permissions across your AWS accounts. Typically, when you log into an AWS account, you use IAM users and must authenticate manually for each account. The first problem with this approach is that you sometimes need to provide AWS credentials to the same person across different accounts, which quickly becomes complex to manage.

Demo

1. Enable SSO

Sign in to the AWS Identity and Access Management console.
Open the Identity Center.
Choose Enable.
Choose Enable with AWS Organizations.
Choose Continue.

After that, you can customize the AWS access portal URL. Stick to conventions and remember it has to be unique.

2. Permission Sets

You can select the session duration and define the amount of time before a user is logged out. For production, I’d recommend 1 hour, but for development and testing you can use 4 hours or more.

3. Users

You can set user information, job-related details, and even custom attributes.

Finally, assign the user to a group.

4. Groups

5. SSO

Finally, you’ll have the AWS access portal URL and credentials for the user—whether provided via email or generated as a random password.

You’ll see the accounts and roles that match the permission sets we defined and allocated to the created user.

Another benefit: you can also implement and configure applications, and enable MFA (which I highly recommend for stronger security).

Conclusion

With AWS SSO, you can create a powerful architecture with a centralized portal to manage users and permissions, while also taking advantage of consolidated billing and guardrails through Organizations and SCPs.

Overall, the benefits are huge: you gain hands-on experience with multi-account projects, keep your learning active, and—most importantly—avoid the risk of experimenting inside your company’s account.

And the best part: there’s no additional cost to use it. My recommendation? Treat it as a default best practice. There are no downsides, only plenty of benefits.

There has never been a better time to be an engineer and create value in society through software.

If you enjoyed the articles, visit my blog at jorgetovar.dev.

Building a Scalable and Resilient Architecture on AWS

Jorge Tovar — Fri, 02 May 2025 14:08:39 +0000

Creating a large-scale system is a complex challenge that often requires first-principles thinking. To design an architecture that supports client needs effectively, it’s important to start with simplicity and evolve toward a more robust and reliable system over time.

Many companies begin with overly complex architectures, but in my view, it’s best to start simple and iteratively build upon it—adapting and scaling as requirements grow.

VPC

The foundation is a robust Virtual Private Cloud (VPC) architecture, including:

A three-tier setup (Public, Application, Database) across three Availability Zones (AZs)
Public subnets with Internet Gateway access
Security Groups to control traffic between layers
High availability through multi-AZ deployment
IPv4 and IPv6 support
EFS for shared storage
Integrated monitoring via CloudWatch

Single Server Setup – EC2

Initially, I would start with a monolithic application deployed in the public subnet. For example, a WordPress instance capable of handling the early traffic. However, if this instance is stopped, it receives a new public IP, causing users to lose access. More critically, both data and files can be lost—something we must avoid from the beginning.

Database

A key early step is to decouple the data layer by deploying a managed database in the DB subnets. This not only isolates application logic from data but also lays the groundwork for scalability. Additionally, databases can be further configured for robustness:

Database Replication

We can start with basic replication—distributing data across multiple instances and configuring regular snapshots for backup and recovery.

Vertical vs. Horizontal Scaling

While vertical scaling (upgrading to a larger instance) can be a quick fix, horizontal scaling (adding more instances) is typically preferred for large-scale systems due to the hard limits of vertical growth.

Load Balancers

Relying on a single server for all incoming traffic is not sustainable at scale. Introducing an Application Load Balancer (ALB) helps distribute traffic across multiple instances, improving availability and performance.

Cache

Caching is essential to reduce load on databases and improve response times. AWS services like ElastiCache (Redis/Memcached) can be used to cache expensive queries or frequently accessed data. Additionally, caching can be applied at:

The frontend (e.g., static resources)
API level (API Gateway Caching)

I typically use cache for non-volatile content, but care must be taken to prevent serving stale data.

Content Delivery Network (CDN)

To optimize performance for global users, a CDN like CloudFront can serve content from edge locations closest to the user.

One caveat: caching introduces complexity, such as stale content. Fortunately, CloudFront allows explicit cache invalidation—whether by resource path (e.g., /resources/*) or individual files.

Configurations

Use AWS Systems Manager Parameter Store to externalize configurations. For example, by storing your configuration data in an external service like Parameter Store, you can easily launch new servers that retrieve and use the same configurations—making it simple to replicate environments and scale consistently.

Stateless Architecture

Stateless applications make scaling easier. Each instance can serve requests independently, so if one fails, another can take over seamlessly. In contrast, stateful applications (like the initial monolith) create dependencies that hinder fault tolerance and scaling.

Event-Driven Architecture

To improve decoupling and resilience, I prefer asynchronous communication using messaging systems:

SQS for point-to-point queues SQS Developer Guide
SNS for publish-subscribe scenarios SNS Developer Guide

This allows components to recover gracefully if others are temporarily unavailable—avoiding ripple effects that can crash the system.

Observability

A scalable system must be observable. AWS Lambda provides this by default with:

Logging
Metrics
Tracing

These capabilities enable you to monitor system health, debug issues, and make informed scaling decisions. Metrics can also feed into Auto Scaling Group (ASG) policies.

Auto Scaling

With observability in place, the next step is implementing Auto Scaling—defining policies that trigger scale-out or scale-in actions. These policies can be based on:

CPU usage
Memory pressure
Network traffic
Time-based patterns (e.g., predictable peak hours)

Conclusion

Building a highly scalable and resilient application in AWS is challenging but achievable. You can start with a simple, monolithic architecture and progressively evolve it to handle millions of users.

Along the way, keep these resilience patterns in mind:

Minimize integration points
Prevent cascading failures
Avoid blocked threads and slow responses
Implement timeouts and circuit breakers
Use bulkheads to isolate failures
Embrace the Fail Fast principle: detect and handle errors early

There has never been a better time to be an engineer and create value in society through software.

If you enjoyed the articles, visit my blog at jorgetovar.dev.

How to deploy a serverless application using Terragrunt

Jorge Tovar — Thu, 06 Feb 2025 14:55:55 +0000

Why Use Terragrunt to Deploy Applications on AWS?

There are many principles and patterns in software engineering that I've found valuable for building reliable and robust applications. Among them are well-known concepts like the Dependency Inversion Principle (DIP), the Law of Demeter (LoD), and Murphy's Law (ML). However, perhaps the easiest one for all of us to remember is Don't Repeat Yourself (DRY).

The core idea of this principle is to avoid code duplication and, even more importantly, to maintain a single source of truth.

Deploying Applications on AWS

In my experience, I have used CloudFormation and Terraform for deploying enterprise applications on AWS. Both are excellent tools capable of handling anything from simple systems to distributed applications serving millions of users.

By default, I prefer CloudFormation because it is AWS-native, doesn't require managing complex Terraform state files, and "just works." However, as applications grow in complexity, the advantages of using Terraform modules and reusing higher-level abstractions become evident, often making Terraform a better fit.

Managing Complexity

As applications expand in terms of resources, dependencies, and environments, it becomes crucial to follow best practices and organize code to efficiently manage multiple resources while keeping environments secure and state isolated.

Typical Code Structure and Its Drawbacks

A common approach is to create isolated folders for each environment (e.g., prod, stage, and dev), ideally living in separate AWS accounts and sharing nothing.

Each environment may use different module versions, either public or stored in the same repository. While this setup offers flexibility, it introduces significant code duplication. Maintaining consistent module usage across environments requires careful manual management and version control, which becomes tedious and error-prone.

Terragrunt as an Alternative

Terragrunt is a tool that acts as a wrapper for Terraform, simplifying the management of complex infrastructure. It allows developers to:

Reduce code duplication by using configuration (HCL) files to define shared resources and modules.
Set a single source of truth for environment configurations.
Maintain isolated state management for different environments.

This approach minimizes the need for separate branches and reduces maintenance overhead, making it easier to maintain and scale infrastructure.

Code Example (GitHub)

For a practical demonstration, you can refer to my example project:

Terragrunt Serverless

Explanation

We follow a typical Terraform project structure to create our infrastructure modules while leveraging AWS Terraform Modules.

The key improvement introduced by Terragrunt is the ability to avoid code duplication across environments by centralizing configuration management. Instead of duplicating code for dev, stage, and prod environments, Terragrunt allows us to define reusable inputs and references, significantly simplifying maintenance.

Below is an example structure:

Environment-Specific Folders

We create two environment-specific folders, stage and dev, where we only define the module source and provide specific input variables:

stage

terraform {
  source = "../../modules/lambda-api"
}

inputs = {
  lambda_description = "My awesome Lambda Function"
  environment        = "stage"
}

dev

terraform {
  source = "../../modules/lambda-api"
}

inputs = {
  lambda_description = "My awesome Lambda Function"
  environment        = "dev"
}

By defining just the source and input variables in environment folders, Terragrunt eliminates the need to copy and maintain separate Terraform files for each environment.

Running Terragrunt Commands

Terragrunt uses the same commands as Terraform but streamlines multi-environment deployments.

Initialize:

   terragrunt init

Apply Configuration:

   terragrunt apply

Test the API:

   curl $(terragrunt output -raw api_gateway_v2_api_api_endpoint)

Destroy Infrastructure:

   terragrunt destroy

Conclusion 🤔

In this article, we explored how Terragrunt simplifies infrastructure deployment on AWS, significantly reducing code duplication by enabling modular and reusable configurations.

By leveraging IaC and automation, we achieve faster, more reliable deployments while maintaining cleaner, maintainable code. Organizing modules thoughtfully allows us to integrate them into infrastructure without requiring an understanding of the entire system.

If you found this useful, check out more of my work:

Visit my blog at jorgetovar.dev for more insights.

Creating a Rest API with Infrastructure as Code (Tofu) and Lambda + Typescript

Jorge Tovar — Tue, 10 Sep 2024 15:44:07 +0000

Deploying an API with Typescript in minutes instead of days

Deploying to production is becoming easier every day. Best practices and tools are available to us to create software that eventually contributes to automating tedious tasks, creating new businesses and startups, and ultimately generating value for society 🌎.

Deployment Anti-Patterns 🙅🏻‍♂️

Manually created infrastructure
Deploying at specific times due to lack of technical capacity (production errors, unstable software, service unavailability) rather than business decisions
Having to send emails to another team for deployment
Branches like "development" that differ from what exists in "main" and sometimes generate conflicts during merging
Manual testing
Tickets for deployment

All the points mentioned above, along with excessive bureaucracy, committees, and meetings, are still the norm in many organizations, and it was my reality for many years. But there is an alternative, a way of seeing the software development process where simplicity is key, both in deployments with pipelines and automation, and in the creation of software, where the focus should be on reducing accidental complexity, building modular, testable applications that are easy to understand for the team.

A Brighter Future 💻

Unit and Integration testing
Trunk-based development
AI, ChatGPT, Copilot
DevOps as a mindset, not just a role
Infrastructure as code
Serverless and containers
REST, SOAP, GraphQL, gRPC

The purpose of this post is to showcase an essential how to create an API with Typescript and Tofu (or Terraform).

Let's talk about the code!

Code on GitHub: tofu-typescript

Project Structure 🏭

The following files are typical in a Tofu project, and this is the convention we are using in this project.

main.tf - Entrypoint for Tofu
variables.tf - Input variables
terraform.tfvars - Variable definition file
providers.tf - Provider declarations
outputs.tf - Output values
versions.tf - Provider version locking

Why Tofu ☁️

Open source
Multi-cloud
Immutable infrastructure
Declarative paradigm
Domain-specific language (DSL)
Does not require agents or master nodes
Free
Large community
Mature software

Why Serverless and Typescript

Automatic scalability
Pay-per-use
High availability
Easy development and deployment
Type Safety
Better documentation
Enhanced developer experience
Integration with managed services

Code

Lambda: The code for a Lambda function should generally be simple and adhere to the single responsibility principle or reason to change. The name of the method should correspond to the entry point defined in the Tofu code.

Types help document the code and provide better IDE support.



import {APIGatewayProxyEvent, APIGatewayProxyResult} from 'aws-lambda';

export const lambdaHandler = async (event: APIGatewayProxyEvent): Promise<APIGatewayProxyResult> => {
    console.log(event.body);
    try {
        return {
            statusCode: 200,
            headers: {
                'Content-Type': 'application/json',
            },
            body: JSON.stringify({
                message: 'Hello World from Typescript + Tofu!',
            }),
        };
    } catch (err) {
        console.log(err);
        return {
            statusCode: 500,
            headers: {
                'Content-Type': 'application/json',
            },
            body: JSON.stringify({
                message: 'Error processing request',
            }),
        };
    }
};

.gitignore: Definition of files that we do not want to add to the repository. It is crucial not to publish AWS credential information.



# Terraform-specific files
.terraform/
terraform.tfstate
terraform.tfstate.backup
*.tfvars

hello_world.zip
response.json
.layer
layer.zip

# AWS credentials file
.aws/credentials

Lambda Resources: Configures the Lambda module to retrieve the source code from a bucket also created by Tofu, and we define lambdaHandler as the method that will receive and process the API event.

Modules are the key ingredient for writing reusable, maintainable code. When planning to deploy AWS infrastructure, it's better to leverage this functionality and be aware of the common gotchas to avoid conflicts with other Tofu configurations.



provider "aws" {
  region = "us-east-1"
}

module "function" {
  source = "../../modules/lambda"

  name = "lambda-sample"

  # Point to the dist directory with transpiled JS
  src_dir = "${path.module}/dist"
  runtime = "nodejs20.x"
  handler = "index.lambdaHandler" # index.js, lambdaHandler

  memory_size = 128
  timeout     = 5

  environment_variables = {
    NODE_ENV = "production"
  }
}

module "gateway" {
  source = "../../modules/api-gateway"

  name         = "lambda-sample"
  function_arn = module.function.function_arn
  api_gateway_routes = ["GET /"]
}

Permissions: Defines the required permissions to execute the Lambda and access AWS services.



data "aws_iam_policy_document" "policy" {
  statement {
    actions = ["sts:AssumeRole"]
    principals {
      type = "Service"
      identifiers = ["lambda.amazonaws.com"]
    }
  }
}

Tofu Deployment

Deploying the API should be as simple as cloning the code and running a couple of Tofu commands.

The tofu init command is used to initialize a project in a working directory. This command downloads provider code and configures the backend, where state and other key data are stored.



tofu init

The tofu apply command is used to apply the changes defined in your infrastructure configuration and create or modify the corresponding resources in your infrastructure provider.

You can use the Node.js runtime to run TypeScript code in AWS Lambda. Because Node.js doesn't run TypeScript code natively, you must first transpile your TypeScript code into JavaScript.



./tf.sh

Finally, you can retrieve the information from the Outputs and make the API call.



http "$(tofu output -raw api_endpoint)"

Conclusion 🤔

In this article, we have explored the combination of Tofu and Serverless to create an API in minutes.

By leveraging IaC, and automated testing, we can achieve faster and more reliable deployments while maintaining high-quality code.

We should organize our modules in a way that we can easily interact with a part of the infrastructure without having to
understand the whole system.

Not too long ago, deploying an API in production was a tedious task that involved multiple actors. Nowadays, we have a wealth of open-source software and developer-focused tools that emphasize immutability, automation, productivity-enhancing AI, and powerful IDEs. In summary, there has never been a better time to be an engineer and create value in society through software.

If you enjoyed the articles, visit my blog at jorgetovar.dev.

The Clojure Paradox

Jorge Tovar — Tue, 27 Aug 2024 20:41:10 +0000

Several years ago, I came across The Python Paradox by Paul Graham. In it, Graham argues that Lisp and Python programmers are often those who genuinely care about programming and, as a result, tend to do it better than, for instance, Java programmers. This perspective fundamentally changed how I viewed Python as a language.

At first, I found Python’s lack of semicolons and reliance on indentation to be strange and uncomfortable. I even saw Python as a tool for building only basic applications. However, with the rise of serverless computing, machine learning, and data science, the immense power of Python has become increasingly apparent. The language is getting faster, and its ecosystem is rapidly growing. Libraries like FastAPI and Pandas are truly remarkable, allowing us to solve problems succinctly.

As programmers, our job is to solve problems, and since we read more code than we write, having fewer lines of code reduces the surface area for bugs to hide and helps us avoid cognitive overload.

When I started working with AWS's Boto3, I realized that tasks that previously took me 30 lines of Java could now be done in just 3 lines of Python. It was mind-blowing. Don’t get me wrong, Java is still one of my favorite programming languages, and with its new release cadence, it’s only getting better. But the amount of ceremony required to accomplish basic tasks in Java is something sometimes we’d all prefer to avoid.

Recently, I've been experimenting with Go. Although it prides itself on simplicity, IMHO I find it too verbose. I know that excellent tools have been built with Go, and there are certain ideas and applications that should be developed with it. Its compilation speed and efficient memory usage make Go a strong contender, it might even be the best combination of developer experience and performance, which is becoming increasingly important in modern, cloud-native applications.

However, after 10 years in the industry and having deployed applications in several languages, I remain a fan of Clojure. Despite its niche status, Clojure incorporates ideas from other languages, such as Go’s goroutines. It’s a Lisp dialect, inherently immutable, and designed with concurrency in mind. What stands out most to me is how Clojure allows you to focus on solving problems without the burden of unnecessary ceremony. The majority of the code is about the problem itself; it’s data-oriented, and I often find that it helps me enter a Flow state (Happiness) where programming becomes truly enjoyable.

With Go, I currently have mixed feelings. While it brings many good ideas to the table in terms of concurrency and simplicity, I find that the codebases tend to be larger and more ceremonious. Clojure, on the other hand, tends to produce code that is less brittle and primarily composed of pure functions.

Timeless ideas

I’ve always been a fan of timeless ideas because they are often the most important and foundational, yet they are also the most overlooked and I feel that Clojure fully embraces of all them.

Programs composed mostly of pure functions are more robust and easier to test.
Immutability reduces complexity.
Smaller programs have fewer bugs.
Software development is fundamentally about composition.
We should minimize
Incidental complexity can make a system harder to understand, maintain, and extend.

Code examples

In this example, I read a book from an online text file and perform basic processing to illustrate how the same problem can be approached in different programming languages and paradigms.

Go

My opinion: The absence of sets and built-in functions such as filter, map and reduce makes the problem more difficult to solve. Basic tasks, such as filtering, often need to be done in an imperative manner.

package main

import (
    "fmt"
    "io"
    "net/http"
    "regexp"
    "sort"
    "strings"
)

var commonWords = map[string]struct{}{
    "a": {}, "able": {}, "about": {}, "across": {}, "after": {}, "all": {}, "almost": {}, "also": {}, "am": {}, "among": {},
    "an": {}, "and": {}, "any": {}, "are": {}, "as": {}, "at": {}, "be": {}, "because": {}, "been": {}, "but": {}, "by": {},
    "can": {}, "cannot": {}, "could": {}, "dear": {}, "did": {}, "do": {}, "does": {}, "either": {}, "else": {}, "ever": {},
    "every": {}, "for": {}, "from": {}, "get": {}, "got": {}, "had": {}, "has": {}, "have": {}, "he": {}, "her": {}, "hers": {},
    "him": {}, "his": {}, "how": {}, "however": {}, "i": {}, "if": {}, "in": {}, "into": {}, "is": {}, "it": {}, "its": {},
    "just": {}, "least": {}, "let": {}, "like": {}, "likely": {}, "may": {}, "me": {}, "might": {}, "most": {}, "must": {},
    "my": {}, "neither": {}, "no": {}, "nor": {}, "not": {}, "of": {}, "off": {}, "often": {}, "on": {}, "only": {}, "or": {},
    "other": {}, "our": {}, "own": {}, "rather": {}, "said": {}, "says": {}, "she": {}, "should": {}, "since": {}, "so": {},
    "some": {}, "than": {}, "that": {}, "the": {}, "their": {}, "them": {}, "then": {}, "there": {}, "these": {}, "they": {},
    "this": {}, "those": {}, "through": {}, "to": {}, "too": {}, "more": {}, "upon": {}, "us": {}, "wants": {}, "was": {},
    "we": {}, "were": {}, "what": {}, "when": {}, "where": {}, "which": {}, "while": {}, "who": {}, "whom": {}, "why": {},
    "will": {}, "with": {}, "would": {}, "yet": {}, "you": {}, "your": {}, "shall": {}, "before": {}, "now": {}, "one": {},
    "even": {},
}

func getBook() string {
    resp, err := http.Get("https://www.gutenberg.org/cache/epub/84/pg84.txt")
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()

    body, err := io.ReadAll(resp.Body)
    if err != nil {
        panic(err)
    }
    return string(body)
}

func getWords(book string) []string {
    re := regexp.MustCompile(`[\w’]+`)
    return re.FindAllString(book, -1)
}

func filterWords(words []string) []string {
    var result []string
    for _, word := range words {
        w := strings.ToLower(word)
        _, ok := commonWords[w]
        if !ok {
            result = append(result, w)
        }
    }
    return result
}

func getFrequentWords(words []string, n int) map[string]int {
    var filteredWords []string
    for _, word := range words {
        _, ok := commonWords[word]
        if !ok {
            filteredWords = append(filteredWords, strings.ToLower(word))
        }
    }
    var unorderedWords = make(map[string]int)
    for _, word := range words {
        unorderedWords[word]++
    }
    type wordFrequency struct {
        word  string
        count int
    }
    var wordFrequencies []wordFrequency
    for word, count := range unorderedWords {
        wordFrequencies = append(wordFrequencies, wordFrequency{word, count})
    }
    sort.Slice(wordFrequencies, func(i, j int) bool {
        return wordFrequencies[i].count > wordFrequencies[j].count
    })

    topN := make(map[string]int)
    for i := 0; i < len(wordFrequencies) && i < n; i++ {
        topN[wordFrequencies[i].word] = wordFrequencies[i].count
    }

    return topN
}

func main() {
    book := getBook()
    words := getWords(book)
    filteredWords := filterWords(words)
    fmt.Println("Total words:", len(words))
    fmt.Println("Frequent words:", getFrequentWords(filteredWords, 10))

}

Java

My opinion: It does the job. Since Java 8, the language has been getting better. Even though it has some verbosity, you'll find that we now have collectors and functions to perform tasks without issues. The tedious part is having to put everything into classes just to solve a problem.

package jorgetovar.book;

import org.springframework.web.client.RestTemplate;

import java.util.*;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.stream.Collectors;

public class Book {

    private static final Set<String> commonWords = Set.of(
            "a", "able", "about", "across", "after", "all", "almost", "also", "am", "among", "an",
            "and", "any", "are", "as", "at", "be", "because", "been", "but", "by", "can", "cannot",
            "could", "dear", "did", "do", "does", "either", "else", "ever", "every", "for", "from",
            "get", "got", "had", "has", "have", "he", "her", "hers", "him", "his", "how", "however",
            "i", "if", "in", "into", "is", "it", "its", "just", "least", "let", "like", "likely",
            "may", "me", "might", "most", "must", "my", "neither", "no", "nor", "not", "of", "off",
            "often", "on", "only", "or", "other", "our", "own", "rather", "said", "says", "she",
            "should", "since", "so", "some", "than", "that", "the", "their", "them", "then",
            "there", "these", "they", "this", "those", "through", "to", "too", "more", "upon",
            "us", "wants", "was", "we", "were", "what", "when", "where", "which", "while", "who",
            "whom", "why", "will", "with", "would", "yet", "you", "your", "shall", "before", "now", "one",
            "even"
    );

    public static String getBook() {
        RestTemplate restTemplate = new RestTemplate();
        String bookUrl = "https://www.gutenberg.org/cache/epub/84/pg84.txt";
        return restTemplate.getForObject(bookUrl, String.class);
    }

    public static List<String> getWords(String book) {
        List<String> words = new ArrayList<>();
        Pattern wordPattern = Pattern.compile("[\\w’]+");
        Matcher matcher = wordPattern.matcher(book);
        while (matcher.find()) {
            words.add(matcher.group());
        }
        return words;
    }

    public static List<Map.Entry<String, Long>> getFrequentWords(List<String> words, int takeN) {
        return words.stream()
                .map(String::toLowerCase)
                .filter(word -> !commonWords.contains(word))
                .collect(Collectors.groupingBy(word -> word, Collectors.counting()))
                .entrySet()
                .stream()
                .sorted((e1, e2) -> Long.compare(e2.getValue(), e1.getValue()))
                .limit(takeN)
                .map(e -> Map.entry(e.getKey(), e.getValue()))
                .toList();
    }

    public static Map<Integer, List<String>> getLongestWords(List<String> words, int takeN) {
        return words.stream()
                .map(String::toLowerCase)
                .distinct()
                .sorted(getLongestWord())
                .limit(takeN)
                .collect(Collectors.groupingBy(String::length));
    }

    public static boolean isPalindrome(String word) {
        return word.contentEquals(new StringBuilder(word).reverse());
    }

    public static List<String> getLongestPalindromes(List<String> words, int takeN) {
        return words.stream()
                .map(String::toLowerCase)
                .filter(word -> !commonWords.contains(word))
                .distinct()
                .filter(Book::isPalindrome)
                .sorted(getLongestWord())
                .limit(takeN)
                .toList();
    }

    private static Comparator<String> getLongestWord() {
        return Comparator.comparingInt(String::length).reversed();
    }

    public static void main(String[] args) {
        String book = getBook();
        List<String> words = getWords(book);
        System.out.println("Total Words: " + words.size());
        System.out.println("Most Frequent Words:");
        getFrequentWords(words, 10).forEach(entry -> System.out.println(entry.getKey() + ": " + entry.getValue()));

        System.out.println("\nLongest Words Grouped by Length:");
        getLongestWords(words, 10).forEach((length, group) -> System.out.println("Length " + length + ": " + group));

        System.out.println("\nLongest Palindromes:");
        getLongestPalindromes(words, 3).forEach(System.out::println);
    }


}

Kotlin

My opinion: For me, this could be the most fun and robust enterprise language. It has good support for functions and immutability.

package jorgetovar.book

import org.springframework.web.client.RestTemplate


val commonWords = setOf(
    "a", "able", "about", "across", "after", "all", "almost", "also", "am", "among", "an",
    "and", "any", "are", "as", "at", "be", "because", "been", "but", "by", "can", "cannot",
    "could", "dear", "did", "do", "does", "either", "else", "ever", "every", "for", "from",
    "get", "got", "had", "has", "have", "he", "her", "hers", "him", "his", "how", "however",
    "i", "if", "in", "into", "is", "it", "its", "just", "least", "let", "like", "likely",
    "may", "me", "might", "most", "must", "my", "neither", "no", "nor", "not", "of", "off",
    "often", "on", "only", "or", "other", "our", "own", "rather", "said", "says", "she",
    "should", "since", "so", "some", "than", "that", "the", "their", "them", "then",
    "there", "these", "they", "this", "those", "through", "to", "too", "more", "upon",
    "us", "wants", "was", "we", "were", "what", "when", "where", "which", "while", "who",
    "whom", "why", "will", "with", "would", "yet", "you", "your", "shall", "before", "now", "one",
    "even"
)

fun getBook(): String {
    val restTemplate = RestTemplate()
    val bookUrl = "https://www.gutenberg.org/cache/epub/84/pg84.txt"
    return restTemplate.getForObject(bookUrl, String::class.java) ?: ""
}

fun getWords(book: String): List<String> {
    return "[\\w’]+".toRegex().findAll(book).map { it.value }.toList()
}

fun getFrequentWords(words: List<String>, takeN: Int): List<Pair<String, Int>> {
    val filteredWords = words
        .map { it.lowercase() }
        .filter { it !in commonWords }

    return filteredWords
        .groupingBy { it }
        .eachCount()
        .toList()
        .sortedByDescending { it.second }
        .take(takeN)

}

fun getLongestWords(words: List<String>, takeN: Int): Map<Int, List<String>> {
    val uniqueWords = words
        .map { it.lowercase() }
        .distinct()
    return uniqueWords
        .sortedByDescending { it.length }
        .take(takeN)
        .groupBy { it.length }
}

fun isPalindrome(word: String): Boolean {
    return word == word.reversed()
}

fun getLongestPalindromes(words: List<String>, takeN: Int): List<String> {
    val uniqueWords = words
        .map { it.lowercase() }
        .filter { it !in commonWords }
        .distinct()
    val palindromes = uniqueWords
        .filter { isPalindrome(it) }
    return palindromes
        .sortedByDescending { it.length }.take(takeN)
}


fun main() {
    val book = getBook()
    val words = getWords(book)
    println("Total Words: ${words.size}")
    println("Most Frequent Words:")
    println(getFrequentWords(words, 10))
    println("Longest Words Grouped by Length:")
    println(getLongestWords(words, 5))
    println("Longest Palindromes:")
    println(getLongestPalindromes(words, 3))
}

Python

My opinion: I really like using this language. Sometimes it can get messy because it's too permissive and allows you to mutate variables, etc. But in general, you'll find that list comprehensions are really good for solving these kinds of problems. I don’t like the result when using classes, but for this example, it was just enough.

import requests
import re

from collections import Counter, defaultdict


def get_book():
    book = requests.get("https://www.gutenberg.org/cache/epub/84/pg84.txt")
    return book.text


def get_words(book):
    return re.findall(r"[a-zA-Z0-9’]+", book)


common_words = {
    "a", "able", "about", "across", "after", "all", "almost", "also", "am", "among", "an",
    "and", "any", "are", "as", "at", "be", "because", "been", "but", "by", "can", "cannot",
    "could", "dear", "did", "do", "does", "either", "else", "ever", "every", "for", "from",
    "get", "got", "had", "has", "have", "he", "her", "hers", "him", "his", "how", "however",
    "i", "if", "in", "into", "is", "it", "its", "just", "least", "let", "like", "likely",
    "may", "me", "might", "most", "must", "my", "neither", "no", "nor", "not", "of", "off",
    "often", "on", "only", "or", "other", "our", "own", "rather", "said", "says", "she",
    "should", "since", "so", "some", "than", "that", "the", "their", "them", "then",
    "there", "these", "they", "this", "those", "through", "to", "too", "more", "upon",
    "us", "wants", "was", "we", "were", "what", "when", "where", "which", "while", "who",
    "whom", "why", "will", "with", "would", "yet", "you", "your", "shall", "before", "now", "one",
    "even"
}


def get_frequent_words(words, take_n):
    frequent_words = [word.lower() for word in words if word.lower() not in common_words]
    word_frequencies = Counter(frequent_words)
    return word_frequencies.most_common(take_n)


def get_longest_words(words, take_n):
    unique_words = set(word.lower() for word in words)
    longest_groups = defaultdict(list)
    sorted_works = sorted(unique_words, key=len, reverse=True)[:take_n]
    for word in sorted_works:
        longest_groups[len(word)].append(word)
    return dict(longest_groups)


def is_palindrome(word):
    return word == word[::-1]


def get_longest_palindromes(words, take_n):
    unique_words = set(word.lower() for word in words if word.lower() not in common_words)
    palindromes = [word for word in unique_words if is_palindrome(word)]
    palindromes.sort(key=len, reverse=True)
    return palindromes[:take_n]


def main():
    book = get_book()
    words = get_words(book)
    print("Total words:", len(words))
    print(get_frequent_words(words, 10))
    print(get_longest_words(words, 10))
    print(get_longest_palindromes(words, 3))


if __name__ == "__main__":
    main()

Clojure

My opinion: The problem with Clojure is its niche nature. It’s usually difficult to understand the basics of the language and its philosophy. The amount of parentheses is unattractive to a lot of people, but in general, I find it the most beautiful implementation.


(ns clojure-book.core
  [:require [clojure.string :as str]]
  (:gen-class))

(def book (slurp "https://www.gutenberg.org/cache/epub/84/pg84.txt"))

(def words (re-seq #"[\w|’]+" book))

(def common-words
  #{"a" "able" "about" "across" "after" "all" "almost" "also" "am" "among" "an"
    "and" "any" "are" "as" "at" "be" "because" "been" "but" "by" "can" "cannot"
    "could" "dear" "did" "do" "does" "either" "else" "ever" "every" "for" "from"
    "get" "got" "had" "has" "have" "he" "her" "hers" "him" "his" "how" "however"
    "i" "if" "in" "into" "is" "it" "its" "just" "least" "let" "like" "likely"
    "may" "me" "might" "most" "must" "my" "neither" "no" "nor" "not" "of" "off"
    "often" "on" "only" "or" "other" "our" "own" "rather" "said" "says" "she"
    "should" "since" "so" "some" "than" "that" "the" "their" "them" "then"
    "there" "these" "they" "this" "those" "through" "to" "too" "more" "upon"
    "us" "wants" "was" "we" "were" "what" "when" "where" "which" "while" "who"
    "whom" "why" "will" "with" "would" "yet" "you" "your" "shall" "before" "now" "one"
    "even"
    })

(defn palindrome? [word]
  (= (seq word) (reverse (seq word)))
  )

(defn frequent-words [take-n]
  (->> words
       (map str/lower-case)
       (remove common-words)
       (frequencies)
       (sort-by val)
       (take-last take-n))
  )

(defn longest-words [take-n]
  (->> words
       (map str/lower-case)
       (distinct)
       (sort-by count)
       (take-last take-n)
       (group-by count)
       )
  )

(defn longest-palindromes [take-n]
  (->> words
       (map str/lower-case)
       (distinct)
       (filter palindrome?)
       (sort-by count)
       (take-last take-n)
       )
  )

(defn -main
  [& args]
  (println (str "Total words:" (count words)))
  (println (take 10 words))
  (println (frequent-words 10))
  (println (longest-words 10))
  (println (longest-palindromes 3))

  )

Conclusion

Software is constantly evolving, and client expectations for the programs they use and build are growing. However, our focus should remain on solving problems, eliminating incidental complexity, and taking pride in our craft. There is no 'best' programming language—only tools that help us address specific problems. Even when working with legacy systems, we have the opportunity to make a positive impact through good naming conventions, best practices, improving the architecture, and generally putting the project in a better state

There has never been a better time to be an engineer and create value in society through software.

If you enjoyed the articles, visit my blog jorgetovar.dev

Build Go Serverless REST APIs and Deploy to AWS using the SAM framework (Amazon Linux 2 Runtime)

Jorge Tovar — Thu, 22 Aug 2024 16:05:55 +0000

Why Another Go Tutorial

AWS has been deprecating several services and runtimes recently. As we’ve seen with the discontinuation of our beloved CodeCommit and other crucial services, Go1.x is no longer supported for AWS Lambda functions.

If you try to deploy most of the outdated tutorials, you might encounter errors like this:



Resource creation Initiated    
CREATE_FAILED                    AWS::Lambda::Function            DemoFunction                     
                                   Resource handler returned message: 
                                   "The runtime parameter of go1.x is no longer supported for 
                                   creating or updating AWS Lambda functions. We recommend you 
                                   use a supported runtime while creating or updating functions. 
                                   (Service: Lambda, Status Code: 400, Request ID:  
                                   81f1f708-0a7a-40d0-8442-b9c16510d01f)" 
ROLLBACK_IN_PROGRESS             AWS::CloudFormation::Stack       lambda-go-gorilla                
                                   The following resource(s) failed to create: 
                                   [DemoFunction]. Rollback requested by user.

The key takeaway is that the only constant in software is change. However, there are some timeless principles that we should always keep in mind:

To address this issue, I decided to create an up-to-date repository with all the infrastructure needed to deploy a Go application. There are two options available:

Deploying with Fargate using Docker containers.
Deploying using the SAM framework on AWS.

You can GitHub find the repository here.

Timeless principles in Software Development

Infrastructure as Code is essential.
Good naming conventions in software are crucial.
Always test your logic.
Availability & scalability
Deployment Pipeline as a mechanism to automate the software delivery process.
Observability Is Mandatory.
Security is a first-class citizen in cloud-native applications.
Go is an excellent option for building APIs.

Infrastructure as Code is Essential

Immutable infrastructure allows us to declare what we want at a higher level and ensures that development and production environments remain as close as possible. For example:



CompoundingFunction:
  Type: AWS::Serverless::Function
  Metadata:
    BuildMethod: makefile
  Properties:
    FunctionName: CompoundingFunction
    Architectures: ["arm64"]
    Handler: bootstrap
    Runtime: provided.al2
    CodeUri: ./functions/CompoundingFunction/
    MemorySize: 512
    Timeout: 10
    Environment:
      Variables:
        COMPOUNDING_TABLE_NAME: !Ref CompoundingTable
    Policies:
      - DynamoDBCrudPolicy:
          TableName: !Ref CompoundingTable
    Events:
      ApiGatewayPost:
        Type: Api
        Properties:
          RestApiId: !Ref ApiGateway
          Path: /compounding
          Method: POST

Good Naming Conventions in Software Are Key

Don’t be afraid to refactor if you have a good suite of tests. Refactoring is an essential activity in software development. Names are important as they appear everywhere in modules, functions, packages, variables, etc.



package main

import (
    "context"
    "encoding/json"
    "fmt"
    "github.com/aws/aws-lambda-go/events"
    "github.com/aws/aws-lambda-go/lambda"
)

// Response is the structure for the response JSON
type Response struct {
    Message      string    `json:"message"`
    GainsPerYear []float64 `json:"gainsPerYear"`
}

type Request struct {
    Principal  float64 `json:"principal"`
    AnnualRate float64 `json:"annualRate"`
    Years      int     `json:"years"`
}

func HelloHandler(ctx context.Context, event events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
    var req Request
    err := json.Unmarshal([]byte(event.Body), &req)
    if err != nil {
        return createResponse(400, "Invalid request body")
    }
    fmt.Println("Request", req)
    gainsPerYear := CalculateCompoundInterest(req.Principal, req.AnnualRate, req.Years)
    fmt.Println(gainsPerYear)
    response := Response{
        Message:      "Calculation successful",
        GainsPerYear: gainsPerYear,
    }

    body, err := json.Marshal(response)
    if err != nil {
        return createResponse(500, "Error marshalling response")
    }

    return createResponse(200, string(body))
}

func createResponse(statusCode int, body string) (events.APIGatewayProxyResponse, error) {
    return events.APIGatewayProxyResponse{
        StatusCode: statusCode,
        Body:       body,
        Headers:    map[string]string{"Content-Type": "application/json"},
    }, nil
}

func main() {
    lambda.Start(HelloHandler)
}

Always Test Your Logic

In serverless applications, unit tests are important, but don’t forget to also include integration tests, as most of these applications rely on integrations and policies to solve business problems.



func TestCalculateCompoundInterest(t *testing.T) {
    principal := 100000000.0
    annualRate := 10.0
    years := 10

    result := CalculateCompoundInterest(principal, annualRate, years)
    lastElement := round(result[len(result)-1], 2)

    expected := round(259374246.01, 2)
    if !reflect.DeepEqual(lastElement, expected) {
        t.Errorf("Expected %v, but got %v", expected, lastElement)
    }
}

Availability & Scalability

Serverless architectures are highly available by default and are event-driven, removing most operational tasks. However, if you choose to rely on ECS and containers, it’s important to include a load balancer to distribute traffic among your servers, ensuring both availability and scalability.



  CompoundingLoadBalancer:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      Name: compounding-nlb
      Scheme: internet-facing
      Type: network
      Subnets:
        - !Ref PublicSubnetOne
        - !Ref PublicSubnetTwo

Deployment Pipeline

A deployment pipeline automates the software delivery process. We created a Makefile to simplify this process, making it easy to deploy and execute repetitive tasks with a single command. This approach enhances efficiency and consistency in your deployment workflow.

Observability Is Mandatory

Ensure you have tracing, logging, and metrics in place. With serverless applications, enabling these features is as simple as adding Tracing: Active. The ability to see all logs in a central place like CloudWatch and monitor the interactions of the service is invaluable.

Security Is a First-Class Citizen in Cloud-Native Applications

Security is paramount in all the application. Using Amazon Cognito provides robust user authentication, while API keys add an additional layer of control and authorization, ensuring that only authorized clients can access your APIs.



Auth:
  DefaultAuthorizer: CompoundingAuthorizer
  Authorizers:
    CompoundingAuthorizer:
      UserPoolArn:  XXXX
    LambdaTokenAuthorizer:
      FunctionArn: !GetAtt LambdaTokenAuthorizerFunction.Arn
      FunctionPayloadType: REQUEST
      Identity:
        Headers:
          - Authorization
        ReauthorizeEvery: 100
  AddDefaultAuthorizerToCorsPreflight: false

Assign the minimal necessary permissions to each service, user, and component to reduce the attack surface and prevent unauthorized access. Least Privilege Principle:



  Policies:
    - DynamoDBCrudPolicy:
        TableName: !Ref CompoundingTable

References

Terraform in Action - Practical uses and strategies for implementing Terraform, a tool for building, changing, and managing infrastructure.
Continuous Delivery Pipelines

Conclusion

Software is constantly evolving, and while some tools and practices will change, the foundational principles remain the same. We need immutable infrastructure, CI/CD, good naming conventions, a robust testing strategy, security in our APIs, and efficiency in our applications. That’s why I decided to recreate this project in a serverless way.

There has never been a better time to be an engineer and create value in society through software.

If you enjoyed the articles, visit my blog jorgetovar.dev

Zero Downtime Deployment in AWS with Tofu/Terraform and SAM

Jorge Tovar — Mon, 12 Aug 2024 22:22:57 +0000

I still remember the days when we had to set flags and display a maintenance page during application deployments. Often, we deployed our applications at midnight or during periods of low traffic to minimize the impact on users, which led to a poor developer experience. Nowadays, as clients, we expect a seamless user experience at all times.

Why Zero-Downtime Deployment

Achieving zero-downtime deployments has become a requirement for modern applications. We are now in an era of software delivery where downtime should be minimal and latency low. As clients, our expectations have raisen. If an application does not function correctly, we perceive it as unreliable and are likely to abandon its use. 🤑

Today, I want to introduce some strategies to implement zero downtime deployments and build trust with your customers.

GitHub Code

Use an instance refresh to update instances in an Auto Scaling group (OpenTofu)
Immutable infrastructure + Mutable Applications (Terraform + Ansible)
Implement Blue/Green deployments
Canary Deployments (Deploying Serverless Applications Gradually)

We aim to avoid creating unreliable applications and instead embrace a cloud-native approach.

Let’s delve into the strategies:

Use an Instance Refresh to Update Instances in an Auto Scaling Group (OpenTofu)

For this example, we are using OpenTofu. I want to emphasize that open-source software is incredibly beneficial—the quality is exceptionally high, and it allows us to leverage numerous resources and build upon code developed by excellent engineers. We can learn significantly from these kinds of projects. That is also one of the reasons that the code here is a synthesis of many ideas and is available to anyone who wants to learn and reuse it.

The Future of Terraform Must Be Open

We need to define three important attributes: min_elb_capacity, lifecycle, and instance_refresh. These attributes will serve the following purposes:

min_elb_capacity: This attribute ensures that the deployment is considered complete only after at least some instances have passed health checks.
lifecycle: This attribute helps prevent the deletion of previous resources after the new ones have been created.
instance_refresh: This attribute defines a strategy for rolling out changes to the Auto Scaling Group (ASG).



resource "aws_autoscaling_group" "zdd" {

  name                = "${var.cluster_name}-${aws_launch_template.zdd.name}"
  vpc_zone_identifier = var.subnets
  target_group_arns = [aws_lb_target_group.asg.arn]
  health_check_type   = "ELB"

  min_elb_capacity = var.min_size

  min_size = var.min_size
  max_size = var.max_size

  lifecycle {
    create_before_destroy = true
  }

  # Use instance refresh to roll out changes to the ASG
  instance_refresh {
    strategy = "Rolling"
    preferences {
      min_healthy_percentage = 50
    }
  }

  launch_template {
    id      = aws_launch_template.zdd.id
    version = "$Latest"
  }

  tag {
    key                 = "Name"
    value               = var.cluster_name
    propagate_at_launch = true
  }

}

The result is an API that returns data reliably without becoming unavailable.

In this example, we rely on immutable infrastructure, so we need to pack and create a new AMI whenever there is a change.



packer build sample-app.pkr.hcl
tofu apply

Immutable infrastructure + Mutable Applications (Terraform + Ansible)

Even though we can do everything with provisioning tools such as Terraform or CloudFormation, sometimes it is better to use them together with configuration management tools like Ansible. Ansible is designed to manage and install software on servers, providing a useful approach for handling zero-downtime deployments.

We can serve our application with Nginx



  - name: Install Nginx
    hosts: all
    become: true

    tasks:
    - name: Install Nginx
      apt:
        name: nginx
        state: present

    - name: Add index page
      template:
        src: index.html
        dest: /var/www/html/index.html

    - name: Start Nginx
      service:
        name: nginx
        state: started

The result is an application that can be easily updated with an Ansible command, leading to zero downtime.

Blue/Green Deployments

The idea behind this approach is to have two environments running different versions of your application. You can gradually roll out a percentage of the changes or do it all at once. The blue environment serves traffic from users in production, while the green environment hosts the new version of the application where you can perform tests and validations. After the green environment is tested, you can redirect traffic to it. If you encounter any problems, you can roll back by redirecting traffic to the blue environment.

One of the most amazing things about AWS and Terraform is their community. As you can see, we can leverage public modules to compose our application. In this case, we switch between environments using a flag. Each time we update our application, the new version remains idle and available, while the former live server continues receiving traffic until we are confident that the idle server is functioning properly and make the switch.



provider "aws" {

  region  = "us-east-2"

}

variable "production" {

  default = "green"

}

module "base" {

  source     = "terraform-in-action/aws/bluegreen//modules/base"

  production = var.production

}

module "green" {

  source      = "terraform-in-action/aws/bluegreen//modules/autoscaling"

  app_version = "v1.0"

  label       = "green"

  base        = module.base

}

module "blue" {

  source      = "terraform-in-action/aws/bluegreen//modules/autoscaling"

  app_version = "v2.0"

  label       = "blue"

  base        = module.base

}

output "lb_dns_name" {

  value = module.base.lb_dns_name

}

Canary Deployments (Deploying Serverless Applications Gradually)

This strategy involves gradually releasing the software to a small subset of users. By doing so, we can validate whether everything is working as expected and continue to increase the percentage of users who can use the newest version of the application. If a problem is identified, we can easily roll back by reducing the amount of traffic to the new version to zero.

With SAM, it is possible to perform traffic validations and rollbacks out of the box using CloudWatch alarms. This is very handy when changes in the logic might be risky.

Canary deployments are more focused on people than servers. In this case, we focus on the number of users who will experience the new version of the Lambda functions. We can monitor our application and automatically roll back if needed. Keep in mind that this process is safer but slower. Here are the options available in AWS SAM:

Canary10PercentMinutes
Linear10PercentEveryMinutes
AllAtOnce



Resources:

  AWSCommunityLambda:

    Type: AWS::Serverless::Function

    Properties:

      Handler: index.py

      Runtime: python3.11

      CodeUri: functions/api/handler

      AutoPublishAlias: live

      DeploymentPreference:

        Type: Canary10Percent10Minutes 

        Alarms:

          - !Ref AliasErrorMetricGreaterThanZeroAlarm

          - !Ref LatestVersionErrorMetricGreaterThanZeroAlarm

        Hooks:

          PostTraffic: !Ref PostTrafficLambdaFunction

Conclusions

Zero-downtime deployments are key to building trust in the highly dynamic world of cloud-native applications.
SAM has powerful options to deploy infrastructure in a safe manner.
Ansible can work in combination with Terraform to create a better DevOps experience by reducing the blast radius through just updating the application.
Immutable infrastructure is usually a better approach because you have the same configuration deployed consistently, rather than a server where many changes have been applied, potentially leading to configuration differences.
Immutable infrastructure also has downsides, such as requiring the rebuilding of an image from a server template and redeploying for a simple change.
OpenTofu is an excellent option for deploying infrastructure. I haven't tried it before, but it seems like the community is doing a great job, and the new releases will have more and more features that are not yet supported by Terraform.

References

Terraform in Action - Practical uses and strategies for implementing Terraform, a tool for building, changing, and managing infrastructure.
Terraform: Up & Running - Friendly introduction to Terraform, including its complex concepts, a lot of examples and how to get started with it.
GruntWork

Not too long ago, deploying in production was a tedious task that involved multiple actors. Nowadays, we have a wealth of open-source software and developer-focused tools that emphasize immutability, automation, productivity-enhancing AI, and powerful IDEs.

There has never been a better time to be an engineer and create value in society through software.

If you enjoyed the articles, visit my blog jorgetovar.dev

Terraform Module Gotchas - Inline block example S3 AWS

Jorge Tovar — Thu, 09 May 2024 21:46:10 +0000

Software is all about composition.

Modules

Modules are the key ingredient for writing reusable, maintainable code. When planning to deploy AWS infrastructure, it's better to leverage this functionality and be aware of the common gotchas to avoid conflicts with other Terraform configurations.

Modules are essentially terraform files with resource definitions that you can use elsewhere.

Terraform

Terraform and Infrastructure as Code are no exceptions when we talk about software composition. We should use modules to create and compose our building blocks,
just as we use functions to compose and create our business logic.

There are two gotchas we need to be aware of when using modules in Terraform:

File paths
Inline blocks

GitHub Repository

Inline blocks

The configuration of some Terraform resources can be done through inline blocks or independent resources.

Inline blocks example

These are properties of the resource that you can define within the same resource. For example, ingress and egress are inline blocks, and there are plenty of them in Terraform, such as versioning in an S3 bucket and many more.

resource "aws_security_group" "alb" {
  name = "${var.cluster_name}-alb"

  ingress {
    from_port   = local.http_port
    to_port     = local.http_port
    protocol    = local.tcp_protocol
    cidr_blocks = local.all_ips
  }

  egress {
    from_port   = local.any_port
    to_port     = local.any_port
    protocol    = local.any_protocol
    cidr_blocks = local.all_ips
  }
}

Separate resource customization example

These are properties of the resource that you define as independent Terraform resources. In this case, we create our bucket using a module, but we modify some properties of this resource using new resources. In the example, we have added versioning to the bucket.

module "s3_bucket" {
  source = "../modules/separate-resource"
  name   = "jorgetovar"
}


resource "aws_s3_bucket_versioning" "versioning" {
  bucket = module.s3_bucket.bucket_name
  versioning_configuration {
    status = "Enabled"
  }
}

Inline blocks vs separate resources is a trade-off, and we must be cautious when using inline properties because we lose some flexibility in defining and customizing resource properties from the client's perspective.

There is also the possibility of conflicts between inline blocks and client resources, which can lead to unexpected results. Finally, inline blocks have been deprecated in Terraform, perhaps for these reasons.

Additionally, it's important to remember that defining Terraform configuration within both the module itself and the file that uses the module can lead to several issues.

Complexity

We should organize our modules in a way that we can easily interact with a part of the infrastructure without having to
understand the whole system.

Our job is to remove the accidental complexity and make the system easier to understand. When we use inline blocks, we
are adding complexity to the system (properties and thing that we may not need to know about).

Abstractions

But we can also leverage abstractions and hide some details that are unnecessary, if all the buckets that we create have
versioning, maybe it makes more sense to move that resource to the module.

File Paths

By default, Terraform uses the path relative to the current working directory, in our case, the localhost root module.
Therefore, the module should be able to read files from its folder instead of the one where we are currently executing the commands.

We can leverage the path variables of terraform to read the files that are in the module folder.The path variables are:

Path.module : The path to the module folder
Path.root : The path to the root module folder
Path.cwd : The path to the current working directory

resource "local_file" "mad_libs" {
  count    = var.num_files
  filename = "build/madlibs-${count.index}.txt"
  content  = templatefile("${path.module}/${element(local.templates, count.index)}",
    {
      nouns      = random_shuffle.random_nouns[count.index].result
      adjectives = random_shuffle.random_adjectives[count.index].result
      verbs      = random_shuffle.random_verbs[count.index].result
      adverbs    = random_shuffle.random_adverbs[count.index].result
      numbers    = random_shuffle.random_numbers[count.index].result
      name       = "Jorge Tovar"
    })
}

Conclusion

Terraform modules may be the best way to deploy our resources in AWS, but we must be aware of the challenges that come with this useful feature.

Modules allow us to create abstractions and reduce the complexity of our infrastructure.

If you enjoyed the articles, visit my blog jorgetovar.dev

Provisioned Concurrency - Reduce Cold Starts in AWS Lambda Functions Part 1

Jorge Tovar — Tue, 12 Mar 2024 20:44:52 +0000

This series of AWS Lambda articles focuses on performance and best practices. This is part 1, where we explore Lambda cold starts, warming a lambda, and provisioned concurrency. In parts 2 and 3, we will discuss Deployment preferences, Powertools, Observability and Auto-scaling Provisioned concurrency using metrics and patterns, all of this with Infrastructure as Code 🧑🏻‍💻.

AWS Lambda functions have become one of the most important cloud services provided by Amazon. They are the cornerstone of many applications, serving as the core business logic for Serverless applications. They are also the default choice for operating and handling asynchronous or simple tasks that don't require a constantly running component. Over time, Lambda has evolved to be a fully event-driven service with numerous triggers and seamless integration with almost any AWS service.

If you want to go straight to the code: 🚀
GitHub Code example

However, one challenge remains in our quest for creating reliable and performant applications: Cold Starts. Despite AWS investing heavily to reduce this time, and for many use cases, it's something we can tolerate, it isn't always the case. Sometimes we need a predictable response latency for our product.

Cold Starts and Latency

A cold start in AWS Lambda refers to the initial process of spinning up a new execution environment for a function when it's invoked for the first time or after a period of inactivity.

To understand this, we need to know the lifecycle of an AWS Lambda function. Here are the steps that need to be performed in order to get a response from our function:

Download the code (Cold start)
Create a new execution environment (Cold start)
Execute initialization code (Cold start)
Execute code handler
Shutdown



# Initialization code
import boto3
s3_client = boto3.client('s3')

# Code handler
def lambda_handler(event, context):

    response = s3_client.list_buckets()

    bucket_names = [bucket['Name'] for bucket in response['Buckets']]

    return {
        'statusCode': 200,
        'body': bucket_names
    }

As illustrated in the code above, we can conceptualize this process as occurring in three phases:

1.Initialization Phase

During this phase, our code is downloaded, the execution environment with extensions and runtime is created, and the only aspect we control is the function initialization code. This code, outside of the handler, executes once during every cold start.

2. Invoke Handler

This phase executes every time we call our lambda function.

3. Shutdown

Environments are shut down after a non-deterministic period of inactivity. In such cases, a cold start occurs upon the next invocation.

As you can deduce, in terms of cold starts, we only have control over the function initialization. Thus, we should aim to take advantage of any additional invocations within the same execution environment, known as a "warm start."

Warm Start

With multiple concurrent calls, AWS can utilize already provisioned environments, eliminating the need to download the code and run the initialization code again. Paradoxically, increased traffic can lead to improved performance.

Deployments

Every time we update our code, it results in a cold start. This behavior ensures that we run the function with the latest code.

Function Warmers

In the serverless community, there are libraries and in-house mechanisms used to "warm" Lambda via pings. However, this doesn't eliminate all cold starts, and sometimes it doesn't help in production environments when we experience spikes in traffic and need all of our concurrent environments available to serve user requests.

We can use EventBridge rules to keep our Lambda warm, or use tools to simulate traffic. However, I'm not a big fan of this approach. In such cases, it may be worth considering the possibility of having a running container.

Reducing Cold Starts with Provisioned Concurrency

If our product requires predictable start times and latency, provisioned concurrency is the way to go. Unlike Lambda's on-demand default behavior, we can avoid the initial phase and recurrent tasks of downloading and executing the initialization code.

These environments are provided ahead of time, but remember:

Everything in software architecture is a trade-off.

We may end up with a higher cost of running our application since we have a running component available instead of getting something on demand.

Provisioned concurrency works in conjunction with Lambda versions instead of using the $LATEST code as we usually do. We also don't need to optimize initialization code since this is executed before creating the execution environment.

We may still have cold starts due to the defined capacity in our provisioned concurrency. Therefore, we can configure the amount of ready-to-use Lambdas in our system.

We can create auto-scaling policies to make this more robust and reduce the cost of operating our application. If we have predictable patterns or metrics, we can reduce the amount of provisioned concurrent executions.

We can also define some rollout and deployment preferences as part of this strategy, for example, using canary or linear deployments.

Hands-on Example

Provisioned concurrency repository

Every time that we make a HTTPs call for the very first time we would see the impact of the Cold start, with a response time that seems to be pretty high -> 2 seconds.



{
    "message": "Hello community builders - from AWS Lambda!"
}

We usually get a new Log stream if we update the Lambda.

Finally in the Logs we would see something like this, where we can validate the Init duration in our case it is 824.46 ms

REPORT RequestId: e8500c25-5c5f-4ffe-867e-60741c3b1df5 Duration: 18.57 ms Billed Duration: 19 ms Memory Size: 128 MB Max Memory Used: 64 MB **Init Duration: 824.46 ms* XRAY TraceId: 1-65f096ba-200e28165f811e694170deef SegmentId: 13f970807474e003 Sampled: true*

We can go to the Configuration, Concurrency tab and set up everything.

Configuration

Create a Lambda version
Configure the number of available execution environments

Results

After the deployment has been successful, if we make an HTTP call, we can avoid most of the cold starts and see consistent response times of 400 ms.

We can implement all of this in our CI/CD and Infrastructure as Code processes, but I wanted to provide an overview with the console so you can feel comfortable playing with the settings and doing a proof of concept before diving deep and deploying this.

Clean Up

Remove all the resources using SAM framework

Conclusion

Provisioned concurrency is the preferred solution for production workloads that rely on predictable response times and want to avoid cold starts.

It is important to emphasize that the biggest problems in the software development and delivery process are organizational and people-related. In the end, building applications is a social activity, but the more we solve technical problems and improve communication, the easier it should be to reach production. Happy coding! 🎉

If you enjoyed the articles, visit my blog jorgetovar.dev

AWS S3 Replication (Cross-Region / Cross-Account)

Jorge Tovar — Fri, 19 Jan 2024 20:16:04 +0000

Why Replication is Important

S3 replication is a crucial feature of the AWS storage service, primarily because it allows us to establish a disaster recovery strategy, potentially minimize latency for global reach, or implement redundancy for maintaining a subset of production data in lower environments, thereby enhancing operational efficiency.

It's a pretty common use case, and it's better to know how to implement it quickly.

Lets get our hands dirty🧑🏻‍💻

Code template

Steps:

Copy the template.
Deploy the resources using CloudFormation.
Validate by placing an object in the source bucket.
If desired, remove the bucketName property to avoid collisions.
Reap the benefits of learning something new and use it wisely.
Delete the stack.




AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'

Resources:
  ReplicationRole:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: 'Allow'
            Principal:
              Service: 's3.amazonaws.com'
            Action: 'sts:AssumeRole'
      Policies:
        - PolicyName: 'ReplicationPolicy'
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: 'Allow'
                Action:
                  - 's3:GetBucketVersioning'
                  - 's3:ListBucket'
                  - s3:GetReplicationConfiguration
                  - s3:GetObjectVersionForReplication
                  - s3:GetObjectVersionAcl
                  - s3:GetObjectVersionTagging
                  - s3:GetObjectRetention
                  - s3:GetObjectLegalHold
                Resource: '*'
              - Effect: 'Allow'
                Action:
                  - 's3:ReplicateObject'
                  - 's3:ReplicateDelete'
                  - 's3:ReplicateTags'
                  - 's3:GetObjectVersionTagging'
                  - 's3:ObjectOwnerOverrideToBucketOwner'

                Resource: '*'

  BucketSource:
    Type: 'AWS::S3::Bucket'
    Properties:
      VersioningConfiguration:
        Status: 'Enabled'
      ReplicationConfiguration:
        Role: !GetAtt ReplicationRole.Arn
        Rules:
          - Destination:
              Bucket: !GetAtt BucketReplica.Arn
            Prefix: ''
            Status: 'Enabled'
      BucketName: 'aws-community-builders-source'

  BucketReplica:
    Type: 'AWS::S3::Bucket'
    Properties:
      VersioningConfiguration:
        Status: 'Enabled'
      BucketName: 'aws-community-builders-replica'

  BucketReplicaPolicy:
    Type: AWS::S3::BucketPolicy
    Properties:
      Bucket: !Ref BucketReplica
      PolicyDocument:
        Version: 2012-10-17
        Statement:
          - Sid: "Object Level Permissions"
            Effect: "Allow"
            Principal:
              AWS: !GetAtt ReplicationRole.Arn
            Action:
              - "s3:ReplicateObject"
              - "s3:ReplicateDelete"
            Resource: !Sub "arn:aws:s3:::${BucketReplica}/*"
          - Sid: "Bucket Level Permissions"
            Effect: "Allow"
            Principal:
              AWS: !GetAtt ReplicationRole.Arn
            Action:
              - "s3:List*"
              - "s3:GetBucketVersioning"
              - "s3:PutBucketVersioning"
            Resource: !Sub "arn:aws:s3:::${BucketReplica}"

Create the template

Deploy the resources

Validate deployment

Put some objects in the Source Bucket
You will see the replication rules under management tab

It takes some time to replicate the object, depending on the size and regions used. However, after a few seconds or minutes, you will see the object in the replica bucket.

Delete the stack
Make sure you empty the bucket before deleting the stack otherwise you will get an error.

Explanation

We created two buckets (for simplicity purposes in the same region), but it can be created even in another account, and the idea remains the same: we need to replicate our data.

We have a role with the necessary privileges to move and replicate the data.

We added the required policies in the replica bucket to allow the role to put the information.

Notes:

We can be more granular in the privileges and policies, for example, to explicitly state the bucket.
We can use batch operations to move data that is already present in the bucket; batch operations will leverage the replication and role configurations in place.

Additional Tools

I always try to automate and validate many things before pushing to main, and tools like this can save you a lot of time.

https://github.com/aws-cloudformation/cfn-lint

You can use cfn-lint to check the CloudFormation template "crr-template.yml" for syntax errors, best practices, and potential issues.

Infrastructure as Code

Infrastructure as Code (IaC) is crucial for automating and managing IT infrastructure efficiently, enabling version control, repeatability, and collaboration in a consistent and scalable manner.

Conclusion

Cross-Replication, or even Cross-Account Replication, is a useful feature to ensure a highly available application, providing redundancy and fault tolerance.

Replication is also fundamental to have batch operations that are a fast way to move TB of data.

S3 remains one of the most useful, cheapest, and powerful services of AWS.

Happy coding! 🎉

If you enjoyed the articles, visit my blog at jorgetovar.dev.

Java Evolution: From Java 8 to Java 17, Compared with Kotlin and Clojure

Jorge Tovar — Thu, 21 Sep 2023 18:24:16 +0000

Java 21 brings exciting news for all developers 🎉 .

Java remains a fantastic language, and the speed at which new features are released is outstanding.

I have a repository that showcases Java's evolution from version 8 to the latest Java 17, complete with comparisons to Kotlin and Clojure. I'm currently working on updating it with Java 21's promising new features.

Explore these new features, especially if you're still using Java 8, and see how they're implemented in different languages. 📚

Local-Variable Syntax for Lambda Parameters
Pattern Matching
Records
Sealed Classes
Text Blocks
Switch Expressions

GitHub

Code example

Local-Variable Syntax for Lambda Parameters

In Java, starting from version 11, you can use the var keyword for lambda parameters, simplifying lambda expressions by allowing type inference. Here's an example:

// Java 11 Lambda Expression with var
(var a, var b) -> a + b;

Pattern Matching

Java introduced pattern matching in later versions, enhancing the way you work with instanceof checks and type casting. It provides more concise and readable code. Here's a simplified example:

// Pattern Matching in Java
public static boolean available(Book book) {
    return switch (book) {
        case FictionBook fictionBook -> fictionBook.available();
        case ProgrammingBook programmingBook -> programmingBook.exists();
        case PsychologicalBook psychologicalBook -> psychologicalBook.imAAvailable();
        case null, default -> false;
    };
}

Records

Records, introduced in Java 16, provide a compact way to declare classes that are simple data carriers. They automatically generate methods like equals(), hashCode(), and toString(). Example:

// Java Record
public record BookResponse(String json, Double version) {
}
BookResponse bookResponse = new BookResponse("{}", 2d);
System.out.println(bookResponse); // Automatically calls toString()

Sealed Classes

Java introduced sealed classes to restrict which other classes or interfaces may extend or implement them. It helps in defining a finite set of subclasses. Example:

// Sealed Class in Java
public sealed class Book permits FictionBook, ProgrammingBook, PsychologicalBook

Text Blocks

Text Blocks, available from Java 13 onwards, simplify working with multi-line strings. They allow you to write formatted text with minimal escape sequences:

// Java Text Block
json += """
        {"title":"%s",
        "author":"%s",
        "pages":%s,
        "karma":%s,
        "eBook":%s,
        "rate":%s,
        "category":"%s"}
        """

Switch Expressions

Java introduced switch expressions in Java 12. They allow you to use a switch statement as an expression, making code more concise and readable:

// Java Switch Expression
var book = switch (category) {
    case fiction -> {
        karma = 25;
        yield new FictionBook(title, author);
    }
    case programming -> {
        karma = 40;
        yield new ProgrammingBook(title, author);
    }
    case psychological -> {
        karma = 30;
        yield new PsychologicalBook(title, author);
    }
};

These features have been introduced to make Java code more expressive and readable while reducing boilerplate code. You can choose the appropriate feature based on your project's requirements and Java version.

Conclusion

While I believe that Java 17 and its new releases are undeniably impressive, I can't help but acknowledge that Kotlin offers a multitude of features that enhance the developer experience and make coding more enjoyable. Clojure, on the other hand, has been a revelation for me, highlighting that you don't necessarily need classes and object-oriented programming to elegantly solve complex business logic challenges. It's a reminder that diverse programming languages can offer unique perspectives and solutions.

How to Manage Terraform State in AWS

Jorge Tovar — Wed, 26 Jul 2023 18:56:27 +0000

You have probably noticed that every time you run the terraform apply or terraform plan commands, Terraform somehow identifies which resources have been created, which have not, and the differences with respect to the configurations. The response is the Terraform state, which is a JSON file containing all the relevant information about the managed resources and their current state.

Code on GitHub

https://github.com/jorgetovar/terraform-aws-remote-state

You may have also noticed that, in order to work as a team, we usually need to share this state. At the beginning of a project with IaC, we usually save the Terraform state in the GitHub repository. However, this is a problem because the Terraform state may contain sensitive information, such as secrets, that we do not want to be public.

In addition, resolving conflicts between infrastructure changes can be a headache. In the past, I made this mistake partly because we did not use a pipeline to deploy resources; instead, we did it from our local environment.

Problems with the local terraform.tfstate 😵

The file that is generated locally by default is terraform.tfstate, and the problem in projects with multiple team members is that somehow we have to share this file. Another problem is race conditions; in fact, two engineers can deploy simultaneously, causing inconsistencies and corruption of the state file.

Finally, it is important to isolate our state file in relation to the deployed environments. It was a challenging project at the time, but as soon as we moved the state to AWS, generated locks with DynamoDB, and executed from a pipeline with strictly necessary permissions, everything returned to normal, and it was even enjoyable to make changes to the infrastructure.

Role and privileges required to create the Remote state 🤖

To create our remote state, we need to create a role with the necessary privileges, a table in DynamoDB, and finally an S3 bucket.

Let's break down the code step by step:

First, a data block named aws_caller_identity is defined, which is used to obtain the identity of the one executing the current infrastructure changes. This is used later to define the role's permissions.

data "aws_caller_identity" "current" {}

Next, a block of local variables is defined. In this case, a local variable named principal_arns is defined. This variable will be assigned to var.principal_arns if it has a value; otherwise, it will be assigned to an array with the ARN (Amazon Resource Name) obtained from the aws_caller_identity data block. This variable will be used later to specify who can assume this role.

locals {
  principal_arns = var.principal_arns != null ? var.principal_arns : [data.aws_caller_identity.current.arn]
}

Next, a resource of type aws_iam_role is defined, which is the IAM role we are creating. It is given a name based on the local.namespace variable, and in the assumption policy, it is specified which entities (in this case, the ARNs stored in local.principal_arns) can assume this role.

resource "aws_iam_role" "iam_role" {
  name = "${local.namespace}-tf-assume-role"

  assume_role_policy = <<-EOF
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Action": "sts:AssumeRole",
          "Principal": {
              "AWS": ${jsonencode(local.principal_arns)}
          },
          "Effect": "Allow"
        }
      ]
    }
  EOF

  tags = {
    ResourceGroup = local.namespace
  }
}

Then, a data block named aws_iam_policy_document is defined, which is used to specify the permissions that will be granted to the entity or service assuming the role. Three policy statements are defined here to allow access to specific resources, such as listing an S3 bucket, accessing objects in the S3 bucket, and accessing a DynamoDB table.

data "aws_iam_policy_document" "policy_doc" {
  statement {
    actions = [
      "s3:ListBucket"
    ]

    resources = [
      aws_s3_bucket.state_bucket.arn
    ]
  }

  statement {
    actions = [
      "s3:GetObject",
      "s3:PutObject",
      "s3:DeleteObject"
    ]

    resources = [
      "${aws_s3_bucket.state_bucket.arn}/*",
    ]
  }

  statement {
    actions = [
      "dynamodb:GetItem",
      "dynamodb:PutItem",
      "dynamodb:DeleteItem"
    ]
    resources = [aws_dynamodb_table.state_lock_table.arn]
  }
}

Next, a resource of type aws_iam_policy is created, which is the IAM policy that contains the permissions defined in the previous data block.

resource "aws_iam_policy" "iam_policy" {
  name   = "${local.namespace}-tf-policy"
  path   = "/"
  policy = data.aws_iam_policy_document.policy_doc.json
}

Finally, the created policy is attached to the IAM role using the aws_iam_role_policy_attachment resource.

resource "aws_iam_role_policy_attachment" "policy_attach" {
  role       = aws_iam_role.iam_role.name
  policy_arn = aws_iam_policy.iam_policy.arn
}

Resources of the Remote state 💾

aws_s3_bucket: Creates an S3 bucket in AWS. It is given a name based on local.namespace, and it is specified whether the bucket should allow forceful deletion or not, using the value of var.force_destroy_state. It is also assigned a tag ResourceGroup based on local.namespace.
aws_s3_bucket_server_side_encryption_configuration: Configures server-side encryption for the previously created S3 bucket. A rule is defined to apply default server-side encryption to all objects in the bucket using a KMS (Key Management Service) key specified in aws_kms_key.kms_key.arn.
aws_s3_bucket_versioning: Enables versioning for the previously created S3 bucket. The versioning_configuration with status = "Enabled" indicates that object versioning is enabled on the bucket.
aws_s3_bucket_public_access_block: Configures public access block for the S3 bucket. This ensures that certain settings do not allow public

access to objects in the bucket, preventing unwanted public policies or permissions.

aws_dynamodb_table: Creates a DynamoDB table in AWS. The table is named based on local.namespace and has a hash key called "LockID" of type "S" (string). The billing mode is set to "PAY_PER_REQUEST," meaning you only pay for the operations performed. It is also assigned a tag ResourceGroup based on local.namespace.

resource "aws_s3_bucket" "state_bucket" {
  bucket        = "${local.namespace}-state-bucket"
  force_destroy = var.force_destroy_state

  tags = {
    ResourceGroup = local.namespace
  }
}

resource "aws_s3_bucket_server_side_encryption_configuration" "example" {
  bucket = aws_s3_bucket.state_bucket.id

  rule {
    apply_server_side_encryption_by_default {
      kms_master_key_id = aws_kms_key.kms_key.arn
      sse_algorithm     = "aws:kms"
    }
  }

}

resource "aws_s3_bucket_versioning" "versioning_example" {
  bucket = aws_s3_bucket.state_bucket.id
  versioning_configuration {
    status = "Enabled"
  }
}

resource "aws_s3_bucket_public_access_block" "s3_bucket" {
  bucket                  = aws_s3_bucket.state_bucket.id
  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

resource "aws_dynamodb_table" "state_lock_table" {
  name         = "${local.namespace}-state-lock"
  hash_key     = "LockID"
  billing_mode = "PAY_PER_REQUEST"
  attribute {
    name = "LockID"
    type = "S"
  }
  tags = {
    ResourceGroup = local.namespace
  }
}

Output 📚

description: It is a description of what this output contains. It provides information about the values that will be included in the output. In this case, it mentions that the output will contain configuration details, such as the name of the created S3 bucket, the AWS region of the S3 bucket, the ARN of the IAM role created for the backend, and the name of the created DynamoDB table for locking.
value: It is the actual value that will be included in the output. Here, a map is defined that contains the following fields:
- bucket: Contains the name of the created S3 bucket. This is obtained through the reference aws_s3_bucket.state_bucket.bucket, where aws_s3_bucket.state_bucket is the resource that creates the bucket, and .bucket refers to the "bucket" attribute of that resource.
- region: Contains the AWS region name where the S3 bucket was created. This is obtained through the reference data.aws_region.current.name, where data.aws_region.current is a data block that retrieves information about the current region, and .name refers to the "name" attribute of that data block.
- role_arn: Contains the ARN (Amazon Resource Name) of the IAM role created for the backend. This is obtained through the reference aws_iam_role.iam_role.arn, where aws_iam_role.iam_role is the resource that creates the IAM role, and .arn refers to the "arn" attribute of that resource.
- dynamodb_table: Contains the name of the DynamoDB table created for locking. This is obtained through the reference aws_dynamodb_table.state_lock_table.name, where aws_dynamodb_table.state_lock_table is the resource that creates the DynamoDB table, and .name refers to the "name" attribute of that resource.

Isolating the state by environment ♟️

To isolate the Terraform state and avoid conflicts when working with different environments (e.g., development, staging, production), there are two main approaches: using workspaces and designing the project layout.

Workspaces (Espacios de trabajo):

Terraform provides the concept of "workspaces" to handle multiple isolated instances of the configuration state. Each workspace is an independent copy of the state, allowing different configurations to coexist without interfering with each other.

To use workspaces:

Create a new workspace: You can create a new workspace with the terraform workspace new <nombre_workspace> command.
Switch workspaces: You can switch between workspaces with the terraform workspace select <nombre_workspace> command.
List workspaces: You can see a list of available workspaces with terraform workspace list.

It is important

to note that workspaces share the same configuration code, so you must be careful when sharing common resources between them to avoid conflicts.

Layout of the project:

The design of the project layout is a practice that involves organizing the Terraform code into different directories to isolate environments and components. Each directory contains its own configuration file and state.

For example:

project
|-- dev
|   |-- main.tf
|   |-- variables.tf
|   |-- ...
|-- staging
|   |-- main.tf
|   |-- variables.tf
|   |-- ...
|-- production
|   |-- main.tf
|   |-- variables.tf
|   |-- ...
|-- modules
|   |-- module-aws-community-builder
|   |   |-- main.tf
|   |   |-- variables.tf
|   |   |-- ...
|   |-- module-jt-state
|   |   |-- main.tf
|   |   |-- variables.tf
|   |   |-- ...

Each directory (dev, staging, production) represents a different environment and contains its own configuration file, variables, and may have its own Terraform state.

The design of the project layout provides a clearer isolation between environments and allows for greater flexibility in managing the state and configurations.

Conclusion 📖

When working with infrastructure as code (IaC), the importance of isolating, locking, and managing the state lies in the severe consequences that errors can have in this context. Unlike developing applications, errors in infrastructure code can affect all applications, databases, etc. Thus, it is crucial to include additional safety mechanisms when working with IaC.

provider "aws" {
  region = "us-west-2"
}

module "remote_state" {
  source  = "jorgetovar/remote-state/aws"
  version = "1.0.2"
}

output "state_config" {
  value = module.s3backend.config                      
}

Happy coding! 🎉

If you enjoyed the articles, visit my blog at jorgetovar.dev.