Forem: Ioannis Moustakis

Atlantis vs. Terraform Cloud / Terraform Enterprise – Comparison

Ioannis Moustakis — Wed, 14 Sep 2022 19:00:41 +0000

This blog post will look into two Infrastructure as Code (IaC) automation tools, Atlantis and Terraform Cloud/Enterprise and analyze their similarities and differences.

Atlantis allows users to orchestrate Terraform automation through pull requests by using comments, and it’s a great tool suited for small-scale projects and casual usage. Terraform Cloud provides a specialized CI/CD platform for Terraform automation and a great remote backend solution. Terraform Cloud is more scalable than Atlantis but offers fewer extensibility options.

What is Atlantis

Atlantis is an open-source and self-hosted Terraform “pull request-based” automation tool. It offers an easy way to automate the Terraform workflow using pull request comments. On every new pull request, Atlantis automatically runs the terraform plan command and comments the output back on the pull request. After the suggested changes have been reviewed, a team member can leave a pull request comment with a special meaning to apply the changes.

A great benefit of the Atlantis workflow is that it doesn’t add a new user interface (UI) for operators and developers but integrates nicely with your choice’s version control system (VCS) provider. It provides the option to perform code reviews and Terraform operations via the same graphical user interface. Users don’t need access credentials for the infrastructure provider, and errors can be caught during the code review step. With the Atlantis model, each pull request contains a detailed audit log of changes made via Terraform.

Atlantis self-hosted runners can be given an identity native to your cloud (e.g., AWS instance profile) for access without credentials to the state and managed resources. They can also be configured to run inside the Virtual Private Cloud (VPC) to access local resources (e.g., VPC-internal database) but need inbound connectivity from the VCS provider to receive webhooks. Its configuration is primarily done using environment variables passed to the statically linked binary and the YAML file.

Atlantis is stateless, but one of its main drawbacks is that it doesn’t support a high-availability setup or any scaling and queueing support. To accommodate scaling and highly available setups, a substantial engineering effort and creativity are required to build a custom in-house solution.

Flexibility is one of the core advantages of Atlantis, as it allows easy integration with other Terraform-helper tools(e.g., tfsec, checkov, Infracost, or Terratag). It can work with Terraform wrappers, such as Terragrunt, out of the box and even add some of Terragrunt’s features to vanilla Terraform – like before and after hooks for every execution stage (init, plan, apply, etc.).

Atlantis has a vibrant and active community, with new versions being released often. Something to note here is that although the development is active and there are regular contributions, the efforts aren’t focused on new major features since the lead contributor moved to Hashicorp.

Overall, it’s a great tool suited for small-scale operations and infrastructure topologies. It is much appreciated by its user community and offers a flexible automation solution for occasional use. Having said that, it is strongly limited by its architecture, and scaling it isn’t straightforward. If your company has large-scale infrastructure needs, other more robust and mature solutions exist.

What is Terraform Cloud

Terraform Cloud is a more comprehensive infrastructure provisioning tool that works exclusively for Terraform, developed by Hashicorp. It provides a scalable solution to automate infrastructure delivery, handle compliance, and manage resources in a cloud-agnostic way, utilizing Terraform. It is Hashicorp’s SaaS managed service offering targeting the Terraform workflow.

One of Its main offerings is a specialized CI/CD platform to standardize Terraform deployments and reduce their time. It supports an excellent remote state backend and an API for remote Terraform operations and integration with existing workflows. It integrates with VCS providers and allows fully automated, or manual approval checks for infrastructure provisioning flows.

Interaction with Terraform Cloud can be achieved with the command-line interface (CLI), UI, API, or CI jobs. The remote or enhanced backend allows teams to run the Terraform binary from their laptops or a third-party CI job, but the operation is executed on a remote machine. This is especially useful for one-off administrative tasks like tainting or migrating resources – things that are not trivial with Atlantis and may require dedicated solutions like tfmigrate.

Terraform Cloud offers basic security essentials such as RBAC with custom workspace permissions and different access levels for different types of users. Its integration with single sign-on (SSO) allows administrators easy user onboarding and management.

Unlike Atlantis, Terraform Cloud architecture is highly scalable, so it will take a while to outgrow it. It offers a shared state, distributed execution, concurrent runs, notifications for workspace events, and VCS integrations to support its scalability.

Teams can leverage Terraform Cloud’s rich API imperatively from external scripts or declaratively from Terraform itself, using their provider. Managing Terraform with Terraform is often a secret to managing IaC at scale in dynamic organizations. It also supports exporting audit logs to external systems via its API.

One drawback of Terraform Cloud compared to Atlantis is that it is less extensible. While Atlantis lets you execute arbitrary shell commands as part of your Terraform job, Terraform Cloud depends on clever hacks like the null resource or an external wrapper. For example, suppose you are a Terragrunt user. In that case, you may want to have a CI job (e.g., Jenkins or GitHub Actions) trigger Terragrunt that’s later shelling out to Terraform, which executes the job on your remote Terraform Cloud environment. This extra layer complicates the architecture and workflow and introduces another party to a sensitive flow.

Terraform Cloud offers some native integrations and third-party tools to incorporate into the Terraform workflow, like their proprietary policy-as-code framework, Sentinel. Leveraging Sentinel, you can create security and compliance guardrails. The disadvantage of Sentinel is that it’s not an industry standard and open-source like Open Policy Agent (OPA). HashiCorp recently announced publishing reusable Sentinel policies in their public Terraform Registry, which may give Sentinel a new lease of life in the Terraform ecosystem.

Last but not least, Terraform Cloud recently announced a drift detection feature, which allows you to monitor the synchronization between your resources and their respective Terraform definitions. This feature can, to some extent, be replicated using /plan and /apply HTTP endpoints from Atlantis, but it’s a far cry from the native, built-in solution that Terraform Cloud offers.

Terraform Cloud vs. Terraform Enterprise

If for any reason (compliance, regulatory needs, etc.), your organization needs an on-premise version of Terraform Cloud, you can use Terraform Enterprise. Terraform Enterprise is a self-hosted distribution of Terraform Cloud.

It provides organizations with a private environment installation of the Terraform Cloud instance and enterprise-grade features like single sign-on, compliance enforcement with policies, and audit logging.

If you plan on hosting your own Terraform Enterprise distribution, have a look at the requirements, reference architectures for common cloud providers, and the installation and configuration guide.

Atlantis and Terraform Cloud Similarities

On-premise Support

Both Atlantis and Terraform Enterprise offer the possibility to host your own installation of the tools.

Integration with VCS providers

Most of the standard VCS providers are supported and integrated seamlessly with Atlantis and Terraform Cloud/Enterprise.

Integration with CI/CD

Both tools can be incorporated into your organization’s existing CI/CD flows and work in parallel with existing continuous integration jobs.

Atlantis and Terraform Cloud Differences

SaaS Offering
Terraform Cloud/Enterprise is a managed service SaaS offering, while Atlantis doesn’t have a similar offering.

User Interface

Atlantis uses the same UI as the VCS provider that you are using and allows operators to trigger automation jobs from pull requests. Terraform Cloud/Enterprise comes with its own UI and portal.

Open Source Availability

Atlantis is entirely open-source and free, while Terraform Cloud/Enterprise is a proprietary solution, although it offers a free version.

Remote State Backend

As part of its offering, Terraform Cloud provides an excellent backend for the Terraform state, while Atlantis doesn’t.

High Availability

Terraform Cloud/Enterprise is designed for scale and offers a highly available setup. Scaling and building highly available setups with Atlantis requires additional effort.

Flexibility/Extensibility

Atlantis is very flexible and can integrate with other helper tools easily. Extending Terraform Cloud functionality is a bit more cumbersome.

Security Features

Atlantis keeps all the infrastructure changes and ties them to pull requests that can be used as audit logs. Terraform Cloud offers more elaborate security essentials like RBAC, single sign-on with SAML, and an audit log.

Drift Detection

Terraform Cloud offers drift detection, whereas Atlantis doesn’t by default, although similar functionality can be replicated with additional effort.

Cost Calculation

Terraform Cloud offers cost estimation, whereas Atlantis doesn’t by default, although similar functionality can be replicated by adding external tooling.

Atlantis and Terraform Cloud Synergies

Actually, Atlantis and Terraform Cloud can be used together since Atlantis integrates seamlessly with Terraform Cloud/Enterprise. It doesn’t matter which flavor or Terraform Cloud/Enterprise your team uses since Atlantis can work with all of them.

If that’s up your alley, you can have the “pull request-based” flow with some of the benefits of a managed solution, like history, access to policies with Sentinel, stopping runs, secret storage, etc. At this point, any generic CI tool would likely do the trick, so there may not be a point in maintaining a self-hosted installation of Atlantis to hand over the work to Terraform Cloud.

Alternative to Terraform Cloud and Atlantis - Try Spacelift

Terraform Cloud has been one of the first players in the space, but it’s not the most feature-rich platform anymore. Atlantis is great for small projects, but missing features and scaling might cause headaches.

If you’re choosing between Atlantis and Terraform Cloud, why not give a chance to Spacelift, a modern collaborative infrastructure delivery tool with a great focus on user experience? It works with Terraform, Terragrunt, and many other IaC frameworks, supports self-hosted on-prem workers, workflow customization, drift detection, and much more.

For more differences between the tools, I encourage you to check the article Spacelift vs. Atlantis and Spacelift vs. Terraform Cloud.

Spacelift provides a more mature way of automating the whole infrastructure provisioning lifecycle. Its flexible and robust workflow allows teams to get up to speed quickly and collaborate efficiently. Spacelift is highly extensible and will enable teams to enhance the Terraform workflow with custom providers, linters, security tools, and any other custom tooling they see fit.

Spacelift connects directly to the version control system of your choice and provides a truly GitOps native approach. It can support setups with multiple repositories or massive monorepos and leverages the APIs of the VCS provider to give you visibility.

Spacelift has a built-in CI/CD functionality for developing custom modules allowing teams to incorporate testing, checks, and linting early into the development phase of modules. Another benefit of using Spacelift is its flexible workflow management. It provides a policy-based process to handle dependencies between projects and deployments with Trigger Policies.

Spacelift provides a plethora of Policies to allow teams to define and automate rules governing the infrastructure as code. By utilizing Open Policy Agent, users can create their own custom policies and ensure the compliance of Terraform resources.

Check out the Getting Started Guide and start automating your infrastructure delivery seamlessly!

Key Points

We have looked into two infrastructure automation and delivery tools, Atlantis and Terraform Cloud. We analyzed each of them and discussed their strengths and weaknesses, along with a feature comparison. Finally, we saw how a modern collaborative infrastructure delivery tool like Spacelift could be used as an alternative.

Thank you for reading, and I hope you enjoyed this article as much as I did.

Terraform Deployments Automation and Ιnfrastructure Provisioning

Ioannis Moustakis — Sat, 03 Sep 2022 09:45:41 +0000

Terraform provides a well-defined and concise way to deploy infrastructure resources and changes. The typical workflow involves manual steps and checks that aren’t easily scalable and depend on human intervention to complete successfully.

This article will look into different approaches to automating infrastructure provisioning with Terraform and the pros and cons of each one.

The Typical Terraform Workflow

Terraform is one of the most prominent tools in the Infrastructure as Code space. Part of its success lies in the straightforward and easy-to-operate workflow it provides. If you aren’t familiar with Terraform, check the various Terraform articles on Spacelift’s blog to get an idea.

The core Terraform workflow consists of three concrete stages. First, we generate the infrastructure as code configuration files representing our environment’s desired state. Next, we check the output of the generated plan based on our manifests. After carefully reviewing the changes, we apply the plan to provision infrastructure resources.

Typically, this workflow involves some manual steps that are easily automatable. For example, using the Terraform CLI we have to run the command terraform plan to check the effect of our newly prepared configuration files. Similarly, we have to execute the command terraform apply to propagate the changes to the live environment.

If you want to have all of the important Terraform commands in one place, take a look at our Terraform Cheat Sheet.

For individual contributors and small teams, operating Terraform with the typical workflow and with manual steps to plan and apply the code might work perfectly fine. When teams get bigger, though, and we want to scale Terraform’s usage across organizations, we quickly reach bottlenecks and issues.

Automating Infrastructure Provisioning with Terraform

As Terraform’s usage across teams matures, adding some kind of deployment automation is beneficial. Let’s look into some of the approaches to running Terraform in automation.

Enhance the Terraform Workflow with Custom Tooling

Some teams continue running Terraform locally, but they add custom tooling, pre-commit hooks, and wrappers(e.g., Terragrunt) to enhance the core Terraform workflow. There are different wrapper tools to choose from that provide extra functionalities, such as keeping your configuration DRY, managing remote state, and managing different environments. Other teams prefer writing their own custom wrapper scripts to prepare Terraform working directories according to some standards.

This semi-manual approach is more native to the core Terraform workflow and allows direct access to output and running operational commands (e.g., terraform import). On the other hand, since it involves manual steps, it is error-prone and requires human intervention. One more thing to note is that usually, this approach requires privileged access to the underlying infrastructure provider as well as the Terraform state file, which might be a security risk.

One step ahead, some teams develop their own custom platform on top of Terraform manifests that allow end users to provision infrastructure resources via tweaking some configuration changes on a UI.

This approach abstracts any unnecessary level of detail and Terraform-specific knowledge from end users and allows them to manage infrastructure on demand without another team blocking them. On the flip side, this path requires substantial engineering effort to develop a useful platform and adds a maintenance overhead to the platform team.

Build infrastructure provisioning pipelines

The most common approach for running Terraform in automation is to build infrastructure provisioning pipelines with a CI/CD tool. With this method, teams can enforce best practices, add tests and checks, and integrate the Terraform workflow to any CI/CD tool they are familiar with.

Building infrastructure delivery pipelines in CI/CD tools brings several challenges as it needs to adjust the core Terraform workflow for non-interactive environments.

The first step for automating Terraform deployments is to embrace Infrastructure as Code and GitOps and store your manifests in the version control system of your choice. Having versioned repositories as the source of truth for automating infrastructure delivery is a core requirement.

Next, the typical Terraform workflow is adjusted for running in remote environments. Since the run might be triggered in ephemeral environments, we have to initialize the Terraform working directory, run any custom checks or tests and produce a planned output for changing resources.

A common tactic is integrating these steps as part of every proposed code change (e.g., Pull Requests). Once other team members review the proposed changes and find the produced plan acceptable, they can approve and merge the Pull Request. Merging new code to the branch that is considered the source of truth triggers a terraform apply to provision the latest changes.

Here are some things to consider for building your own automated Terraform delivery pipelines:

Your code should be stored in a version control system.
Leverage the -input=false flag to disable interactive prompts. The command line, environment variables, or configuration files should provide any necessary input.
Use a backend that supports remote Terraform State to allow runs on different machines and state locking for safety against race conditions.
Prepare an environment to run Terraform with any dependencies pre-installed. To avoid downloading the provider plugins every time with the init command, use the flag -plugin-dir to provide the path of preconfigured plugins on the automation system.
To allow changing the default backend configuration to deploy with different permissions or to different environments, you can utilize the -backend-config=path flag when initializing. If you only need to run checks on the Terraform files that don’t require initializing the backend (e.g., terraform validate), consider using the flag -backend=false.
Integrate Terraform formatting, validating, linting, checking policies, and custom testing to the CI/CD pipelines to ensure your code conforms to your organization’s standards.
Usually, CI/CD pipelines run on distributed systems. To ensure that we will apply the correct plan, we can output the plan to a file and package the whole terraform working directory after each plan. These artifacts will be stored somewhere to be fetched by the apply step to avoid accidentally applying different changes to the ones reviewed.
Optionally, use the flag -auto-approve to apply the changes without human intervention.
Use environment variables prefixed with TF_VAR_ to pass any necessary values using the CI/CD tool mechanisms.
Set the environment variable TF_IN_AUTOMATION to indicate that Terraform is running in automation mode. This adjusts the output of some commands to avoid outputting messages that are misleading in an automation environment.

Integrating the Terraform workflow in CI/CD infrastructure provisioning pipelines is a great way to automate infrastructure delivery. Running Terraform in CI/CD automation eliminates the need for people’s privileged access, enforces a consistent workflow and way of working, and removes any human intervention.

On the other hand, we have to take into account considerations for running on distributed systems, spend substantial engineering time and put a lot of effort into building a custom pipeline that satisfies our team’s needs.

How Spacelift Can Help You Automate Terraform Deployments

A more robust approach to automating your Terraform workflows end-to-end would be to use Spacelift, a collaborative infrastructure delivery tool. Spacelift provides a more mature way of automating the whole infrastructure provisioning lifecycle. Its flexible and robust workflow allows teams to get up to speed quickly and collaborate efficiently.

The Spacelift runners are fully customizable Docker containers. This allows teams to enhance the Terraform workflow with custom providers, linters, security tools, and any other custom tooling you might see fit.

Sometimes, the actual state of live environments drifts from the desired state, a concept known as configuration drift. Spacelift can assist you in automatically detecting and if desired, reconciling configuration drift.

Check out the Getting Started Guide and start automating your infrastructure delivery easily!

Key Points

In this blog post, we discussed different approaches and strategies to automate Terraform deployments and provision infrastructure in an automated fashion. We looked into the typical Terraform workflow and saw how we can enhance it with orchestration tools. Finally, we saw how Spacelift greatly assists us in bringing our Terraform automation to the next level.

Thank you for reading, and I hope you enjoyed this article as much as I did.

Terraform vs. Kubernetes : Key Differences and Comparison

Ioannis Moustakis — Sat, 03 Sep 2022 09:28:27 +0000

This article will compare two of the most dominant tools in the cloud infrastructure space, Terraform and Kubernetes. These two tools share some similarities but are built to serve different purposes. Terraform is a tool focused on infrastructure provisioning and operates in the Infrastructure as code space. Kubernetes focuses on running container workloads and operates in the container orchestration space.

We will briefly take a look at each one of them and discuss their similarities and differences.

To learn more about these two foundational cloud infrastructure technologies, check the multiple tutorials on Spacelift’s blog around Kubernetes and Terraform.

Terraform in a Nutshell

Terraform is an open-source software tool that allows us to safely and predictably manage infrastructure at scale using cloud-agnostic and infrastructure as code principles. It is a powerful tool developed by Hashicorp that enables infrastructure provisioning both on the cloud and on-premises.

Terraform is written in a declarative configuration language, Hashicorp Configuration Language (HCL), and facilitates the automation of infrastructure management in any environment. It allows IT professionals to collaborate and perform changes safely on cloud environments and scale them on-demand according to the business needs.

Modules provide excellent reusability and code-sharing opportunities to boost the collaboration and productivity of teams operating on the cloud. Providers are plugins that offer integration and interaction with different APIs and are one of the main ways to extend Terraform’s functionality.

Terraform keeps an internal state of the managed infrastructure, which represents resources, configuration, metadata, and their relationships. The state is actively maintained by Terraform and utilized to create plans, track changes, and enable modifications of infrastructure environments. The state should be stored remotely to allow teamwork and collaboration as a best practice.

Kubernetes in a Nutshell

Kubernetes (K8s) is an open-source system for container orchestration, automating deployments, and managing containerized apps. Its powerful orchestration system enables applications to scale seamlessly and achieve high availability. It has been designed and developed by Google, leveraging its vast experience in running and maintaining critical workloads in production.

Kubernetes strives to be cloud agnostic at its core, providing great flexibility in running workloads across cloud and on-premises environments. Additionally, it is designed with extensibility in mind, providing the option to add features and custom tooling to clusters easily.

One of its main benefits is the self-healing capabilities it provides. Containers that fail are automatically restarted and rescheduled, nodes can be configured to be automatically replaced, and traffic is served only by healthy components based on health checks.

Rollouts are handled progressively, and Kubernetes provides smart mechanisms to monitor application health during deployments. Rolling back problematic changes happens automatically if the application health doesn’t report a healthy status after a new deployment. Keeping the application running while rolling out new software versions has been a hot topic in the Kubernetes ecosystem over the past years, with many possible deployment strategies.

Kubernetes handles service discovery and load-balances traffic between similar pods natively without the need for complex external solutions. It has extendable built-in mechanisms to manage configuration and secrets for your applications. Scaling your applications has never been easier since it provides autoscaling options, scaling through commands, or via a UI.

Kubernetes provides a cluster of nodes, a group of worker machines that run containerized applications. Each node hosts pods that hold application workload containers. The brain of the whole system is the control plane. Each cluster consists of several components that manage the worker nodes and pods and guarantee operational continuity.

The API server is the component exposing the Kubernetes API and operates as the front-end of the control plane by handling all the communication between other parts. The etcd component is used to store all cluster data and state. The scheduler manages how pods are assigned to nodes and takes all the workload scheduling decisions. The controller manager components run different controller processes to ensure that the cluster’s desired state matches its current state. The cloud controller manager integrates Kubernetes clusters with external cloud providers, embeds their logic, and links the Kubernetes API with the Cloud Provider’s API. On each node, the kubelet is the agent responsible for running containers in pods, and kube-proxy is the component that adds the necessary networking capabilities for communication between pods and nodes.

Check out this article to learn more about the Key Kubernetes Cluster Components.

Kubernetes and Terraform Differences

These two modern technologies have many similarities but also fundamental differences. Let’s look into some of them in more detail.

1) Area of Focus

First and foremost, Terraform and Kubernetes have different purposes and try to solve different problems. Terraform focuses on provisioning infrastructure components and targets the Infrastructure as Code space. Kubernetes aims to enable us to run container workloads and targets the container orchestration space.

2) Configuration Language and CLI

Manifests in Terraform are written in HCL language, while in Kubernetes in YAML or JSON. Each tool has its own command line utility and tool-specific internals to understand before being productive.

3) Tool workflow

The Terraform workflow is generally considered easy to understand and provides a welcoming experience for new users. On the other hand, to be effective in running applications in Kubernetes, one has to understand a lot of cluster internal components and mechanics, and usually, it takes more time for users to get up to speed with Kubernetes.

4) Configuration Drift & Planning Phase

Terraform provides a native way to detect and inform you about configuration drift and unwanted changes by leveraging the planning phase of the typical workflow. In contrast, Kubernetes doesn’t support this functionality out of the box.

Kubernetes and Terraform Similarities

1) DevOps Tools

Both tools operate in the DevOps space and are typically set up and configured by the same type of IT practitioners; Site Reliability, DevOps, and Cloud engineers.

2) Open Source & Cloud Agnostic

Both tools are open source with various contributions by their online communities. They also take a similar approach to striving to be as cloud, platform, and API agnostic as possible to accommodate workloads across different environments. Even though they try to keep the core of the projects agnostic to external providers, both tools have mature and actively maintained integrations with the most common cloud providers.

3) Declarative Configuration

Although they use different languages, Terraform and Kubernetes take a similar approach conceptually to define the configuration. Manifests in both tools are written with the declarative approach.

4) State Management

The notion of the state exists in both tools, although it is implemented differently. Terraform and Kubernetes apply some logic to reconcile the desired state configured in declarative configuration files with the running state.

5) Extensibility

Both tools are highly extensible by leveraging external plugins, connecting to external APIs, or defining custom resources if necessary.

6) Well-Suited for Scale

Terraform and Kubernetes are battle-tested technologies that can support huge scale since they are designed and architected with scaling considerations for modern cloud-native environments.

7) CI/CD Compatibility

Since both tools offer highly and easily automatable workflows, they can be integrated and combined with CI/CD pipelines to automate their lifecycle.

Kubernetes and Terraform Synergies

By decomposing all the information we discussed, we realize that Kubernetes and Terraform complement each other since they operate at two different levels and can be utilized in parallel.

A typical model that cloud practitioners adopt is to use Terraform to provision infrastructure resources (e.g. Kubernetes clusters) and use Kubernetes to manage the containerized apps that run on top of the clusters.

Terraform’s approach simplifies and standardizes the complex task of provisioning Kubernetes clusters. Terraform, in this case, enables a unified flow for provisioning Kubernetes clusters across providers with a declarative approach that is preferred over command line utilities. This approach works great, but users must use separate flows to manage infrastructure and application resources.

Another approach is to use Terraform to manage Kubernetes-specific application components as well. This model has the advantage of adding the Terraform workflow to Kubernetes components. This way, IT operators can detect configuration drift on Kubernetes and manage infrastructure and application resources with the same workflow and configuration language.

This approach has a significant disadvantage since Terraform requires a well-defined schema for each managed resource. Thus each Kubernetes resource needs to be translated into a Terraform schema to be available. This dependency makes maintaining Kubernetes resources through Terraform cumbersome at times.

Kubernetes and Terraform with Spacelift

Spacelift supports both Terraform and Kubernetes and enables users to create stacks based on them. Leveraging Spacelift, you can build CI/CD pipelines to combine them and get the best of each tool. This way, you will use a single tool to manage your Terraform and Kubernetes resources lifecycle, allow your teams to collaborate easily, and add some necessary security controls to your workflows.

You could, for example, deploy Kubernetes clusters with Terraform stacks and then, on separate Kubernetes stacks, deploy your containerized applications to your clusters. With this approach, you can easily integrate drift detection into your Kubernetes stacks and enable your teams to manage all your stacks from a single place.

To take this one step further, you could add custom policies to harden the security and reliability of your configurations and deployments. Spacelift provides different types of policies and workflows easily customizable to fit every use case. You could, for instance, add plan policies to restrict or warn about security or compliance violations or approval policies to add an approval step during deployments. The possibilities are endless with Spacelift since it provides a great way to blend Terraform and Kubernetes and enhance their capabilities with extra functionality.

Take a look at the Getting Started Guide to liftoff with Spacelift!

Key Points

We delved into two of the most used modern DevOps tools, Kubernetes and Terraform. We discovered what makes each of them appealing and what functionalities they provide to IT operators and developers. We discussed their similarities, differences, and synergies and explored ways to combine them with Spacelift.

Thank you all for reading, and I hope you enjoyed this as much as I did.

Ansible Modules – How To Use Them Efficiently (Examples)

Ioannis Moustakis — Fri, 29 Jul 2022 17:16:00 +0000

This article delves into Ansible modules, one of the core building blocks of Ansible. In this blog post, we will examine the purpose and usage of modules, along with information on how to build them and best practices.

If you are interested in other Ansible concepts, these Ansible tutorials posted on Spacelift’s blog might be helpful for you.

What Are Ansible Modules?
Modules represent distinct units of code, each one with specific functionality. Basically, they are standalone scripts written for a particular job and are used in tasks as their main functional layer.

We build Ansible modules to abstract complexity and provide end-users with an easier way to execute their automation tasks without needing all the details. This way, some of the cognitive load of more complex tasks is abstracted away from Ansible users by leveraging the appropriate modules.

Here’s an example of a task using the apt package manager module to install a specific version of Nginx.

- name: "Install Nginx to version {{ nginx_version }} with apt module"
   ansible.builtin.apt:
     name: "nginx={{ nginx_version }}"
     state: present

Modules can be executed as well directly from the command line. Here’s an example of running the ping module against all the database hosts from the command line.

ansible databases -m ping

Working With Ansible Modules

A well-designed module provides a predictable and well-defined interface that accepts arguments that make sense and are consistent with other modules. Modules take some arguments as input and return values in JSON format after execution.

Ansible modules should follow idempotency principles, which means that consecutive runs of the same module should have the same effect if nothing else changes. Well-designed modules detect if the current and desired state match and avoid making changes if that’s the case.

We can utilize handlers to control the flow execution of modules and tasks in a playbook. Modules can trigger additional downstream modules and tasks by notifying specific handlers.

As mentioned, modules return data structures in JSON data. We can store these return values in variables and use them in other tasks or display them to the console. Look at the common return values for all modules to get an idea.

For custom modules, the return values should be documented along with other useful information for the module. The command-line tool ansible-doc displays this information.

Here’s an example output of running the ansible-doc command.

ansible-doc apt

In the latest versions of Ansible, most modules are part of collections, a distribution format that includes roles, modules, plugins, and playbooks. Many of the core modules we use extensively are part of the Ansible.Builtin collection. To find other available modules have a look at the Collection docs.

12 Useful & Common Ansible Modules

In this part, we explore some of the most used and helpful modules, and for each, we provide a working example. The modules in this list are picked based on their popularity within the Ansible community and functionality to perform everyday automation tasks.

Package Manager Modules yum & apt

The apt module is part of ansible-core and manages apt packages for Debian/Ubuntu Linux distributions. Here’s an example that updates the repository cache and updates the Nginx package to the latest version:

- name: Update the repository cache and update package "nginx" to latest version
  ansible.builtin.apt:
    name: nginx
    state: latest
    update_cache: yes

The yum module is also part of ansible-core and manages packages with yum for RHEL/Centos/Fedora Linux distributions. Here’s the same example as above with the yum module:

- name: Update the repository cache and update package "nginx" to latest version
   ansible.builtin.yum:
     name: nginx
     state: latest
     update_cache: yes

Service Module

The service module controls services on remote hosts and can leverage different init systems depending on their availability in a system. This module provides a nice abstraction layer for underlying service manager modules. Here’s an example of restarting the docker service:

- name: Restart docker service
   ansible.builtin.service:
     name: docker
     state: restarted

File Module

The file module handles operations to files, symlinks, and directories. Here’s an example of using this module to create a directory with specific permissions:

- name: Create the directory "/etc/test" if it doesnt exist and set permissions
  ansible.builtin.file:
    path: /etc/test
    state: directory
    mode: '0750'

Copy Module

The copy module copies files to the remote machine and handles file transfers or moves within a remote system. Here’s an example of copying a file to the remote machine with permissions set:

- name: Copy file with owner and permissions
  ansible.builtin.copy:
    src: /example_directory/test
    dest: /target_directory/test
    owner: joe
    group: admins
    mode: '0755'

Template Module

The template module assists us to template files out to target hosts by leveraging the Jinja2 templating language. Here’s an example of using a template file and some set Ansible variables to generate an Nginx configuration file:

- name: Copy and template the Nginx configuration file to the host
  ansible.builtin.template:
    src: templates/nginx.conf.j2
    dest: /etc/nginx/sites-available/default

Lineinfile & Blockinfile Modules

The lineinfile module adds, replaces, or ensures that a particular line exists in a file. It’s pretty common to use this module when we need to update a single line in configuration files.

- name: Add a line to a file if it doesnt exist
  ansible.builtin.lineinfile:
    path: /tmp/example_file
    line: "This line must exist in the file"
    state: present

The blockinfile module inserts, updates, or removes a block of lines from a file. It has the same functionality as the previous module but is used when you want to manipulate multi-line text blocks.

- name: Add a block of config options at the end of the file if it doesn’t exist
  ansible.builtin.blockinfile:
    path: /etc/example_dictory/example.conf
    block: |
      feature1_enabled: true
      feature2_enabled: false
      feature2_enabled: true
    insertafter: EOF

Cron Module

The cron module manages crontab entries and environment variables entries on remote hosts.

- name: Run daily DB backup script at 00:00
  ansible.builtin.cron:
    name: "Run daily DB backup script at 00:00"
    minute: "0"
    hour: "0"
    job: "/usr/local/bin/db_backup_script.sh > /var/log/db_backup_script.sh.log 2>&1"

Wait_for Module

The wait_for module provides a way to stop the execution of plays and wait for conditions, amount of time to pass, ports to become open, processes to finish, files to be available, strings to exist in files, etc.

- name: Wait until a string is in the file before continuing
  ansible.builtin.wait_for:
    path: /tmp/example_file
    search_regex: "String exists, continue"

Command & Shell Modules

The command and shell modules execute commands on remote nodes. Their main difference is that the command module bypasses the local shell, and consequently, variables like $HOSTNAME or $HOME aren’t available, and operations like “<”, “&” don’t work. If you need these features, you have to use the shell module.

On the other hand, the remote local environment won’t affect the command module, so its outcome is considered more predictable and secure.

Usually, it’s always preferred to use specialized Ansible modules to perform tasks instead of command and shell. There are cases, though, where you won’t be able to get the functionality that you need from specialized modules, and you will have to use one of these two. Use command and shell with care, and always try to check if there is a specialized module that can serve you better before relying on them.

- name: Execute a script in remote shell and capture the output to file
  ansible.builtin.shell: script.sh >> script_output.log

Building Ansible Modules

For advanced users, there is always the option to develop custom modules if they have needs that can’t be satisfied by existing modules. Since modules always should return JSON data, they can be written in any programming language.

Before jumping into module development, ensure that a similar module doesn’t exist to avoid unnecessary work. Αdditionally, you might be able to combine different modules to achieve the functionality you need. In this case, you might be able to replicate the behavior you want by creating a role that leverages other modules. Another option is to use plugins to enhance Ansible’s basic functionality with logic and new features accessible to all modules.

Next, we will go through an example of creating a custom module that takes as input a string that represents an epoch timestamp and converts it to its human-readable equivalent of type datetime in Python. You can find the code for this tutorial on this repository.

First, let’s create a library directory on the top of our repository to place our custom module. Playbooks with a ./library directory relative to their YAML file can add custom ansible modules that can be recognized in the ansible module path. This way, we can group custom modules and their related playbooks.

We create our custom Python module epoch_converter.py inside the library directory. This simple module takes as input the argument epoch_timestamp and converts it to datetime type. We use another argument, state_changed, to simulate a change in the target system by this module.

library/epoch_converter.py

#!/usr/bin/python

from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
import datetime

DOCUMENTATION = r'''
---
module: epoch_converter

short_description: This module converts an epoch timestamp to human-readable date.

# If this is part of a collection, you need to use semantic versioning,
# i.e. the version is of the form "2.5.0" and not "2.4".
version_added: "1.0.0"

description: This module takes a string that represents a Unix epoch timestamp and displays its human-readable date equivalent.

options:
   epoch_timestamp:
       description: This is the string that represents a Unix epoch timestamp.
       required: true
       type: str
   state_changed:
       description: This string simulates a modification of the target's state.
       required: false
       type: bool

author:
   - Ioannis Moustakis (@Imoustak)
'''

EXAMPLES = r'''
# Convert an epoch timestamp
- name: Convert an epoch timestamp
 epoch_converter:
   epoch_timestamp: 1657382362
'''

RETURN = r'''
# These are examples of possible return values, and in general should use other names for return values.
human_readable_date:
   description: The human-readable equivalent of the epoch timestamp input.
   type: str
   returned: always
   sample: '2022-07-09T17:59:22'
original_timestamp:
   description: The original epoch timestamp input.
   type: str
   returned: always
   sample: '16573823622'

'''

from ansible.module_utils.basic import AnsibleModule


def run_module():
   # define available arguments/parameters a user can pass to the module
   module_args = dict(
       epoch_timestamp=dict(type='str', required=True),
       state_changed=dict(type='bool', required=False)
   )

   # seed the result dict in the object
   # we primarily care about changed and state
   # changed is if this module effectively modified the target
   # state will include any data that you want your module to pass back
   # for consumption, for example, in a subsequent task
   result = dict(
       changed=False,
       human_readable_date='',
       original_timestamp=''
   )

   # the AnsibleModule object will be our abstraction working with Ansible
   # this includes instantiation, a couple of common attr would be the
   # args/params passed to the execution, as well as if the module
   # supports check mode
   module = AnsibleModule(
       argument_spec=module_args,
       supports_check_mode=True
   )

   # if the user is working with this module in only check mode we do not
   # want to make any changes to the environment, just return the current
   # state with no modifications
   if module.check_mode:
       module.exit_json(**result)

   # manipulate or modify the state as needed (this is going to be the
   # part where your module will do what it needs to do)
   result['original_timestamp'] = module.params['epoch_timestamp']
   result['human_readable_date'] = datetime.datetime.fromtimestamp(int(module.params['epoch_timestamp']))

   # use whatever logic you need to determine whether or not this module
   # made any modifications to your target
   if module.params['state_changed']:
       result['changed'] = True

   # during the execution of the module, if there is an exception or a
   # conditional state that effectively causes a failure, run
   # AnsibleModule.fail_json() to pass in the message and the result
   if module.params['epoch_timestamp'] == 'fail':
       module.fail_json(msg='You requested this to fail', **result)

   # in the event of a successful module execution, you will want to
   # simple AnsibleModule.exit_json(), passing the key/value results
   module.exit_json(**result)


def main():
   run_module()


if __name__ == '__main__':
   main()

To test our module, let’s create a test_custom_module.ymlplaybook in the same directory as our library directory.

test_custom_module.yml

- name: Test my new module
  hosts: localhost
  tasks:
  - name: Run the new module
    epoch_converter:
      epoch_timestamp: '1657382362'
      state_changed: yes
    register: show_output
  - name: Show Output
    debug:
      msg: '{{ show_output }}'

Last stop, let’s execute the playbook to test our custom module. Since we opted to set the state_changed argument, we expect the task state to appear as changed and displayed in yellow.

If you wish to contribute to an existing Ansible collection or create and publish a new one with your custom modules, look at Distributing collections and Ansible Community Guide, where you can find information on how to configure and distribute Ansible content.

Ansible Modules Best Practices
Use specialized modules over shell or command: Although It might be tempting to use the shell or command module often, it’s considered a best practice to leverage more specific modules for each job. Specialized modules are typically recommended because they implement the concept of desired state and idempotency, have been tested, and fulfill basic standards, like error handling.

Specify arguments when it makes sense: Some module arguments have default values that can be omitted. To be more transparent and explicit, we can opt to specify some of these arguments like the state in our playbook definitions.

Prefer multi-tasks in a module over loops: The most efficient way of defining a list of similar tasks, like installing packages, is to use multiple tasks in a single module.

- name: Install Docker dependencies
  ansible.builtin.apt:
     name:
       - curl
       - ca-certificates
       - gnupg2
       - lsb-release
     state: latest

The above method should be preferred over the loop or defining multiple separate tasks using the same module.

Custom modules should be simple and tackle a specific job: If you decide to build your own module, focus on solving a particular problem. Each module should have a concise functionality, be as simple as possible, and perform one thing well. If what you try to achieve goes beyond the scope of a single module, consider developing a new collection.

Custom modules should have predictable parameters: Try to enable others to use your module by defining a transparent and predictable user interface. The arguments should be well-scoped and understandable, and their structures should be as simple as possible. Follow the typical convention of parameter names in lowercase and use underscores as the word separator.

Document and test your custom modules: Every custom module should include examples, explicitly document dependencies, and describe return responses. New modules should be tested thoroughly before releasing. You can create roles and playbooks to test your custom modules and different test cases.

Key Points

We deep-dived into Ansible modules and examined their use and functionality in detail. We discussed best practices and showed practical examples of leveraging the most commonly-used modules. Lastly, we went through a complete example of developing a custom module.

Thank you for reading, and I hope you enjoyed this article as much as I did.

9 DevOps Best Practices – What You Should Do and NOT Do

Ioannis Moustakis — Sun, 26 Jun 2022 14:58:07 +0000

Utilizing DevOps practices to maximize speed and value creation has been a hot topic in the software industry for the past decade. We have embraced these practices and changed how we work and think about development, operations, project management, code quality, observability, and continuous feedback.

As organizations started applying these practices, we noticed many anti-patterns emerging. In this article, we will see some DevOps best practices and ways to improve our workflows, but also we will explore some of the typical DevOps anti-patterns and how to avoid them.

What is DevOps, and Why it's Important?

Probably you have seen hundreds of different definitions of DevOps by now. For me, DevOps is a set of best practices around the software development lifecycle and the effort to continually improve and deliver value more efficiently.

On its basis, DevOps is a culture that spreads equally between developers and operations rather than a specific role. In reality, the term has been used as an umbrella term to characterize the engineering roles of cloud-savvy people who share the pains and responsibilities of devs and ops. At the same time, they strive to enable and promote DevOps practices within organizations.

So why all this fuss? Implementing these practices has proven to improve software quality. Different software and operations teams collaborate more efficiently, reduce friction and lead time, integrate and test their code continuously, and deploy more often.

It’s all about finding the inefficiencies in our workflows and building a culture of continuous communication and trust. Other aspects of it are handling failures and unplanned work, leveraging automation, and focusing significantly on observability to get meaningful feedback.

DevOps Best Practices to Follow

Now that we set the foundation let’s explore some DevOps best practices without further ado. The list isn’t supposed to be exhaustive but rather a guide with hints and pointers that will ease your journey towards adopting a healthy DevOps culture.

1. Foster a Culture of Collaboration and Blameless Communication

First and foremost, for this journey to be successful, we must focus intensely on growing a culture that allows people to collaborate freely and remove the fear of failure. Organizations and teams that promote values like trust and empathy tend to have a heads-up advantage in adopting DevOps practices. Break down the silos between teams and make them work together towards a common goal, bringing value to the company.

One of the tools that deliver an enhanced collaboration layer for IaC is Spacelift. At Spacelift, you can invite security and compliance teams to collaborate on and approve workflows and policies.

2. Adopt Continuous Integration and Delivery (CI/CD)

Integrating small batches of code frequently into a central code repository is a practice that allows developers to collaborate efficiently. With this approach, the repository is always kept in a good state since we introduce small changes that are easier to handle. Continuous Integration (CI) enables early error detection and improves code quality since these small batches of changes are validated each time with automated builds and tests.

The next step after integrating our code is deploying it to our environment. Continuous Delivery (CD) is the practice of getting the code into a deployable state continuously for every small batch of change. This simplifies our deployments and provides our developers with an easy and automated method to push code to production.

3. Set up Automated Testing

A continuation of the previous point and an integral part of DevOps success is setting up and curating meaningful automated tests as part of our CI/CD pipelines. This way, we don’t rely on humans to run manual tests on our code; instead, we set up automated tests that run on every minor change introduced.

By increasing the testing frequency and the number of tests, we reduce our chances of introducing bugs to production systems. The tests vary depending on the use case but typically could include unit testing, integration testing, end-to-end testing, load testing, smoke testing, etc.

4. Focus on Observability and Find the Right Metrics

DevOps practices are based on getting feedback and continuously improving our processes. We need to find and track the right metrics to achieve that and measure our results. Figuring out the right metrics is an arduous journey that each organization has to go through.

These metrics will differ from organization to organization and from team to team, depending on the goals and key results targeted. Still, it’s a crucial exercise for achieving success. Some typical examples of DevOps metrics are deployment time, frequency of deploys, deployment failure rate, availability of critical services, mean time to detect, mean time to restore, cost per unit, code coverage, and change lead time.

Moving one step forward, we also have to focus on the observability of our apps and software running in production. We have to define a strategy for effectively storing, managing, and distributing logs, traces, and metrics of our applications to quickly solve issues, improve the understandability of our systems and enable our teams to operate efficiently.

5. Avoid Manual Work with Automation

By reducing manual work and automating recurring tasks, we accelerate our processes and provide increased consistency to our results. Automation can allow us to focus on what is essential and avoid human intervention and time spent on chores. It also provides more confidence in our systems and processes, removes human errors and miscommunication, and speeds up the performance of teams. If you need an automation layer for your cloud resources, take a look at Spacelift’s self-service infrastructure (automated workflow management feature especially).

6. Incorporate Security Early in the Development Lifecycle

Security shouldn’t be one of the last things to integrate into software development. The birth of DevSecOps emphasizes thinking about application and infrastructure security early in the development lifecycle, incorporating security into the initial design and integrating it into the CI/CD pipelines.

Security should be a responsibility shared among different teams and through the entire application lifecycle and be considered an integral part of the process, not an optional add-on. Recently, a strong focus has been given to securing the software supply chain due to increased malicious attacks over the last years.

In the world of infrastructure, even the tiniest of mistakes can cause major outages. That’s why Spacelift adds an extra layer of policy that allows you to control – separately from your infrastructure project – what code can be executed, what changes can be made, when and by whom. This isn’t only useful to protect yourself from the baddies but allows you to implement an automated code review pipeline.

7. Learn from Incidents and Build Processes around Them

Incidents are inevitable in the IT world. It doesn’t matter how well prepared your team is; eventually, there will be an incident that you will have to address. In these cases, it’s essential to focus on blameless communication, understand the issue, communicate effectively with the affected parties, and collaborate to find a solution.

Equally crucial to fixing the issue is to have a process to log incidents and learn from them. After the incident has been tackled, spend some time with your team to craft a post-incident review, and discuss how the incident was handled. Try to find any possible improvements in the incident handling process that can help you next time.

8. Focus first on Concepts, then Find the Right Tools

The DevOps landscape is moving extremely fast, with new tools and services emerging daily. Instead of continuously integrating new shiny tools and services, concentrate on understanding the core concepts that allow companies to accelerate their business with DevOps practices.

Only once you understand the concepts and prioritize the missing pieces accordingly, you will be successful in selecting the right tools for the job. Remember, you won’t be able to build everything within your team. It’s ok to rely on DevOps tooling and managed services when it makes sense. Be smart about how you use your team’s time, try to understand your in-house expertise and needs, put effort into building custom tooling when it makes sense, and rely on external tooling and services for the rest.

9. Embrace Infrastructure as Code (IaC) and Push for a Self-Service Infra Model

Cloud infrastructure should be considered an integral part of software development and treated equally to application code. By leveraging infrastructure as code, we can incorporate the best practices we use for software development, such as version control and CI/CD, into infrastructure creation. This model removes the need to manually set up and configure resources via UIs and further strengthens our automation efforts across the IT landscape. Changes are always auditable and transparent, and we can quickly roll back infrastructure systems to a previous state when there are issues.

Thinking one step ahead, instead of adding another bottleneck of waiting for cloud infrastructure engineers to create the necessary resources, push for a self-service infrastructure model. In this model, the developers and anyone who needs infra resources can leverage some tooling to generate the required pieces. This way, we increase productivity and speed while giving autonomy to our developers, all via a single workflow.

Check out how Spacelift can improve your IaC, assist your team in adopting a collaborative infrastructure model and achieving operational excellence.

DevOps Anti-Patterns

Along with the rise of DevOps, we have also seen many anti-patterns emerge. During the quest to adopt DevOps practices, people have misinterpreted their scope and made mistakes that lead to common anti-patterns. Let’s look at some common challenges, traps, and misconceptions companies face while implementing DevOps principles.

1. Don’t Create Separate DevOps Team

Common mistake companies make when adopting DevOps practices is creating a separate team to handle the DevOps transformation. Unfortunately, this adds one more silo to the process and breaks the central promise of DevOps, which is increased collaboration and shared ownership between the existing teams.

Similarly, we see operations teams rebranding into DevOps teams without actual changes in organizations’ culture, communication, and collaboration. DevOps is about bringing the different groups closer, not creating a new one.

2. Avoid Having the DevOps Hero

At times, specific team members are more involved in the DevOps practices than others. That could be due to accumulated knowledge, a higher level of experience, or increased effort by a person. When this pattern emerges, it could lead quickly to the DevOps hero anti-pattern where a specific team member becomes indispensable to the team.

This situation is highly problematic since the team’s performance and velocity depend on a single person. At the same time, this person may face an extremely high amount of work that eventually leads to burnout and potentially leaving the company. To avoid this anti-pattern, ensure the knowledge is spread across teams and team members. Divide the work equally and don’t rely on heroes but on teamwork and rigid processes to achieve results.

3. Don’t Attempt to Automate and Change Everything at Once

Starting from scratch to apply DevOps practices in an organization could be daunting at first. As with most things, attempting to tackle everything at once is not the way to go. First, analyze the current situation and processes within your company. People usually don’t happily accept many changes, so you need to think strategically. Prioritize the tasks accordingly, find quick wins, automate the stuff that will have a higher impact, and focus on one thing at a time.

4. Avoid Chasing New Tools

As new services and tools pop up almost every day, adopting and using these new shiny toys is always tempting. It’s common for engineers to fall into this trap of introducing a new tool just because it’s trending without proper analysis of whether it’s needed or the best choice.

Picking the right tools for the job is critical but is a process that should be reviewed meticulously. For every new service or tool we add, we should also consider its maintainability and the operational overhead, dependencies, complexity, and new cognitive load that we introduce in the process.

5. Don’t Sacrifice Quality for Speed

Since one of the main factors of DevOps success is velocity, many teams try to speed up their processes at the cost of quality and usually security. Many of the typical DevOps metrics are based on how fast we deliver, deploy and provide value, but they are not enough by themselves as they only tell half the story. Due to this disproportionate focus on speed, it’s easy to lose perspective of what is important; delivering quality software. Treat speed and quality equally, add meaningful automated tests and avoid cutting corners just to ship faster.

6. Don’t Give up on Continuous Improvement

Applying effective DevOps practices is a dynamic process that should be curated continuously. It might be tempting to rest and relax after implementing all the DevOps best practices in the roadmap, but unfortunately, this process never stops.

Every step of the way, we should focus on reviewing our workflows and continuously improving our systems, processes, and products. We have to set up flows of constant feedback that allow us to review and reflect on our choices and ultimately improve. New paradigms, best practices, and improved models always appear, and we should be restless if we want our teams to survive, perform, and succeed.

7. Don’t Neglect Documentation and Information Sharing

By definition, successful adoption of DevOps practices relies on sharing information efficiently within an organization and creating a workplace where collaboration thrives organically. Unfortunately, neglecting documentation and efficient information sharing is an anti-pattern that occurs too often in software teams. Documentation, when done right, could be a handy tool for developers.

Try to integrate documentation tasks on team backlogs and treat docs as first-class citizens within your organization. Docs aren’t static and should be kept up to date, created consistently, and accessible to anyone who needs them.

Key Points

We have explored different DevOps best practices and paradigms and analyzed how we can incorporate them to accelerate team performance and value creation. We also saw some hidden traps and anti-patterns to be aware of and avoid while pursuing DevOps excellence.

Thank you for reading, and I hope you enjoyed this article as much as I did.

Originally published at spacelift.io.

AWS IAM Policies : Best Practices & How to Create an IAM Policy

Ioannis Moustakis — Sat, 18 Jun 2022 13:43:50 +0000

Securely accessing cloud resources is one of the most critical requirements for IT teams. To achieve this, cloud administrators leverage web services that assist with managing access to cloud environments.

In this article, we will explore the AWS IAM service, and more specifically, we will look into IAM policies, learn how they are structured, how to create them, and some best practices.

What is AWS IAM?

AWS Identity and Access Management(IAM) is a web service that assists with managing and controlling access and permissions to resources and other AWS services. Leveraging this service, we can set up the building blocks to control authentication and authorization to our AWS accounts.

Before we continue talking about IAM, let’s define some terminology for its essential components that we will reference throughout the article.

Authentication: The process of validating a user’s identity. Basically, verifying who someone claims to be.
Authorization: Defining the level of permissions and access rights for a given user. Authorization happens after authentication and establishes what someone is allowed to do.
Principals: Persons or applications that try to perform actions on AWS. Users: Identities in IAM that correspond to users in an organization. They represent actual persons or applications and service accounts.
Groups: IAM users are grouped logically with IAM groups. This way, we can assign permissions to multiple users in a bulk fashion. A common pattern is to create groups according to different roles in an organization.
Roles: Roles provide temporary access to resources and aren’t associated with specific users. Instead, roles enable principals to temporarily assume a set of permissions to complete an operation.
Policies: To manage access on AWS we generate IAM policies that define levels of permissions and attach them to IAM identities(users, groups, roles) or AWS resources.
Requests: Principals attempt to perform actions on AWS by instantiating requests to AWS via the AWS API, Management Console, or CLI.
Actions: Every request has an action definition that declares the specific operation requested. If authentication and authorization are cleared the action is approved.
Resources: A resource in this context can be considered any AWS object within a service.

Policy Types

IAM Policies are one of the most basic blocks of access management in AWS since they define the permissions of an identity or a resource. For every request, these policies are evaluated, and based on their definition; the requests are allowed or denied. Let’s look at the different types of policies that exist in AWS.

Identity-based policies are policies attached to identities(users, groups, roles) and provide them with the required permissions. Identity-based policies could be either Managed policies or Inline policies. Managed policies are either prepared and managed by AWS for common use cases(AWS managed policies) or custom policies created by users(Customer managed policies) suitable for achieving fine-grained control. Inline policies are used when we need to make a policy part of a principal’s entity and maintain a strict one-to-one relationship between them.
Resource-based policies are attached directly to resources and specify permissions for specific actions on the resource by some principals.
IAM permissions boundaries define the maximum permissions for an IAM entity and are used as safeguards.
Access control lists(ACLs) are attached to resources and control cross-account permissions for principals from other accounts.
Organizations Service Control Policies(SCPs) specify the maximum level of permissions for an organization’s accounts. These policies are used to limit the permissions that can be assigned within member accounts.
Session policies are advanced policies used during temporary sessions for roles or federated users.

Policy Document Structure

Now that we have seen the different types of IAM policies that exist in the context of AWS, let’s see how they are structured. Most of them are stored in JSON format and are attached to resources or identities. The only type of policy that uses a different format is ACL, but we won’t focus on ACLs in this article.

Each JSON policy document might have optional informational elements at the top of the document and must have one or more statements. A statement contains all the necessary information about a permission that we would like to declare.

Here’s a simple identity-based policy that allows the principal who has it attached to list the objects of a single S3 bucket named this-is-an-example-s3-bucket-name.

{
   "Version": "2012-10-17",
   "Statement": {
       "Sid": "ListObjectsInBucket",
       "Effect": "Allow",
       "Action": "s3:ListBucket",
       "Resource": "arn:aws:s3:::this-is-an-example-s3-bucket-name"
   }
}

IAM Policies are built using a combination of the below elements:

Version: Defines the version of the policy language. Always use the latest version.
Statement: This argument is used as a parent element for the different statements in the policy.
Sid: This is an optional element that allows us to define a statement ID.
Effect: This element can have the values Allow or Deny.
Action: The list of actions related to the policy.
Resource: Defines the list of resources to which the policy is applied. For resource-based policies, this is optional since the policy applies to the resource that has it attached.
Principal: Defines the identities that are allowed or denied access to resource-based policies.
Condition: Defines some conditions under which the policy applies. This element is practical when we need to achieve custom rules for fine-grained access.

There are options for resources, principals, and actions to define deny access controls by using NotPrincipal, NotAction, or NotResource. Note that these are mutually exclusive with their contrasting elements. For example, you can’t use both Resource and NotResource elements in a statement.

If you want to learn more about these elements and the JSON policy components, check out the IAM JSON policy reference.

Next, we will see an example of a more complex IAM policy with more than one statement using conditions.

{
   "Version": "2012-10-17",
   "Statement": [
       {
           "Sid": "DenyAllOutsideRequestedRegions",
           "Effect": "Deny",
           "NotAction": [
               "cloudfront:*",
               "iam:*",
               "route53:*",
               "support:*"
           ],
           "Resource": "*",
           "Condition": {
               "StringNotEquals": {
                   "aws:RequestedRegion": [
                       "eu-west-1",
                       "eu-central-1"
                   ]
               }
           }
       },
       {
           "Sid": "DenyAccessFromNonCorporateIPs",
           "Effect": "Deny",
           "Action": "*",
           "Resource": "*",
           "Condition": {
               "NotIpAddress": {
                   "aws:SourceIp": [
                       "192.0.1.0/24"
                   ]
               },
               "Bool": {"aws:ViaAWSService": "false"}
           }
       }
   ]
}

In the policy above, we define two separate statements. The first defines some rules to deny access based on the requested AWS region. More specifically, it denies all actions for regions not defined in the condition, except for the actions mentioned in the NotAction element.

The second one defines a policy to deny access to AWS based on the source IP of the requests. Here, since we have two conditions, they are combined with a logical AND. More specifically, the policy denies all AWS actions when requests originate from an IP that doesn’t come from the corporate IP range(in this example: 192.0.1.0/24) and when an AWS service isn’t making the call.

Note that both policies don’t actually allow any actions and are used to restrict access to AWS resources.

Finally, here’s a resource-based policy for an S3 bucket.

{
   "Version": "2012-10-17",
   "Statement": [
     {
       "Sid": "DenyS3AccessWithNoMFA",
       "Effect": "Deny",
       "Principal": "*",
       "Action": "s3:*",
       "Resource": "arn:aws:s3:::this-is-an-example-s3-bucket-name/example-directory/*",
       "Condition": { "Null": { "aws:MultiFactorAuthAge": true }}
     }
   ]
}

By associating this policy with the bucket this_is_an_example_s3_bucket_name every action on the bucket is denied if the request isn’t authenticated with a multi-factor authentication mechanism. Note the use of the Principal element here since we are using a resource-based policy.

Creating an IAM Policy

There are three main ways to create an IAM policy. We can either use the AWS console, the AWS CLI, or the AWS API. For this demo, we will go through the exercise of creating an IAM customer-managed policy via the AWS Console.

The easiest option to create a policy is to use the helpful visual editor that the AWS console provides without having to write a JSON file and care about proper syntax. After signing in to the AWS Management Console, head to IAM and select Policies and Create Policy.

From this screen, you can choose to either use the Visual editor or JSON.

Let’s replicate our first example policy from above that allows listing the objects in an S3 bucket. For Service we select S3, for Actions choose ListBucket, and for Resources use the arn of the S3 bucket arn:aws:s3:::this-is-an-example-s3-bucket-name. Then, select Next: Tags.

Note that by default, the policy allows the actions chosen. To deny actions, you have to select Switch to deny permissions before selecting the actions.

You can optionally add some tags to your customer-managed identity-based policy on the next page. For example, we could add tags for name, environment, department, etc. Then, select Next: Review.

Finally, add a Name to your policy and optionally a description for its purpose and select Create policy.

Nice, we have replicated the IAM policy we introduced in the first example.

If you feel comfortable with JSON syntax, you can instead use the JSON tab to define an IAM Policy like this:

Another option using the AWS API under the hood would be to create your IAM resources with the Infrastructure as Code tool of your choice. Here’s an example of using Terraform to define the same policy.

resource "aws_iam_policy" "list_bucket_policy_example" {

 name        = "list_bucket_policy_example"
 path        = "/"
 description = "AWS IAM Policy example for listing the objects of a bucket"
 policy      = <<EOF
{
 "Version": "2012-10-17",
 "Statement": [
   {
     "Action": "s3:ListBucket",
     "Resource": "arn:aws:s3:::this-is-an-example-s3-bucket-name",
     "Effect": "Allow",
     "Sid": "ListObjectsInBucket"
   }
 ]
}
EOF
}

Validating IAM Policies

When creating or editing IAM policies using the AWS Management Console, their syntax and grammar are inspected to verify they comply with the IAM policy grammar. If there is any issue or error, you get notified accordingly and have to fix the policy.

AWS provides IAM Access Analyzer with additional policy checks and recommendations to improve policies. When creating or editing a policy using the JSON tab, a policy validation pane below the policy provides different findings for the policy. There you get information for issues categorized as Security, Errors, Warnings, and Suggestions. Update your policy accordingly to resolve the findings.

Here’s an example of a malformed policy that triggers three separate findings.

We get an Error finding because we don’t specify a correct ARN field for the Resource element.

Next, on the Warnings tab, we get notified that we omitted the top-level Version element.

Finally, on the Suggestions tab, we get notified that our Action element includes no actions.

Testing IAM Policies

Another handy tool for AWS administrators and users who manage IAM policies is the IAM Policy Simulator. Using this simulator, you can troubleshoot issues with different policy types and try to understand why some requests are allowed or denied.

The IAM Policy Simulator console provides a testing playground for IAM policies and an easy way to test which actions are allowed or denied to specific principals for specific resources. The simulator doesn’t actually make the requests, so it’s a safe space to experiment and get information. You can even test new policies that don’t yet exist on our account and simulate real-world scenarios by defining complex conditions. Finally, since the returned output is a message with the result and relevant information, you can identify which statement in a policy denies or allows access.

For example, we have created an IAM user test-user and attached the policy ListBucketContents that we created earlier. We can use the AWS Policy Simulator to validate that this user can indeed list the objects of the example bucket.

After defining all the parameters for the simulation, we choose Run Simulation and check that the Permission result is allowed. Check out the detailed Troubleshooting IAM Policies guide for issues with IAM policies.

Versioning IAM Policies

A valuable feature of IAM Policies that you should be aware of is that AWS automatically keeps five versions of each policy. Whenever you make changes to a policy and save it, a new version is created instead of overwriting the existing one.

When the limit of five kept versions is reached, you can select which older version to remove. This feature is convenient because it allows us to quickly revert to a previous policy version in case of issues.

IAM Policies Best Practices

Follow Least Privilege Principles: When creating IAM policies grant only the necessary permissions to perform the job. Specify specific actions, resources, and principles and add custom conditions to achieve the required controls.
Generate policies based on access activity: Although it’s considered a best practice, it might be challenging to implement fine-grained policies that grant the least privilege when starting with IAM policies. For this reason, you can initially use policies that allow more permissions than they should and then refine them by generating a new policy based on the access activity of an IAM entity. This is a more pragmatic approach to achieving the least privilege for inexperienced users that makes the journey to improving security and access management smoother.
Get started with managed policies: It’s totally ok to start with AWS Managed policies as you start experimenting and learning. These policies cover common use cases and help your team set up necessary permissions to start quickly. When you feel comfortable enough with IAM, consider restricting your policies by creating customer-managed policies that follow the least privileged approach.
Assign permissions with roles and groups & avoid inline policies: It’s considered best to avoid assigning permissions directly to users or using inline policies. For easier access management and better security controls, attach policies to groups or roles.
Validate policies: Every time you create or edit policies, validate them using the AWS helper tools, as we have seen in the section Validating IAM Policies.
Use IAM Roles for EC2 instances(EC2 Instance Profiles): For apps that run on EC2 instances, attach the necessary permissions to IAM roles and specify the role as a launch parameter for the instance.
Use policy conditions: Leverage policy conditions to achieve complex policies with custom rules. By combining different conditions in policies, we can comply with specific requirements according to our organization’s standards. Check out the Policy Elements: Condition reference page for more information.
Test IAM policies with IAM Policy Simulator: Test existing and new policies quickly and without affecting any environment with the IAM Policy Simulator. Use it to understand your policies better and troubleshoot different issues.
Review IAM policies on a regular basis: Cloud environments aren’t static; they evolve over time. For this reason, IAM policies should be reviewed and updated to reflect changes. Following best practices and least privilege, it’s not something that you configure once and forget but requires continuous effort.
Tag IAM policies: Tag your IAM policies to add metadata to them the same way you tag other AWS resources.
Control access to resources with tags: If you have a proper tagging strategy for your AWS resources, you can create elaborate IAM policies that control access to them based on tags. This method becomes handy when you want to separate access to resources based on owners or teams.
Use IaC to create your IAM policies: In this tutorial, we have seen how to create IAM policies from the AWS Console, and this might be ok as a starting point, but eventually, you should treat your IAM resources as infrastructure components and apply the same Infrastructure as Code principles as any other AWS resource.

Key Points

We have explored IAM concepts in the context of AWS, and more specifically, we focused on IAM policies. We saw how to create and test policies, assign necessary permissions, and combine different IAM mechanisms to achieve efficient access management controls on AWS.

Here you can learn more about Spacelift integration with AWS, and here about the usage of policies with Spacelift.

Thank you for reading, and I hope you enjoyed this article as much as I did.

Ansible Roles: Basics & How to Combine Them With Playbooks

Ioannis Moustakis — Thu, 09 Jun 2022 19:05:54 +0000

This blog post explores the concept of Ansible roles, their structure, and how we can combine them with our playbooks. We will analyze their functionality and usage along with ways to create new roles and retrieve public shared roles from Ansible Galaxy, a public repository for Ansible resources.

If you are new to Ansible, you might also find these tutorials helpful Ansible Tutorial for Beginners, Working with Ansible Playbooks, and How to Use Different Types of Ansible Variables.

Why Roles Are Useful in Ansible

When starting with Ansible, it’s pretty common to focus on writing playbooks to automate repeating tasks quickly. As new users automate more and more tasks with playbooks and their Ansible skills mature, they reach a point where using just Ansible playbooks is limiting. Ansible Roles to the rescue!

Roles enable us to reuse and share our Ansible code efficiently. They provide a well-defined framework and structure for setting your tasks, variables, handlers, metadata, templates, and other files. This way, we can reference and call them in our playbooks with just a few lines of code while we can reuse the same roles over many projects without the need to duplicate our code.

Since we have our code grouped and structured according to the Ansible standards, it is quite straightforward to share it with others. We will see an example of how we can accomplish that later with Ansible Galaxy.

Organizing our Ansible content into roles provides us with a structure that is more manageable than just using playbooks. This might not be evident in minimal projects but as the number of playbooks grows, so does the complexity of our projects.

Lastly, placing our Ansible code into roles lets us organize our automation projects into logical groupings and follow the separation of concerns design principle. Collaboration and velocity are also improved since different users can work on separate roles in parallel without modifying the same playbooks simultaneously.

Ansible Role Structure

Let’s have a look at the standard role directory structure. For each role, we define a directory with the same name. Inside, files are grouped into subdirectories according to their function. A role must include at least one of these standard directories and can omit any that isn’t actively used.

To assist us with quickly creating a well-defined role directory structure skeleton, we can leverage the command ansible-galaxy init . The ansible-galaxy command comes bundled with Ansible, so there is no need to install extra packages.

Create a skeleton structure for a role named test_role:

This command generates this directory structure:

Ansible checks for main.yml files, possible variations, and relevant content in each subdirectory. It’s possible to include additional YAML files in some directories. For instance, you can group your tasks in separate YAML files according to some characteristic.

defaults – Includes default values for variables of the role. Here we define some sane default variables, but they have the lowest priority and are usually overridden by other methods to customize the role.
files – Contains static and custom files that the role uses to perform various tasks.
handlers – A set of handlers that are triggered by tasks of the role. meta – Includes metadata information for the role, its dependencies, the author, license, available platform, etc.
tasks – A list of tasks to be executed by the role. This part could be considered similar to the task section of a playbook.
templates – Contains Jinja2 template files used by tasks of the role.
tests – Includes configuration files related to role testing.
vars – Contains variables defined for the role. These have quite a high precedence in Ansible.

Another directory that wasn’t automatically generated by the ansible-galaxy init command but is mentioned in the official Ansible docs, and you might find helpful in some cases, is the library directory. Inside it, we define any custom modules and plugins that we have written and used by the role. Finally, we also have a preconfigured README.md file that we can fill with details and useful information about our role.

Creating Roles

A common tactic is to refactor an Ansible playbook into a role. To achieve that, we have to decompose the different parts of a playbook and stitch them together into an Ansible role using the directories we’ve just seen in the previous section.

This section will go through an example of creating a new role for installing and configuring a minimal Nginx web server from scratch. If you wish to follow along, you will need VirtualBox, Ansible, and Vagrant installed locally.

Ansible searches for referenced roles in common paths like the orchestrating playbook’s directory, the roles/ directory, or the configured roles_path configuration value. It’s also possible to set a custom path when referencing a role:

- hosts: all
  roles:
    - role: "/custom_path/to/the/role"

Using the ansible-galaxy init command, we generate the initial directory structure for a role named webserver inside a parent directory named roles. Let’s go ahead and delete the tests directory since we won’t be using it. We will see how to utilize all the other directories during our demo.

The final structure of our role looks like this:

First, let’s define the most fundamental part of our role, its tasks. Head to the tasks directory and edit the main.yml file.

roles/webserver/tasks/main.yml

---
# tasks file for webserver
- name: Update and upgrade apt
  ansible.builtin.apt:
    update_cache: yes
    cache_valid_time: 3600
    upgrade: yes

- name: "Install Nginx to version {{ nginx_version }}"
  ansible.builtin.apt:
    name: "nginx={{ nginx_version }}"
    state: present

- name: Copy the Nginx configuration file to the host
  template:
    src: templates/nginx.conf.j2
    dest: /etc/nginx/sites-available/default
- name: Create link to the new config to enable it
  file:
    dest: /etc/nginx/sites-enabled/default
    src: /etc/nginx/sites-available/default
    state: link

- name: Create Nginx directory
  ansible.builtin.file:
    path: "{{ nginx_custom_directory }}"
    state: directory

- name: Copy index.html to the Nginx directory
  copy:
    src: files/index.html
    dest: "{{ nginx_custom_directory }}/index.html"
  notify: Restart the Nginx service

Here, we define a handful of tasks that update the operating system, install an Nginx web server, and set up a minimal custom configuration for demo purposes.

Next, we move to the defaults directory, where we will set default values for the variables used in the tasks. If there is no other definition for these variables, they will be picked up and used by the role, but usually, they are meant to be easily overwritten.

roles/webserver/defaults/main.yml

---
# defaults file for webserver
nginx_version: 1.18.0-0ubuntu1.3
nginx_custom_directory: /var/www/example_domain

Moving on to our vars directory, we define values with higher precedence that aren’t meant to be overridden at the play level. Here, we override the default variable that defines the Nginx custom directory.

roles/webserver/vars/main.yml

---
# vars file for webserver
nginx_custom_directory: /home/ubuntu/nginx

In the handlers directory, we define any handler that is triggered by our tasks. One of our tasks includes a notify keyword since it needs to trigger our Restart the Nginx service handler.

roles/webserver/handlers/main.yml

---
# handlers file for webserver
- name: Restart the Nginx service
  service:
    name: nginx
    state: restarted

In the templates directory, we leverage a Jinja2 template file for the Nginx configuration that gets the Nginx custom directory value from one of our previously defined variables.

roles/webserver/templates/nginx.conf.j2

server {
        listen 80;
        listen [::]:80;
        root {{ nginx_custom_directory }};
        index index.html;
        location / {
                try_files $uri $uri/ =404;
        }
}

In the files directory, we define a static file index.html that will serve as our static demo webpage.

roles/webserver/files/index.html

<html>
 <head>
   <title>Hello from Ngnix </title>
 </head>
 <body>
 <h1>This is our test webserver</h1>
 <p>This Nginx web server was deployed by Ansible.</p>
 </body>
</html>

We use the meta directory to add metadata and information about the role. Any role dependencies by other roles go here as well.

roles/webservers/meta/main.yml

galaxy_info:
  author: Ioannis Mosutakis
  description: Installs Nginx and configures a minimal test webserver
  company: ACME Corp
  license: Apache-2.0
  role_name: websercer

  min_ansible_version: "2.1"

 # If this is a Container Enabled role, provide the minimum Ansible Container version.
 # min_ansible_container_version:

 #
 # Provide a list of supported platforms, and for each platform a list of versions.
 # If you don't wish to enumerate all versions for a particular platform, use 'all'.
 # To view available platforms and versions (or releases), visit:
 # https://galaxy.ansible.com/api/v1/platforms/
 #
  platforms:
  - name: Ubuntu
    versions:
      - bionic
      - focal

  galaxy_tags:
    - nginx
    - webserver
    - development
    - test

dependencies: []
 # List your role dependencies here, one per line. Be sure to remove the '[]' above,
 # if you add dependencies to this list.

Lastly, we update the README.md _file accordingly. The autogenerated file provided by the _ansible-galaxy init command includes many pointers and guidance on filling this nicely.

roles/webserver/README.md

Role Name
=========

Role Name
=========

This is a role created for demonstration purposes that configures a basic nginx webserver with a minimal configuration.

Requirements
------------

Any prerequisites that may not be covered by Ansible itself or the role should be mentioned here. For instance, if the role uses the EC2 module, it may be a good idea to mention in this section that the boto package is required.

* Ansible
* Jinja2

Role Variables
--------------

A description of the settable variables for this role should go here, including any variables that are in defaults/main.yml, vars/main.yml, and any variables that can/should be set via parameters to the role. Any variables that are read from other roles and/or the global scope (ie. hostvars, group vars, etc.) should be mentioned here as well.

### defaults/main.yml
Default nginx installation variables.

* nginx_version: Specific version of nginx to install
* nginx_custom_directory: Custom directory for nginx installation

### vars/main.yml
Here we define variables that have high precedence and aren't intended to be changed by the play.

* nginx_custom_directory: Custom directory for nginx installation

Dependencies
------------

A list of other roles hosted on Galaxy should go here, plus any details in regards to parameters that may need to be set for other roles, or variables that are used from other roles.

Example Playbook
----------------

Including an example of how to use your role (for instance, with variables passed in as parameters) is always nice for users too:

   - hosts: all
     become: true
     roles:
       - webserver

License
-------

Apache-2.0

Author Information
------------------

Ioannis Moustakis

Using Roles

Once we have defined all the necessary parts of our role, it’s time to use it in plays. The classic and most obvious way is to reference a role at the play level with the roles option:

- hosts: all
  become: true
  roles:
    - webserver

With this option, each role defined in our playbook is executed before any other tasks defined in the play.

This is an example play to try out our new webserver role. Let’s go ahead and execute this play. To follow along, you should first run the vagrant up command from the top directory of this repository to create our target remote host.

Sweet! All the tasks have been completed successfully. Let’s also validate that we have configured our Ngnix web server correctly.

Use the command vagrant ssh host1 to connect to our demo Vagrant host. Then execute systemctl status nginx to verify that Nginx service is up and running. Finally, run the command curl localhost to check if the web server responds with the custom page that we configured.

When using the roles option at the play level, we can override any of the default role’s variables or pass other keywords, like tags. Tags are added to all tasks within the role.

- hosts: all
 become: true
 roles:
   - role: webserver
     vars:
       nginx_version: 1.17.10-0ubuntu1
     tags: example_tag

Here, we override the default variable nginx_version with another version.

Apart from defining roles at the play level with the roles option, we can use them also at the tasks level with the options include_role dynamically and import_role statically. These are useful when we would like to run our role tasks in a specific order and not necessarily before any other playbook tasks. This way, roles run in the order they are defined in plays.

- hosts: all
  tasks:
    - name: Print a message
      ansible.builtin.debug:
        msg: "This task runs first and before the example role"

    - name: Include the example role and run its tasks
      include_role:
        name: example

    - name: Print a message
      ansible.builtin.debug:
        msg: "This task runs after the example role"

    - name: Include the example_2 role and run its tasks in the end
      include_role:
        name: example_2

Check the official docs for achieving more fine-grained control of your role’s task execution order.

Even if we define a role multiple times, Ansible will execute it only once. Occasionally, we might want to run a role multiple times but with different parameters. Passing a different set of parameters to the same roles allows executing the role more than once.

Example of executing the role test three times:

- hosts: all
  roles:
    - role: test
      message: "First time"
    - role: test
      message: "Second time"
    - role: test
      message: "Third time"

Sharing Roles with Ansible Galaxy

Ansible Galaxy is an online open-source, public repository of Ansible content. There, we can search, download and use any shared roles and leverage the power of its community. We have already used its client, ansible-galaxy, which comes bundled with Ansible and provides a framework for creating well-structured roles.

You can use Ansible Galaxy to browse for roles that fit your use case and save time by using them instead of writing everything from scratch. For each role, you can see its code repository, documentation, and even a rating from other users. Before running any role, check its code repository to ensure it’s safe and does what you expect. Here’s a blog post on How to evaluate community Ansible roles. If you are curious about Galaxy, check out its official documentation page for more details.

To download and install a role from Galaxy, use the ansible-galaxy install command. You can usually find the installation command necessary for the role on Galaxy. For example, look at this role that installs a PostgreSQL server.

Install the role with:

ansible-galaxy install geerlingguy.postgresql

Then use it in a playbook while overriding the default role variable postgresql_users to create an example user for us.

---
- hosts: all
  become: true
  roles:
    - role: geerlingguy.postgresql
      vars:
        postgresql_users:
          - name: christina

Ansible Roles Tips & Tricks

This section gathers some tips and tricks that might help you along your journey with Ansible roles.

Always use descriptive names for your roles, tasks, and variables. Document the intent and the purpose of your roles thoroughly and point out any variables that the user has to set. Set sane defaults and simplify your roles as much as possible to allow users to get onboarded quickly.
Never place secrets and sensitive data in your roles YAML files. Secret values should be passed to the role at execution time by the play as a variable and should never be stored in any code repository.
At first, it might be tempting to define a role that handles many responsibilities. For instance, we could create a role that installs multiple components, a common anti-pattern. Try to follow the separation of concerns design principle as much as possible and separate your roles based on different functionalities or technical components.
Try to keep your roles as loosely coupled as possible and avoid adding too many dependencies.
To control the execution order of roles and tasks, use the import_role or include_role tasks instead of the classic roles keyword.
When it makes sense, group your tasks in separate task files for improved clarity and organization.

Key Points

We deep dived into Ansible Roles and their utility and saw how to refactor our playbooks into roles or generate them from scratch. We went through a complete example of creating and using a role and explored how we can benefit from the Ansible Galaxy community.

Thank you for reading, and I hope you enjoyed this “Ansible Roles” blog post as much as I did.

Terraform Output Values : Complete Guide & Examples

Ioannis Moustakis — Sun, 15 May 2022 19:08:02 +0000

This blog post will deep dive into how Terraform handles output and how we can leverage and use output values efficiently across our Terraform projects. Output values allow us to share data between modules and workspaces while also providing us the flexibility to pass values to external systems for automation purposes.

You have come to the right place if you are new to Terraform! Spacelift has curated a ton of valuable material, tutorials, and blog posts around Terraform and how industry experts use it on its Spacelift blog.

Output vs Input Values

Input variables permit us to customize Terraform configurations without hardcoding any values. This way, we can reuse Terraform modules while assigning custom values based on our needs. Usually, we refer to them as just variables in the context of Terraform.

To define input variables, we must declare them using a variable block:

variable "aws_region" {
  description = "AWS region"
  type        = string
}

variable "ec2_instance_type" {
  description = "Instance type for EC2 instances"
  type        = string
  default     = "t2.small"
}

The variable’s name is the label we set following the variable keyword. For every variable, we have the option to set some arguments such as default, type, description, validation, sensitive, and nullable. Check the official documentation about these arguments and how to set them in detail here.

After declaring our input variables, we can utilize them in modules by referencing them like this var. where matches the label following the variable keyword. For example, to reference the variable ec2_instance_type that we defined above:

resource "aws_instance" "web_server" {
  ami           = data.aws_ami.amazon_linux.id
  instance_type = var.ec2_instance_type
}

On the other hand, output values empower us to export helpful information from our Terraform projects that we have defined and provisioned with Terraform. In the context of Terraform, we refer to output values as just outputs for simplicity.

Combining input and output variables, we get the flexibility to customize, automate, reuse and share our Terraform code easily. Input variables are similar to function arguments in traditional programming, while output variables work similarly to the return values of a function. Both are equally important to make our Terraform projects functional and facilitate data’s incoming and outgoing flow.

Terraform Outputs Use Cases

More specifically, output values are quite helpful in certain use cases:

We can expose information from child modules to a parent module using outputs.
Even more, from a root module, we can print outputs in the command line or pass these output values to external systems for automation purposes. When we use a remote state, we can access the root module outputs by other configurations using the terraform_remote_state data source. Output values from child modules aren’t accessible.

Declaring and Using Output Values

In order to define an output value, we have to use the output block:

output "instance_public_ip" {
  description = "Public IP of EC2 instance"
  value       = aws_instance.web_server.public_ip
}

In the above example, we define an output value with the name instance_public_ip. This way, we can pass the value to the parent module or display it to the end-user if it’s an output of the root module.

The value argument, which is the returned output value, takes an expression referencing other resources or module attributes. Terraform only renders and displays outputs when executing terraform apply and not when executing terraform plan.

To use outputs of nested modules from parent modules, we have to reference them as:

module.<module_name>.<output_value_name>

For example, to reference the output value instance_public_ip that we have declared above in a module named aws_web_server_instance from its parent module, we have to use:

module.aws_web_server_instance.instance_public_ip

Let’s examine how we can use all this in a real-world example. In this GitHub repository, we define the Terraform configuration for this example’s infrastructure. To follow along, you will need to install Terraform, have an AWS account ready, and authenticate with your AWS keys via the command line. Note that you might be charged a few dollars in your AWS account if you follow along.

In this example, we create the necessary infrastructure for a webserver. For the needs of this demo, we split our Terraform configuration into three modules, the root one and two child modules responsible for handling VPC-related resources and EC2 instance-related resources.

The project structure looks like this:

For each module, we define a main.tf file that handles the main functionality of the module.

Variable’s declarations and default values are populated in variables.tf files, while for the root module, we also use a terraform.tfvars file to set some variable values.

A good practice is to define our outputs in separate outputs.tf files, as you can see in the above example project structure. By declaring output values in an outputs.tf file per module, we improve the clarity of our modules as it’s easier for users to understand what outputs to expect from them quickly.

The root module utilizes and configures the aws provider and then just simply calls two child modules aws_web_server_vpc and aws_web_server_instance in main.tf of the top directory.

root module main.tf

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "3.16.0"
    }
  }
}

provider "aws" {
  region = var.aws_region
}

module "aws_web_server_vpc" {
  source = "./modules/aws-web-server-vpc"
}

module "aws_web_server_instance" {
  source            = "./modules/aws-web-server-instance"
  ec2_instance_type = var.ec2_instance_type
  vpc_id            = module.aws_web_server_vpc.vpc_id
  subnet_id         = module.aws_web_server_vpc.subnet_id
}

We notice that when calling the module aws_web_server_instance, we are passing two expressions using output values from the aws_web_server_vpc module with the notation module.. we have seen earlier.

root module outputs.tf

output "vpc_id" {
  description = "ID of the vpc"
  value       = module.aws_web_server_vpc.vpc_id
}

output "instance_id" {
  description = "ID of EC2 instance"
  value       = module.aws_web_server_instance.instance_id
}

output "instance_public_ip" {
   description = "Public IP of EC2 instance"
   value       = module.aws_web_server_instance.instance_public_ip
}

We define three output values for our root module, and we expect to see them at the command line after our infrastructure is provisioned. Checking the value parameter of each block, we notice that all of them are coming from output values of the two child modules, and by declaring them as output values of the root module, we are able to pass them through to the command line.

root module variables.tf

variable "aws_region" {
  description = "AWS region"
  type        = string
}

variable "ec2_instance_type" {
  description = "Instance type for EC2 instances"
  type        = string
  default     = "t2.small"
}

root module terraform.tfvars

aws_region        = "us-east-1"
ec2_instance_type = "t2.nano"

Let’s examine next our two child modules and how we use output values to pass parameters between them.

aws-web-server-vpc module main.tf

resource "aws_vpc" "web_server" {
  cidr_block       = var.vpc_cidr_block
  instance_tenancy = "default"

  tags = {
    Name = var.vpc_name
  }
}

resource "aws_subnet" "web_server" {
  vpc_id                  = aws_vpc.web_server.id
  cidr_block              = var.subnet_cidr_block
  map_public_ip_on_launch = true
  availability_zone       = var.aws_az

  tags = {
    Name = var.subnet_name
  }
}

resource "aws_internet_gateway" "web_server" {
  vpc_id = aws_vpc.web_server.id

  tags = {
    Name = var.igw_name
  }
}

resource "aws_route_table" "web_server" {
  vpc_id = aws_vpc.web_server.id
  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.web_server.id
  }

  tags = {
    Name = var.rt_name
  }
}

resource "aws_route_table_association" "web_server" {
  subnet_id      = aws_subnet.web_server.id
  route_table_id = aws_route_table.web_server.id
}

In the above module, we define some resources necessary for the networking layer of our infrastructure.

aws-web-server-vpc module variables.tf

variable "vpc_cidr_block" {
  description = "CIDR block for webserver VPC"
  type        = string
  default     = "10.0.0.0/16"
}

variable "vpc_name" {
  description = "Name of the vpc"
  type        = string
  default     = "web_server"
}

variable "subnet_cidr_block" {
  description = "CIDR block for the webserver subnet"
  type        = string
  default     = "10.0.0.0/24"
}

variable "subnet_name" {
  description = "Name for the webserver subnet"
  type        = string
  default     = "web_server"
}

variable "aws_az" {
  description = "Availability Zone for the webserver subnet"
  type        = string
  default     = "us-east-1a"
}

variable "igw_name" {
  description = "Name for the Internet Gateway of the webserver vpc"
  type        = string
  default     = "web_server"
}

variable "rt_name" {
  description = "Name for the route table of the webserver vpc"
  type        = string
  default     = "web_server"
}

aws-web-server-vpc module outputs.tf

output "vpc_id" {
  description = "ID of the VPC"
  value       = aws_vpc.web_server.id
}

output "subnet_id" {
  description = "ID of the VPC subnet"
  value       = aws_subnet.web_server.id
}

The two outputs we export here from this module are passed to the aws-web-server-instance module as parameters in order to create the EC2 instance inside the vpc and subnet that we have just created. We saw how this was handled in the main.tf file of the root module. The output value vpc_id is passed along as an output of the root module and should be printed in the command line after we apply the plan.

Finally, the Terraform configuration for the aws-web-server-instance module uses the passed info from the aws-web-server-vpc module. It creates and configures the web server instance accordingly.

aws-web-server-instance module main.tf

data "aws_ami" "amazon_linux" {
  most_recent = true
  owners      = ["amazon"]
  filter {
    name   = "name"
    values = ["amzn2-ami-hvm-*-x86_64-gp2"]
  }
}

resource "aws_security_group" "web_server" {
  name        = var.ec2_security_group_name
  description = var.ec2_security_group_description
  vpc_id      = var.vpc_id

  ingress {
    description      = "Allow traffic on port 80 from everywhere"
    from_port        = 80
    to_port          = 80
    protocol         = "tcp"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }

  egress {
    from_port        = 0
    to_port          = 0
    protocol         = "-1"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }

  tags = {
    Name = var.ec2_security_group_name
  }
}

resource "aws_instance" "web_server" {
  ami           = data.aws_ami.amazon_linux.id
  instance_type = var.ec2_instance_type

  subnet_id              = var.subnet_id
  vpc_security_group_ids = [aws_security_group.web_server.id]

  tags = {
    Name = var.ec2_instance_name
  }

  user_data = <<-EOF
   #!/bin/bash
   sudo yum update -y
   sudo yum install httpd -y
   sudo systemctl enable httpd
   sudo systemctl start httpd
   echo "<html><body><div>This is a test webserver!</div></body></html>" > /var/www/html/index.html
   EOF
}

aws-web-server-instance module variables.tf

variable "ec2_instance_name" {
  description = "Name for web server EC2 instance"
  type        = string
  default     = "web_server"
}

variable "ec2_instance_type" {
  description = "Instance type for web server EC2 instance"
  type        = string
  default     = "t2.micro"
}

variable "ec2_security_group_name" {
  description = "Security group name for web server EC2 instance"
  type        = string
  default     = "web_server"

}

variable "ec2_security_group_description" {
  description = "Security group description for web server EC2 instance"
  type        = string
  default     = "Allow traffic for webserver"
}

variable "vpc_id" {
  description = "VPC id for web server EC2 instance"
  type        = string
}

variable "subnet_id" {
  description = "Subnet id for web server EC2 instance"
  type        = string
}

The two output values that we pass through the root module are also defined in this module’s outputs.tf file.

output "instance_id" {
  description = "ID of EC2 instance"
  value       = aws_instance.web_server.id
}

output "instance_public_ip" {
  description = "Public IP of EC2 instance"
  value       = aws_instance.web_server.public_ip
}

Time to wrap up everything and execute the plan to provision our demo infrastructure.

Our terraform plan shows 7 new resources to be added and displays the changes to our three output values declared in the root module. Let’s go ahead and apply the plan.

As expected, the three outputs declared in the root module are displayed at the command line, sweet!

We could use these values to automate other parts of our systems and process, but for now, we can get the value from instance_public_ip and head to http://, and we should see our demo web server up and running.

Great! Everything works as expected.

Terraform Output Command

Output values are stored in the state Terraform file. Since we have successfully applied our plan, we can now access these output values at will. We can leverage the terraform output command for this purpose.

Additionally, we can query individual output values by name like this

terraform output

To get the raw value without quotes, use the -raw flag.

_terraform output -raw _

To get the JSON-formatted output, we can use the -json flag.

This is quite useful when we want to pass the outputs to other tools for automation since JSON is way easier to handle programmatically. Note that Terraform does not protect sensitive output values when using the -json flag.

When we are done, let’s go ahead and delete all these resources to avoid paying for them.

From the top of our repository, execute:

terraform destroy

Output Values Options & Arguments

When defining output values, we have a couple of options that might help us better define and organize them.

The argument description is optional, but it is always considered good practice to include it in our output declarations to document their purpose. This argument should briefly explain each output’s intent and should be used as a helper description for the users of the module. We have already seen examples like this since we defined the description argument in all our output block declarations in our previous demo.

In cases where we want to handle sensitive values and suppress them in command line output, we can declare an output value as sensitive. Terraform will redact the values of sensitive outputs when planning, applying, destroying, or querying outputs to avoid printing them to the console. In practice, this is a good use case when we would like to pass values to other Terraform modules or automation tools without exposing them to the intermediate users.

output "example_password" {
  description = "An example DB password"
  value       = aws_db_instance.database.password
  sensitive   = true
}

Note that Terraform won’t redact sensitive output values when you query a specific output by name. After we apply a plan with an output declared as sensitive, the console displays a message with the value redacted.

These values are still recorded in the state files, so anyone who can access them can also access any sensitive values of our Terraform configuration.

The depends_on argument on output declarations is used to define dependencies explicitly when this is necessary. Most of the time, Terraform handles this automatically, but there are some rare uses cases where you might find this option handy when it’s not the case. Consider including a comment when you use this option to explain why this is necessary.

Terraform Remote State Data Source

Occasionally, we might need to share data between different Terraform configurations with separate states. This is where the terraform_remote_state data sources come into play. We can retrieve the root module outputs from another Terraform configuration using this data source. This built-in data source is available without any extra configuration needed.

Following up on our previous example, let’s say that we would like to create a new subnet in the vpc of our aws-web-server-vpc module. This time, the new subnet needs to be defined in a completely separate Terraform configuration that has its own state. We can leverage the terraform_remote_state to get the value of the vpc_id defined as an output of our previous example’s root module. In this case, we use the local backend to reach the state of another configuration in the local machine. The backend could be any remote backend that points to a Terraform state in a real-world scenario.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "3.16.0"
    }
  }
}

provider "aws" {
  region = "us-east-1"
}

data "terraform_remote_state" "terraform_output" {
  backend = "local"

  config = {
    path = "../terraform-output/terraform.tfstate"
  }
}

resource "aws_subnet" "test_terraform_remote_state_subnet" {
  vpc_id            = data.terraform_remote_state.terraform_output.outputs.vpc_id
  cidr_block        = "10.0.1.0/24"
  availability_zone = "us-east-1b"

  tags = {
   Name = "test_terraform_remote_state_subnet"
  }
}

Note that only the output values of the root module are accessible from the remote state. If we want to pass values from nested modules, we have to configure a passthrough output value declaration as we defined earlier in the root module of our previous example.

Although this option is handy for some use cases, it also has some caveats. To use this data source, the user must have access to the entire state snapshot, which could potentially expose sensitive data. Check out the official docs to find alternative ways to share data between configurations.

Key Points
We have seen how Terraform handles and exports output values between modules and the different options for outputs configuration. Even more, we compared input and output variables and examined multiple use cases where the use of outputs is helpful. Finally, we went through a complete example of using output values in our Terraform configuration between different modules and printing them to the console.

Thank you for reading, and I hope you enjoyed this “Terraform Outputs” blog post as much as I did.

How to Use Different Types of Ansible Variables(Examples)

Ioannis Moustakis — Sun, 15 May 2022 18:21:47 +0000

This blog post deep dives into Ansible Variables, which allow us to parametrize different Ansible components. Variables store values for reuse inside an Ansible project.

If you are still learning how to use Ansible, you might also find helpful the introductory Ansible Tutorial or Working with Ansible Playbooks blog posts. You can find this article’s code on this repository if you wish to follow along.

Why Variables Are Useful in Ansible

The use of variables simplifies the management of dynamic values throughout an Ansible project and can potentially reduce the number of human errors. We have a convenient way to handle variations and differences between different environments and systems with variables.

Another advantage of variables in Ansible is that we have the flexibility to define them in multiple places with different precedence according to our use case. We can also register new variables in our playbooks by using the returned value of a task.

Ansible facts are a special type of variables that Ansible retrieves from any remote host for us to leverage them in Ansible projects. For example, we can get information regarding the operating system distribution with ansible_distribution, information about devices on the host, the python version that Ansible is using with ansible_python_version, and the system architecture, among others. To access this data, we have to reference the ansible_facts variable.

Variable Name Rules

Ansible has a strict set of rules to create valid variable names. Variable names can contain only letters, numbers, and underscores and must start with a letter or underscore. Some strings are reserved for other purposes and aren’t valid variable names, such as Python Keywords or Playbook Keywords.

Defining and Referencing Simple Variables

The simplest use case of variables is to define a variable name with a single value using standard YAML syntax. Although this pattern can be used in many places, we will show an example in a playbook for simplicity.

- name: Example Simple Variable
  hosts: all
  become: yes
  vars:
    username: bob

  tasks:
  - name: Add the user {{ username }}
    ansible.builtin.user:
      name: "{{ username }}"
      state: present

In the above example, after the vars block, we define the variable username, and assign the value bob. Later, to reference the value in the task, we use Jinja2 syntax like this "{{ username }}"

If a variable’s value starts with curly braces, we must quote the whole expression to allow YAML to interpret the syntax correctly.

List, Dictionary & Nested Variables

There are many other options to define more complex variables like lists, dictionaries, and nested structures. To create a variable with multiple values, we can use YAML lists syntax:

vars:
  version:
    - v1
    - v2
    - v3

To reference a specific value from the list we must select the correct field. For example, to access the third value v3:

version: "{{ version[2] }}"

Another useful option is to store key-value pairs in variables as dictionaries. For example:

vars:
  users: 
    - user_1: maria
    - user_2: peter
    - user_3: sophie

Similarly, to reference the third field from the dictionary, use the bracket or dot notation:

users['user_3']
users.user_3

Note that the bracket notation is preferred as you might encounter problems using the dot notation in special cases.

Sometimes, we have to create or use nested variable structures. For example, facts are nested data structures. We have to use a bracket or dot notation to reference nested variables.

vars:
  cidr_blocks:
      production:
        vpc_cidr: "172.31.0.0/16"
      staging:
        vpc_cidr: "10.0.0.0/24"

tasks:
- name: Print production vpc_cidr
  ansible.builtin.debug:
    var: cidr_blocks['production']['vpc_cidr']

Special Variables

There are certain types of variables that we consider special in the context of Ansible. These include magic variables, connection variables, and facts. The names of these variables are reserved.

Ansible allows us to access information about itself, hosts, groups, inventory, roles, and other Ansible manifests with the so-called magic variables. For a complete list of different options, have a look here.

We already talked about facts. These variables contain all the information that Ansible can get from the current host. To use them, Ansible has to gather them first. To see all the facts that you can gather on a host, execute:

ansible <hostname> -m ansible.builtin.setup

Lastly, we have connection variables. They are used to configure Ansible execution behavior and actions on hosts. The most common ones configure the user that Ansible logs in, set privilege escalation, set the IP of the target host, etc.

Registering Variables

During our plays, we might find it handy to utilize the output of a task as a variable that we can use in the following tasks. We can use the keyword register to create our own custom variables from task output.

- name: Example Register Variable Playbook
  hosts: all

  tasks:
  - name: Run a script and register the output as a variable
    shell: "find hosts"
    args:
      chdir: "/etc"
    register: find_hosts_output
  - name: Use the output variable of the previous task
    debug:
      var: find_hosts_output

In the above example, we register the output of the command find /etc/hosts, and we showcase how we can use the variable in the next task by printing its value.

A powerful pattern is to combine registered variables with conditionals to create tasks that will only be executed when certain custom conditions are true.

- name: Example Registered Variables Conditionals
  hosts: all

  tasks:
  - name: Register an example variable
    shell: cat /etc/hosts
    register: hosts_contents

  - name: Check if hosts file contains the word "localhost"
    debug:
      msg: "/etc/hosts file contains the word localhost"
    when: hosts_contents.stdout.find("localhost") != -1
      var: find_hosts_output

Here, we registered in the variable hosts_contents the contents of /etc/hosts file, and we execute the second task only if the file contains the word localhost.

Since registered variables are stored in memory, it’s not possible to use them in future plays, and they are only available for the current playbook run.

Share Variables with YAML Anchors and Aliases

When we want to reuse and share variables, we can leverage Y_AML anchors and aliases_. They provide us with great flexibility in handling shared variables and help us reduce the repetition of data.

Anchors are defined with &, and then referenced with an alias denoted with *****. Let’s go and check a hands-on example in a playbook.

- name: Example Anchors and Aliases
  hosts: all
  become: yes
  vars:
    user_groups: &user_groups
     - devs
     - support
    user_1:
        user_info: &user_info
            name: bob
            groups: *user_groups
            state: present
            create_home: yes
    user_2:
        user_info:
            <<: *user_info
            name: christina
    user_3:
        user_info:
            <<: *user_info
            name: jessica
            groups: support

  tasks:
  - name: Add several groups
    ansible.builtin.group:
      name: "{{ item }}"
      state: present
    loop: "{{ user_groups }}"

  - name: Add several users
    ansible.builtin.user:
      <<: *user_info
      name: "{{ item.user_info.name }}"
      groups: "{{ item.user_info.groups }}"
    loop:
      - "{{ user_1 }}"
      - "{{ user_2 }}"
      - "{{ user_3 }}"

Here, since some options are shared between users, instead of rewriting the same values, we share the common ones with the anchor &user_info. For every subsequent user declaration, we use the alias *user_info to avoid repeating ourselves as much as possible.

The values for state and create_home are the same for all the users, while name and groups are replaced using the merge operator <<.

Similarly, we reuse the user_groups declaration in the definition of the user_info anchor. This way, we don’t have to type the same groups again for user_2 while we still have the flexibility to override the groups, as we do for user_3.

The result is that user_1 and user_2 are added to groups devs and support, while user_3 is added only to the support group.

Variable Scope

Ansible provides many options on setting variables, and the ultimate decision on where to set them lies with us based on the scope we would like them to have. Conceptually, there are three main options available for scoping variables.

First, we have the global scope where the values are set for all hosts. This can be defined by the Ansible configuration, environment variables, and command line.

We set values for a particular host or group of hosts using the host scope. For example, there is an option to define some variables per host in the inventory file.

Lastly, we have the play scope, where values are set for all hosts in the context of a play. An example would be the vars section we have seen in previous examples in each playbook.

Variable Setting Options & Precedence

Variables can be defined with Ansible in many different places. There are options to set variables in playbooks, roles, inventory, var files, and command line. Let’s go and explore some of these options.

As we have previously seen, the most straightforward way is to define variables in a play with the vars section.

- name: Set variables in a play
  hosts: all
  vars:
    version: 12.7.1

Another option is to define variables in the inventory file. We can set variables per host or set shared variables for groups. This example defines a different ansible user to connect for each host as a host variable and the same HTTP port for all web servers as a group variable.

[webservers]
webserver1 ansible_host=10.0.0.1 ansible_user=user1
webserver2 ansible_host=10.0.0.2 ansible_user=user2

[webservers:vars]
http_port=80

To better organize our variables, we could gather them in separate host and group variables files. In the same directory where we keep our inventory or playbook files, we can create two folders named group_vars and host_vars that would contain our variable files. For example:

group_vars/databases 
group_vars/webservers
host_vars/host1
host_vars/host2

Variables can also be set in custom var files. Let’s check an example that uses variables from an external file and the group_vars and host_vars directories.

- name: Example External Variables file
  hosts: all
  vars_files:
    - ./vars/variables.yml

  tasks:
  - name: Print the value of variable docker_version
    debug: 
      msg: "{{ docker_version}} "

  - name: Print the value of group variable http_port
    debug: 
      msg: "{{ http_port}} "

  - name: Print the value of host variable app_version
    debug: 
      msg: "{{ app_version}} "

The vars/variables.yml file:

docker_version: 20.10.12

The group_vars/webservers file:

http_port: 80
ansible_host: 127.0.0.1
ansible_user: vagrant

The host_vars/host1 file:

app_version: 1.0.1
ansible_port: 2222
ansible_ssh_private_key_file: ./.vagrant/machines/host1/virtualbox/private_key

The host_vars/host2 file:

app_version: 1.0.2
ansible_port: 2200
ansible_ssh_private_key_file: ./.vagrant/machines/host2/virtualbox/private_key

The inventory file contains a group named webservers that includes our two hosts, host1 and host2:

[webservers]
host1 
host2

If we run this playbook, we notice the same value is used in both hosts for the group variable http_port but a different one for the host variable app_version.

A good use case for having separate variables files is that you can keep in them sensitive values without storing them in playbooks or source control systems.

Occasionally we might find it helpful to define or override variables at runtime by passing them at the command line with --extra-vars or –e argument. For example:

ansible-playbook example-external-vars.yml --extra-vars "app_version=1.0.3"

Since variables can be set in multiple places, Ansible applies variable precedence to select the variable value according to some hierarchy. The general rule is that variables defined with a more explicit scope have higher priority.

For example, role defaults are overridden by mostly every other option. Variables are also flattened to each host before each play so all group and hosts variables are merged. Host variables have higher priority than group variables.

Explicit variables definitions like the vars directory or an include_vars task override variables from the inventory. Finally, extra vars defined at runtime always win precedence. For a complete list of options and their hierarchy, look at the official documentation Understanding variable precedence.

Where to Set Variables & Best Practices

Since Ansible provides a plethora of options to define variables, it might be a bit confusing to figure out the best way and place to set them. Let’s go and check some common & best practices around setting variables that might help us better organize our Ansible projects.

Always give descriptive and clear names to your variables. Taking a moment to properly think about how to name variables always pays off long-term.
If there are default values for common variables, set them in group_vars/all
Prefer setting group and host vars in group_vars and host_vars directories instead of in the inventory file.
If variables related to geography or behavior are tied to a specific group, prefer to set them as group variables.
If you are using roles, always set default role variables in roles/your_role/defaults/main.yml
When you call roles, pass variables that you wish to override as parameters to make your plays easier to read.

roles:
       - role: example_role
         vars:
            example_var: 'example_string'

You can always use --extra-vars or –e to override every other option.
Don’t store sensitive variables in your source code repository in plain text. You can leverage Ansible Vault in these cases.

In general, try to keep variables usage as simple as possible. You don’t have to use all the existing options and spread variables definition all over the place because that makes debugging your Ansible projects difficult. Try to find a structure that suits your needs best and stick to it!

Key Points
In this article, we deep-dived into Ansible Variables and saw how we can define and use them in playbooks. Moreover, we explored different options for sharing, setting, and referencing them, along with some guidelines and best practices to make our Ansible journey easier.

Thank you for reading, and I hope you enjoyed this “Ansible Variables” article as much as I did.

Working with Ansible Playbooks – Tips & Tricks with Examples

Ioannis Moustakis — Sun, 15 May 2022 17:14:57 +0000

In this article, we are exploring Ansible Playbooks, which are basically blueprints for automation actions. Playbooks allow us to define a recipe with all the steps we would like to automate in a repeatable, simple, and consistent manner.

If you are entirely new to Ansible, check out this introductory Ansible Tutorial first.

What is an Ansible Playbook?

Playbooks are one of the basic components of Ansible as they record and execute Ansible’s configuration. Generally, a playbook is the primary way to automate a set of tasks that we would like to perform on a remote machine.

They help our automation efforts by gathering all the resources necessary to orchestrate ordered processes or avoid repeating manual actions. Playbooks can be reused and shared between persons, and they are designed to be human-friendly and easy to write in YAML.

Playbook Structure

A playbook is composed of one or more plays to run in a specific order. A play is an ordered list of tasks to run against the desired group of hosts.

Every task is associated with a module responsible for an action and its configuration parameters. Since most tasks are idempotent, we can safely rerun a playbook without any issue.

As discussed, playbooks are written in YAML using the standard extension .yml with minimal syntax.

We must use spaces to align data elements that share the same hierarchy for indentation. Items that are children of other items must be indented more than their parents. There is no strict rule for the number of spaces used for indentation, but it’s pretty common to use two spaces while Tab characters are not allowed.

Below is an example simple playbook with only two plays, each one having two tasks:

---
- name: Example Simple Playbook
  hosts: all
  become: yes

  tasks:
  - name: Copy file example_file to /tmp with permissions
    ansible.builtin.copy:
      src: ./example_file
      dest: /tmp/example_file
      mode: '0644'

  - name: Add the user 'bob' with a specific uid 
    ansible.builtin.user:
      name: bob
      state: present
      uid: 1040

- name: Update postgres servers
  hosts: databases
  become: yes

  tasks:
  - name: Ensure postgres DB is at the latest version
    ansible.builtin.yum:
      name: postgresql
      state: latest

  - name: Ensure that postgresql is started
    ansible.builtin.service:
      name: postgresql
      state: started

We define a descriptive name for each play according to its purpose on the top level. Then we represent the group of hosts on which the play will be executed, taken from the inventory. Finally, we define that these plays should be executed as the root user with the become option set to yes.

You can also define many other Playbook Keywords at different levels such as play, tasks, playbook to configure Ansible’s behavior. Even more, most of these can be set at runtime as command-line flags in the ansible configuration file, ansible.cfg, or the inventory. Check out the precedence rules to understand how Ansible behaves in these cases.

Next, we use the tasks parameter to define the list of tasks for each play. For each task, we define a clear and descriptive name. Every task leverages a module to perform a specific operation.

For example, the first task of the first play uses the ansible.builtin.copy module. Along with the module, we usually have to define some module arguments. For the second task of the first play, we use the module ansible.builtin.user that helps us manage user accounts. In this specific case, we configure the name of the user, the state of the user account, and its uid accordingly.

Running a Playbook

When we are running a playbook, Ansible executes each task in order, one at a time, for all the hosts that we selected. This default behavior could be adjusted according to different use cases using strategies.

If a task fails, Ansible stops the execution of the playbook to this specific host but continues to others that succeeded. During execution, Ansible displays some information about connection status, task names, execution status, and if any changes have been performed.

At the end, Ansible provides a summary of the playbook’s execution along with failures and successes. Let’s see these in action by running the example playbook we saw earlier with the ansible-playbook command.

From the output, we notice the Play names, the Gathering Facts task, the Play tasks, and the Play Recap in the end. Since we didn’t define a databases hosts group, the second play of the playbook was skipped.

We can use the --limit flag to limit the Playbook’s execution to specific hosts. For example:

ansible-playbook example-simple-playbook.yml --limit host1

Using Variables in Playbooks

Variables are placeholders for values that you can reuse throughout a Playbook or other Ansible objects. They can only contain letters, numbers, and underscores and start with letters.

Variables can be defined in Ansible in multiple levels, so look at variable precedence to understand how they are applied. For example, we can set variables at the global scope for all hosts, at the host scope for a particular host, or at the play scope for a specific play.

To set host and group variables, create the directories group_vars and host_vars. For example, to define group variables for the databases group, create the file group_vars/databases. Set common default variables in a group_vars/all file.

Even more, to define host variables for a specific host, create a file with the same name as the host under the hosts_vars directory.

To substitute any variables during runtime, use the -e flag.

The most straightforward method to define variables is to use a vars block at the beginning of a play. They are defined using standard YAML syntax.

- name: Example Variables Playbook
  hosts: all
  vars:
    username: bob
    version: 1.2.3

Another way is to define variables in external YAML files.

- name: Example Variables Playbook
  hosts: all
  vars_files:
    - vars/example_variables.yml

To use them in tasks, we have to reference them by placing their name inside double braces using the Jinja2 syntax:

- name: Example Variables Playbook
  hosts: all
  vars:
    username: bob

  tasks:
  - name: Add the user {{ username }}
    ansible.builtin.user:
      name: "{{ username }}"
      state: present

If a variable’s value starts with curly braces, we must quote the whole expression to allow YAML to interpret the syntax correctly.

We can also define variables with multiple values as lists.

package:
  - foo1
  - foo2
  - foo3

It’s also possible to reference individual values from a list. For example, to select the first value foo1:

package: "{{ package[0] }}"

Another possible option is to define variables using YAML dictionaries. For example:

dictionary_example: 
  - foo1: one
  - foo2: two

Similarly, to get the first field from the dictionary:

dictionary_example['foo1']

To reference nested variables, we have to use a bracket or dot notation. For example, to get the example_name_2 value from this structure:

vars:
  var1:
    foo1:
      field1: example_name_1
      field2: example_name_2

tasks:
- name: Create user for field2 value
  user: 
    name: "{{ var1['foo1']['field2'] }}"

We can create variables using the register statement that captures the output of a command or task and then use them in other tasks.

- name: Example-2 Variables Playbook
  hosts: all

  tasks:
  - name: Run a script and register the output as a variable
    shell: "find example_file"
    args:
      chdir: "/tmp"
    register: example_script_output

  - name: Use the output variable of the previous task
    debug:
      var: example_script_output

Handling Sensitive Data

At times, we would need to access sensitive data (API keys, passwords, etc.) in our playbooks. Ansible provides Ansible Vault to assist us in these cases. Storing them as variables in plaintext is considered a security risk so we can use the ansible-vault command to encrypt and decrypt these secrets.

After the secrets have been encrypted with a password of your choice, you can safely put them under source control in your code repositories. Ansible Vault protects only data at rest. After the secrets are decrypted, it’s our responsibility to handle them with care and not accidentally leak them.

We have the option to encrypt variables or files. Encrypted variables are decrypted on-demand only when needed, while encrypted files are always decrypted as Ansible doesn’t know in advance if it needs content from them.

In any case, we need to think about how are we going to manage our vault passwords. To define encrypted content, we add the !vault tag, which tells Ansible that the content needs to be decrypted and the | character before our multi-line encrypted string.

To create a new encrypted file:

ansible-vault create new_file.yml

Then, an editor is launched to add our content to be encrypted. It’s also possible to encrypt existing files with the encrypt command:

ansible-vault encrypt existing_file.yml

To view an encrypted file:

ansible-vault view existing_file.yml

To edit an encrypted file in place, use the edit command to decrypt the file temporarily:

ansible-vault edit existing_file.yml

To use a different password on an encrypted file, use the rekey command by using the original password:

ansible-vault rekey existing_file.yml

In case you need to decrypt a file, you can do so with the decrypt command:

ansible-vault decrypt existing_file.yml

Similarly, we use the encrypt_string command to encrypt individual strings that we can use later in variables and include them in playbooks or variables files:

ansible-vault encrypt_string <password_source> '<string_to_encrypt>' –'<variable_name>'

For example, to encrypt the db_password string ‘12345679’ using the ansible vault:

Since we omitted the , we manually entered the Vault password. This could also be achieved by passing a password file with --vault-password-file.

To view the contents of the above example encrypted variable that we saved in the vars.yml file, use the same password as before with the --ask-vault-pass flag:

ansible localhost -m ansible.builtin.debug -a var="db_password" -e "@vars.yml" --ask-vault-pass

Vault password:

localhost | SUCCESS => {
    "changed": false,
    "db_password": "12345678"
}

For managing multiple passwords, use the option --vault-id to set a label. For example, to set the label dev on a file and prompt for a password to use:

ansible-vault encrypt existing_file.yml --vault-id dev@prompt

To suppress output from a task that might log a sensitive value to the console, we use the no_log: true attribute:

tasks:
- name: Hide sensitive value example
  debug:
    msg: "This is sensitive information"
  no_log: true

If we run this task we will notice that the message isn’t printed on the console:

TASK [Hide sensitive value example] ***********************************
ok: [host1]

Finally, let’s use the example encrypted variable we created above in a playbook and execute it.

Nice, we verified that we could decrypt the value successfully and use it in tasks.

Triggering tasks on change with Handlers

In general, Ansible modules are idempotent and can be executed safely multiple times, but there are cases where we would like to run a task only when a change is made on the host. For example, we would like to restart a service only when updating its configuration files.

Ansible uses handlers triggered when notified by other tasks to solve this use case. Tasks only notify their handlers, with the notify: parameter, when the tasks actually change something.

Handlers should have globally unique names, and it’s common to author them at the bottom of the playbooks.

- name: Example with handler - Update apache config
  hosts: webservers

  tasks:
  - name: Update the apache config file
    ansible.builtin.template:
      src: ./httpd.conf
      dest: /etc/httpd.conf
    notify:
    - Restart apache

  handlers:
    - name: Restart apache
      ansible.builtin.service:
        name: httpd
        state: restarted

In the above example, the Restart apache task will only be triggered when we change something in the configuration. In reality, handlers can be considered inactive tasks waiting to be triggered with a notify statement.

An important thing to note about handlers is that they run by default after all the other tasks have been completed. This way, the handlers only run once, even if triggered many times.

To control this behavior, we can leverage the **meta: flush_handlers* task that triggers any handlers that have been already notified at that time.

It’s also possible for a task to notify more than one handler in its notify statement.

Conditional Tasks

To further control execution flow in Ansible, we can leverage conditionals. Conditionals allow us to run or skip tasks based on if certain conditions are met. Variables, facts, or results of previous tasks along with operators, can be used to create such conditions.

Some examples of use cases could be to update a variable based on a value of another variable, skip a task if a variable has a specific value, execute a task only if a fact from the host returns a value higher than a threshold.

To apply a simple conditional statement, we use the when parameter on a task. If the condition is met, the task is executed. Otherwise, it is skipped.

- name: Example Simple Conditional
  hosts: all
  vars:
    trigger_task: true

  tasks:
  - name: Install nginx
    apt:
      name: "nginx"
      state: present
    when: trigger_task

In the above example, the task is executed since the condition is met.

Another common pattern is to control task execution based on attributes of the remote host that we can obtain from facts. Check out this list with commonly-used facts to get an idea of all the facts we can utilize in conditions.

- name: Example Facts Conditionals 
  hosts: all
  vars:
    supported_os:
      - RedHat
      - Fedora

  tasks:
  - name: Install nginx
    yum:
      name: "nginx"
      state: present
    when: ansible_facts['distribution'] in supported_os

It’s possible to combine multiple conditions with logical operators and group them with parenthesis:

when: (colour=="green" or colour=="red") and (size="small" or size="medium")

Then when statement supports using a list in cases when we have multiple conditions that all need to be true:

when:
  - ansible_facts['distribution'] == "Ubuntu"
  - ansible_facts['distribution_version'] == "20.04"
  - ansible_facts['distribution_release'] == "bionic"

Another option is to use conditions based on registered variables that we have defined in previous tasks:

- name: Example Registered Variables Conditionals
  hosts: all

  tasks:
  - name: Register an example variable
    ansible.builtin.shell: cat /etc/hosts
    register: hosts_contents

  - name: Check if hosts file contains "localhost"
    ansible.builtin.shell: echo "/etc/hosts contains localhost"
    when: hosts_contents.stdout.find(localhost) != -1

Loops

Ansible allows us to iterate over a set of items in a task to execute it multiple times with different parameters without rewriting it. For example, to create several files, we would use a task that iterates over a list of directory names instead of writing five tasks with the same module.

To iterate over a simple list of items, use the loop keyword. We can reference the current value with the loop variable item.

- name: "Create some files"
  ansible.builtin.file:
    state: touch
    path: /tmp/{{ item }}
  loop:
    - example_file1
    - example_file2
    - example_file3

The output of the above task that uses loop and item:

TASK [Create some files] *********************************
changed: [host1] => (item=example_file1)
changed: [host1] => (item=example_file2)
changed: [host1] => (item=example_file3)

It’s also possible to iterate over dictionaries:

- name: "Create some files with dictionaries"
  ansible.builtin.file:
    state: touch
    path: "/tmp/{{ item.filename }}"
    mode: "{{ item.mode }}"
  loop:
    - { filename: 'example_file1', mode: '755'}
    - { filename: 'example_file2', mode: '775'}
    - { filename: 'example_file3', mode: '777'}

Another useful pattern is to iterate over a group of hosts of the inventory:

- name: Show all the hosts in the inventory
  ansible.builtin.debug:
    msg: "{{ item }}"
  loop: "{{ groups['databases'] }}"

By combining conditionals and loops, we can select to execute the task only on some items in the list and skip it for others:

- name: Execute when values in list are lower than 10
  ansible.builtin.command: echo {{ item }}
  loop: [ 100, 200, 3, 600, 7, 11 ]
  when: item < 10

Finally, another option is to use the keyword until to retry a task until a condition is true.

- name: Retry a task **until** we find the word "success" in the logs
  shell: cat /var/log/example_log
  register: logoutput
  until: logoutput.stdout.find("success") != -1
  retries: 10
  delay: 15

In the above example, we check the file example_log 10 times, with a delay of 15 seconds between each check until we find the word success. If we let the task run and add the word success to the example_log file after a while, we notice that the task stops successfully.

TASK [Retry a task until we find the word “success” in the logs] *********
FAILED - RETRYING: Retry a task until we find the word "success" in the logs (10 retries left).
FAILED - RETRYING: Retry a task until we find the word "success" in the logs (9 retries left).
changed: [host1]

Check out the official Ansible guide on Loops for more advanced use cases.

Ansible Playbooks Tips and Tricks

Keeping these tips and tricks in mind when building your playbooks will help you be more productive and improve your efficiency.

1) Keep it as simple as possible

Try to keep your tasks simple. There are many options and nested structures in Ansible, and by combining lots of features, you can end up with fairly complex setups. Spending some time simplifying your Ansible artifacts pays off in the long term.

2) Place your Ansible artifacts under version control

It’s considered best practice to store playbooks in git or any other version control system and take advantage of its benefits.

3) Always give descriptive names to your tasks, plays, and playbooks

Choose names that help you and others quickly understand the artifact’s functionality and purpose.

4) Strive for readability

Use consistent indentation and add blank lines between tasks to increase readability.

5) Always mention the state of tasks explicitly

Many modules have a default state that allows us to skip the state parameter. It’s always better to be explicit in these cases to avoid confusion.

6) Use comments when necessary

There will be times when the task definition won’t be enough to explain the whole situation, so feel free to use comments for more complex parts of playbooks.

Key Points

In this article, we had a look into Ansible’s core automation component, playbooks. We saw how to create, structure, and trigger playbook runs.

Moreover, we explored leveraging variables, handling sensitive data, controlling task execution with handlers and conditions, and iterating over tasks with loops.

Thank you for reading, and I hope you enjoyed this “Ansible: Working with Playbooks” article as much as I did.

Btw. If you are looking to manage infrastructure as code, Spacelift is the way to go. It supports Git workflows, policy as code, programmatic configuration, context sharing, and many more great features. It currently works with Terraform, Pulumi, and CloudFormation, with support for Ansible on the way! You can test drive it for free, by going here and creating a trial account.

Ansible Tutorial for Beginners: Playbook & Examples

Ioannis Moustakis — Sun, 15 May 2022 16:47:38 +0000

Ansible is one of the most used tools for managing cloud and on-premises infrastructure. If you are looking for a flexible and powerful tool to automate your infrastructure management and configuration tasks Ansible is the way to go.

In this introductory guide, you will learn everything you need to get started with Ansible and start building robust automation solutions.

What is Ansible?

Ansible is a software tool that enables cross-platform automation and orchestration at scale and has become over the years the standard choice among enterprise automation solutions.

It’s mostly addressed to IT operators, administrators & decision-makers helping them to achieve operational excellence across their entire infrastructure ecosystem.

Backed by RedHat and a loyal open source community, it is considered an excellent option for configuration management, infrastructure provisioning, and application deployment use cases.

Its automation opportunities are endless across hybrid clouds, on-prem infrastructure, and IoT and it’s an engine that can greatly improve the efficiency and consistency of your IT environments.

Ready to automate everything? Let’s go!

How does Ansible work?

Ansible uses the concepts of control and managed nodes. It connects from the control node, any machine with Ansible installed, to the managed nodes sending commands and instructions to them.

The units of code that Ansible executes on the managed nodes are called modules. Each module is invoked by a task, and an ordered list of tasks together forms a playbook. Users write playbooks with tasks and modules to define the desired state of the system.

The managed machines are represented in a simplistic inventory file that groups all the nodes into different categories.

Ansible leverages a very simple language, YAML, to define playbooks in a human-readable data format that is really easy to understand from day one.

Even more, Ansible doesn’t require the installation of any extra agents on the managed nodes so it’s simple to start using it.

Typically, the only thing a user needs is a terminal to execute Ansible commands and a text editor to define the configuration files.

Benefits of using Ansible

A free and open-source community project with a huge audience.
Battle-tested over many years as the preferred tool of IT wizards.
Easy to start and use from day one, without the need for any special coding skills.
Simple deployment workflow without any extra agents.
Includes some sophisticated features around modularity and reusability that come in handy as users become more proficient.
Extensive and comprehensive official documentation that is complemented by a plethora of online material produced by its community.
To sum up, Ansible is simple yet powerful, agentless, community-powered, predictable, and secure.

Basic Concepts & Terms

Host: A remote machine managed by Ansible.

Group: Several hosts grouped together that share a common attribute.

Inventory: A collection of all the hosts and groups that Ansible manages. Could be a static file in the simple cases or we can pull the inventory from remote sources, such as cloud providers.

Modules: Units of code that Ansible sends to the remote nodes for execution.

Tasks: Units of action that combine a module and its arguments along with some other parameters.

Playbooks: An ordered list of tasks along with its necessary parameters that define a recipe to configure a system.

Roles: Redistributable units of organization that allow users to share automation code easier.

YAML: A popular and simple data format that is very clean and understandable by humans.

How to install Ansible

To start using Ansible, you will need to install it on a control node, this could be your laptop for example. From this control node, Ansible will connect and manage other machines and orchestrate different tasks.

Installation Requirements

Your control node can be any machine with Python 3.8 or newer, but Windows is not supported.

For the managed nodes, Ansible needs to communicate with them over SSH and SFTP (this can also be switched to SCP via the ansible.cfg file) or WinRM for Windows hosts. The managed nodes also need Python 2 (version 2.6 or later) or Python (version 3.5 or later) and in the case of Windows nodes PowerShell 3.0 or later and at least .NET 4.0 installed.

The exact installation procedure depends on your machine and operating system but the most common way would be to use pip.

To install pip, in case it’s not already available on your system:

$ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
$ python get-pip.py --user

After pip is installed:

$ python -m pip install --user ansible

Since there are different ways to install it for every operating system you can also have a look here to find the official suggested way for your environment. For example, check this guide for Ubuntu.

You can test on your terminal if it’s successfully installed by running:

$ ansible --version

Demo requirements

Now that we have Ansible installed, we are going to create our first demo setup. I am going to use my personal laptop as the control node and Vagrant along with VirtualBox to generate locally 2 Ubuntu machines that we will manage with Ansible.

If you wish to follow along, install Vagrant and VirtualBox. You can also find all the files that we are going to be using in this demo in this GitHub repository.

From the top of this GitHub repository execute:

$ vagrant up

This command will spin up 2 Ubuntu hosts in VirtualBox so that we can use them as our managed hosts in this demo exercise. You can also open VirtualBox to verify the existence of your 2 virtual machines.

Finally, to get the info necessary to build our hosts file run the vagrant ssh-config command:

$ vagrant ssh-config
Host host1
  HostName 127.0.0.1
  User vagrant
  Port 2222
  UserKnownHostsFile /dev/null
  StrictHostKeyChecking no
  PasswordAuthentication no
  IdentityFile /Users/ioannis/Desktop/blog/ansible_intro/.vagrant/machines/host1/virtualbox/private_key
  IdentitiesOnly yes
  LogLevel FATAL

Host host2
  HostName 127.0.0.1
  User vagrant
  Port 2200
  UserKnownHostsFile /dev/null
  StrictHostKeyChecking no
  PasswordAuthentication no
  IdentityFile /Users/ioannis/Desktop/blog/ansible_intro/.vagrant/machines/host2/virtualbox/private_key
  IdentitiesOnly yes
  LogLevel FATAL

Ansible Inventory

As previously mentioned, the inventory is the collection of the machines that we would like to manage. Usually, the default location for inventory is /etc/ansible/hosts but we can also define a custom one in any directory.

In the GitHub repository you will see a file named hosts that looks like this:

host1 ansible_host=127.0.0.1 ansible_user=vagrant ansible_port=2222 ansible_ssh_private_key_file=./.vagrant/machines/host1/virtualbox/private_key

host2 ansible_host=127.0.0.1 ansible_user=vagrant ansible_port=2200 ansible_ssh_private_key_file=./.vagrant/machines/host2/virtualbox/private_key

We used the information acquired by Vagrant to populate our hosts file. Currently, our hosts file contains only 2 entries for the hosts that we want to manage.

The host1 and host2 are the aliases we used to name them.

We specified some variables, such as the host, user, and SSH connection parameters necessary to connect to our managed nodes. Here’s a full list of inventory parameters that we can configure per host.

We will keep this inventory simple for the time being, but check out this guide to explore other inventory options such as creating host groups, adding ranges of hosts, and grouping variables.

For example, we can define groups of hosts like this:

[webservers]
webserver1.example.com
webserver2.example.com
webserver3.example.com
192.0.6.45

[databases]
database1.example.com
database2.example.com

In the above example, we defined two groups of hosts, webservers and databases.

Two special groups always exist by default; all that includes every host and ungrouped that includes all the hosts that aren’t in any groups.

Ansible ad hoc commands

Using ad hoc commands is a quick way to run a single task on one or more managed nodes.

Some examples of valid use cases are rebooting servers, copying files, checking connection status, managing packages, gathering facts, etc.

The pattern for ad hoc commands looks like this:

$ ansible [host-pattern] -m [module] -a “[module options]”

host-pattern: the managed hosts to run against

-m: the module to run

-a: the list of arguments required by the module

This is a good opportunity to use our first Ansible ad hoc command and at the same time validate that our inventory is configured as expected. Let’s go ahead and execute a ping command against all our hosts:

$ ansible -i hosts all -m ping

host1 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    },
    "changed": false,
    "ping": "pong"
}

host2 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    },
    "changed": false,
    "ping": "pong"
}

Nice, seems like we can successfully ping the 2 hosts that we have defined in our hosts file.

Next, run a live command only to the host2 node by using the --limit flag

$ ansible all -i hosts --limit host2 -a "/bin/echo hello"

host2 | CHANGED | rc=0 >>
hello

Another example would be to copy a file to our remote nodes:

$ ansible all -i hosts -m ansible.builtin.copy -a "src=./hosts dest=/tmp/hosts"
host1 | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    },
    "changed": true,
    "checksum": "0bd8efb12ac716fdddf6dd8feedb750a7fc8c370",
    "dest": "/tmp/hosts",
    "gid": 1000,
    "group": "vagrant",
    "md5sum": "f425732ff83fe576b00f37dd63d94544",
    "mode": "0664",
    "owner": "vagrant",
    "size": 291,
    "src": "/home/vagrant/.ansible/tmp/ansible-tmp-1645033325.338188-14454-35489936437202/source",
    "state": "file",
    "uid": 1000
}
host2 | CHANGED => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    },
    "changed": true,
    "checksum": "0bd8efb12ac716fdddf6dd8feedb750a7fc8c370",
    "dest": "/tmp/hosts",
    "gid": 1000,
    "group": "vagrant",
    "md5sum": "f425732ff83fe576b00f37dd63d94544",
    "mode": "0664",
    "owner": "vagrant",
    "size": 291,
    "src": "/home/vagrant/.ansible/tmp/ansible-tmp-1645033325.356349-14456-242443746329447/source",
    "state": "file",
    "uid": 1000
}

Perfect, the files have been copied! We can verify this ourselves by “sshing” into one of the managed nodes:

$ vagrant ssh host1
vagrant@vagrant:~$ cat /tmp/hosts

host1 ansible_host=127.0.0.1 ansible_user=vagrant ansible_port=2222 ansible_ssh_private_key_file=./.vagrant/machines/host1/virtualbox/private_key
host2 ansible_host=127.0.0.1 ansible_user=vagrant ansible_port=2200 ansible_ssh_private_key_file=./.vagrant/machines/host2/virtualbox/private_key

Most modules of Ansible are idempotent, which implies that the changes are applied only if needed.

If we try to run a command with an idempotent module, such as copy, we will see that the second time since the file already exists the tasks succeed without performing any actions.

Notice the different colors (green indicates no actions) of the command output after we execute the same command a second time:

If you would like to get more information about Ansible ad hoc commands, check out the Intro to ad hoc commands official user guide.

Intro to Ansible Playbooks

Playbooks are the simplest way in Ansible to automate repeating tasks in the form of reusable and consistent configuration files. Playbooks are defined in YAML files and contain any ordered set of steps to be executed on our managed nodes.

As mentioned, tasks in a playbook are executed from top to bottom. At a minimum, a playbook should define the managed nodes to target and some tasks to run against them.

In playbooks, data elements at the same level must share the same indentation while items that are children of other items must be indented more than their parents.

Let’s look at a simple playbook to get an idea of how that looks in practice.

For the needs of this demo, we will use a simple playbook that runs against all hosts and copies a file, creates a user, and upgrades all apt packages on the remote machines.

---
- name: Intro to Ansible Playbooks
  hosts: all

  tasks:
  - name: Copy file hosts with permissions
    ansible.builtin.copy:
      src: ./hosts
      dest: /tmp/hosts_backup
      mode: '0644'
  - name: Add the user 'bob'
    ansible.builtin.user:
      name: bob
    become: yes
    become_method: sudo
  - name: Upgrade all apt packages
    apt:
      force_apt_get: yes
      upgrade: dist
    become: yes

On the top section, we define the group of hosts on which to run the playbook and its name. After that, we define a list of tasks. Each of the tasks contains some information about the task and the module to be executed along with the necessary arguments.

To avoid specifying the location of our inventory file every time we can define this via a configuration file (ansible.cfg). To find out more about Ansible configuration options check here.

[defaults]
inventory=./hosts

You can validate that this works as expected by running the ansible-inventory command.

ansible-inventory --list
{
    "_meta": {
        "hostvars": {
            "host1": {
                "ansible_host": "127.0.0.1",
                "ansible_port": 2222,
                "ansible_ssh_private_key_file": "./.vagrant/machines/host1/virtualbox/private_key",
                "ansible_user": "vagrant"
            },
            "host2": {
                "ansible_host": "127.0.0.1",
                "ansible_port": 2200,
                "ansible_ssh_private_key_file": "./.vagrant/machines/host2/virtualbox/private_key",
                "ansible_user": "vagrant"
            }
        }
    },
    "all": {
        "children": [
            "ungrouped"
        ]
    },
    "ungrouped": {
        "hosts": [
            "host1",
            "host2"
        ]
    }
}

Now we are ready to run our first playbook using the ansible-playbook command.

$ ansible-playbook intro_playbook.yml
 ___________________________________
< PLAY [Intro to Ansible Playbooks] >
 -----------------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

 ________________________
< TASK [Gathering Facts] >
 ------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

ok: [host2]
ok: [host1]
 _____________________________________________
< TASK [Copy file with owner and permissions] >
 ---------------------------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

changed: [host1]
changed: [host2]
 ___________________________
< TASK [Add the user 'bob'] >
 ---------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

changed: [host1]
changed: [host2]
 _________________________________
< TASK [Upgrade all apt packages] >
 ---------------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

[WARNING]: Updating cache and auto-installing missing dependency: python-apt
changed: [host2]
changed: [host1]
 ____________
< PLAY RECAP >
 ------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

host1                      : ok=4    changed=3    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
host2                      : ok=4    changed=3    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Our playbook has been executed successfully and we can follow the ordered execution of the tasks per host by checking the command output. At the bottom, a summary of the playbook execution is provided by Ansible.

Something that you might find useful at times, is to validate the syntax of a playbook with the flag --syntax-check.

Another handy option is to use the -C flag to perform a dry run of the playbook’s execution. This option doesn’t actually make any changes but it just reports the changes that will happen during a real run.

The official Working with playbooks user guide includes many more details and options about playbooks so make sure to read it when you start moving into more advanced use cases.

Using Variables in Playbooks

Variables can be defined in Ansible at more than one level and Ansible chooses the variable to use based on variable precedence.

Let’s see how we can use variables at the playbook level.

The most common method is to use a vars block at the beginning of each playbook. After declaring them, we can use them in tasks. Use **{{ variable_name }}** to reference a variable in a task.

---
- name: Variables playbook
  hosts: all
  vars:
      state: latest
      user: bob
  tasks:
  - name: Add the user {{ user }}
    ansible.builtin.user:
      name: "{{ user }}"
  - name: Upgrade all apt packages
    apt:
      force_apt_get: yes
      upgrade: dist
  - name: Install the {{ state }} of package "nginx"
    apt:
      name: "nginx"
      state: "{{ state }}"

In the above example, we have used the variables user and state. When referencing a variable as another variable’s value, we must add quotes around the value as shown in our example.

During the playbook run below, we see that the variable’s substitution happens successfully.

Take a look at the Using Variables official user guide to learn more about advanced use cases of their usage in Ansible.

Key Points
In this article, we explored Ansible’s basic concepts, features, and functionality while we also explained why it is such a great tool for automation purposes.

Even more, we configured an Ansible demo setup with examples on how to create an inventory, execute ad hoc commands, write and run simple playbooks with variables.

Thank you for reading and I hope you enjoyed this "Intro to Ansible" article as much as I did.

Terraform Best Practices for Better Infrastructure Management

Ioannis Moustakis — Sun, 15 May 2022 14:39:53 +0000

In this article, we explore best practices for managing Infrastructure as Code (IaC) with Terraform. Terraform is one of the most used tools in the IaC space that enables us to safely and predictably apply changes to our infrastructure.

Starting with Terraform can feel like an intimidating task at first, but a beginner can quickly reach a basic understanding of the tool. After the initial learning period, a new user can then start running commands, creating, and refactoring Terraform code. During this process, many new users face nuances and issues around how to structure their code correctly, use advanced features, apply software development best practices in their IaC process, etc.

Let’s check together some best practices that will assist you in pushing your Terraform skills to the next level. If you are entirely new to Terraform, look at the Terraform Spacelift Blog, where you can find a plethora of material, tutorials, and examples to boost your skills.

Terraform Key Concepts

In this section, we will describe some key Terraform concepts briefly. If you are already familiar with these, you can skip this section.

Terraform Configuration Language

Terraform uses its own configuration language to declare infrastructure objects and their associations. The goal of this language is to be declarative and describe the system’s state that we want to reach.

Resources

Resources represent infrastructure objects and are one of the basic blocks of the Terraform language.

Data Sources

Data sources feed our Terraform configurations with external data or data defined by separate Terraform projects.

Modules

Modules help us group several resources and are the primary way to package resources in Terraform for reusability purposes.

State

Terraform keeps the information about the state of our infrastructure to store track mappings to our live infrastructure and metadata, create plans and apply new changes.

Providers

To interact with resources on cloud providers, Terraform uses some plugins named providers.

IaC Best Practices

Before moving to Terraform, first, let’s check some fundamental best practices that apply to all Infrastructure as Code projects. These should be used in your processes regardless of your tool to manage your cloud infrastructure.

1. Use version control and prevent manual changes

This seems like an obvious statement in 2022, but it’s the basis of everything else. We should treat our infrastructure configurations as application code and apply the same best practices for managing, testing, reviewing, and bringing it to production. We should embrace a GitOps approach that fits our use case and implement an automated CI/CD workflow for applying changes.

2. Shift your culture to Collaborative IaC

Keeping your infrastructure in version-controlled repositories is the first step to improving your manual infrastructure management. Next, we should strive to enable usage across teams with self-service infrastructure, apply policies and compliance according to our organization’s standards, and access relevant insights and information. Thankfully, Spacelift can assist you along the way to achieving these.

How to Structure Your Terraform Projects

This section will explore some strategies for structuring our Terraform projects. In the world of Terraform, there is no right or wrong way to structure our configurations, and most of the suggested structures that you will find online are heavily opinionated.

When deciding how to set up your Terraform configuration, the most important thing is to understand your infrastructure needs and your use case and craft a solution that fits your team and the project.

If we are dealing with a small project with limited infrastructure components, it’s not a bad idea to keep our Terraform configuration as simple as possible. In these cases, we can configure only the necessary files for our root module, which are the configuration files that exist in the root directory. A small project can contain just these files main.tf, variables.tf, README.md. Some other files that you might find handy to use are the outputs.tf to define the output values of your project, versions.tf to gather any pinned versions for the configurations, and providers.tf to configure options related to the providers you use, especially if there are multiple.

Our primary entry point is main.tf, and in simple use cases, we can add all our resources there. We define our variables in variables.tf and assign values to them in terraform.tfvars. We use the file outputs.tf to declare output values. You can find a similar example project structure here.

When dealing with larger projects, things get a bit more complicated, and we have to take a step back to figure out the best structure for our projects.

We first have to break down our Terraform code into reusable components that abstract details from consumers and different teams can use and customize accordingly. We can achieve this by creating separate modules for pieces of infrastructure that should be reused in different environments, projects, and teams.

A common practice is to separate our modules according to ownership and responsibility, rate of change, and ease of management. For every module, we need to define its inputs and outputs and document them thoroughly to enable consumers to use them effectively. We can then leverage outputs and terraform_remote_state to reference values across modules or even different Terraform states. Beware that using the terraform_remote_state data source implies access to the entire state snapshot and this might post a security issue. Another option to share parameters between different states is to leverage an external tool for publishing and consuming the data like Amazon SSM Parameter Store or HashiCorp Consul.

We have to make the next decision to either keep all our Terraform code in a single repository (monorepo) or to separate our Terraform configurations in multiple code repositories. This is considered a great debate since both approaches have drawbacks and benefits. There is a tendency in the industry to avoid gigantic monorepos and use separate configurations to enable faster module development and flexibility. Personally, this is the approach I prefer too.

Usually, we have to deal with a plethora of different Infrastructure environments, and there are multiple ways to handle this in Terraform. A good and easy practice to follow is to have separate Terraform configurations for different environments. This way, different environments have their own state and can be tested and managed separately, while shared behavior is achieved with shared or remote modules.

One option is using a separate directory per environment and keeping a separate state for each directory. Another option would be to keep all the Terraform configurations in the same directory and pass different environment variables per environment to parametrize the configuration accordingly. Check out the Evolving Your Infrastructure with Terraform and How I Manage More Environments with Less Code in Terraform talks for some inspiration around structuring your projects.

Here, you can find an example structure for three different environments per directory: production, staging, and test. Each environment has its own state and is managed separately from the others while leveraging common or shared modules. Although this approach comes with some code duplication, we gain improved clarity, environment isolation, and scalability.

As a general rule, we want to define Terraform configurations with limited scope and blast radius with specific owners. To minimize risk, we should try to decompose our projects into small workspaces/stacks and segment access to them using Role-based access control(RBAC).

Terraform Specific Best Practices

Alright, in the previous sections, we talked about some generic IaC best practices. We explored some options for optimizing our Terraform code according to our organizational structure and needs.

In this part, we are deep-diving into specific points that will take our Terraform code to the next level. This list isn’t exhaustive, and some parts are opinionated based on my personal preferences and experiences.

The goal here is to give you hints and guidance on experimenting, researching, and implementing the practices that make sense to your use case.

1. Remote state

It’s ok to use the local state when experimenting, but use a remote shared state location for anything above that point. Having a single remote backend for your state is considered one of the first best practices you should adopt when working in a team. Pick one that supports state locking to avoid multiple people changing the state simultaneously. Treat your state as immutable and avoid manual state changes at all costs. Make sure you have backups of your state that you can use in case of a disaster. For some backends, like AWS S3, you can enable versioning to allow for quick and easy state recovery.

2. Use existing shared and community modules

Instead of writing your own modules for everything and reinventing the wheel, check if there is already a module for your use case. This way, you can save time and harness the power of the Terraform community. If you feel like it, you can also help the community by improving them or reporting issues. You can check the Terraform Registry for available modules.

3. Import existing infrastructure

If you inherited a project that is a couple of years old, chances are that some parts of its infrastructure were created manually. Fear not, you can import existing infrastructure into Terraform and avoid managing infrastructure from multiple endpoints.

4. Avoid variables hard-coding

It might be tempting to hardcode some values here and there but try to avoid this as much as possible. Take a moment to think if the value you are assigning directly would make more sense to be defined as a variable to facilitate changes in the future. Even more, check if you can get the value of an attribute via a data source instead of setting it explicitly. For example, instead of finding our AWS account id from the console and setting it in terraform.tfvars as

aws_account_id=”99999999999”

We can get it from a data source.

data "aws_caller_identity" "current" {}

locals {
    account_id    = data.aws_caller_identity.current.account_id
}

5. Always format and validate

In IaC, consistency is essential long-term, and Terraform provides us with some tools to help us in this quest. Remember to run terraform fmt and terraform validate to properly format your code and catch any issues that you missed. Ideally, this should be done auto-magically via a CI/CD pipeline or pre-commit hooks.

6. Use a consistent naming convention

You can find online many suggestions for naming conventions for your Terraform code. The most important thing isn’t the rules themselves but finding a convention that your team is comfortable with and trying collectively to be consistent with it. If you need some guidance, here’s a list of rules that are easy to follow:

Use underscores(_) as a separator and lowercase letters in names.
Try not to repeat the resource type in the resource name.
For single-value variables and attributes, use singular nouns. For lists or maps, use plural nouns to show that it represents multiple values.
Always use descriptive names for variables and outputs, and remember to include a description.

7. Tag your Resources

A robust and consistent tagging strategy will help you tremendously when issues arise or trying to figure out which part of your infrastructure exploded your cloud vendor’s bill. You can also craft some nifty access control policies based on tags when needed. Like when defining naming conventions, try to be consistent and always tag your resources accordingly.

The Terraform argument tags should be declared as the last argument(only depends_on or lifecycle arguments should be defined after tags if relevant). A handy option to help you with tagging is defining some default_tags that apply to all resources managed by a provider. Check out this example to see how to set and override default tags for the AWS provider. If the provider you use doesn’t support default tags, you must manually pass these tags through to your modules and apply them to your resources.

8. Introduce Policy as Code

As our teams and infrastructure scale, our trust in individual users is generally reduced. We should set up some policies to ensure our systems continue to be operational and secure. Having a Policy as Code process in place allows us to define the rules of what is considered secure and acceptable at scale and automatically verify these rules. Spacelift leverages an open-source engine to achieve this, Open Policy Agent(OPA).

9. Implement a Secrets Management Strategy

Usually, users won’t admit that they have secrets in their Terraform code, but we have all been there. When you are starting with Terraform, it’s normal that secret management isn’t your top priority, but eventually, you will have to define a strategy for handling secrets.

As you probably heard already, never store secrets in plaintext and commit them in your version control system. One technique that you can use is to pass secrets by setting environment variables with TF_VAR and marking your sensitive variables with sensitive = true.

A more mature solution would be to set up a secret store like Hashicorp Vault or AWS Secrets Manager to handle access to secrets for you. This way, you can protect your secrets at rest and enforce encryption without too much trouble. You can also opt for more advanced features like secret rotation and audit logs. Beware that this approach usually comes with the cost of using this managed service.

10. Test your Terraform code

As with all other code, IaC code should be tested properly. There are different approaches here, and again, you should find one that makes sense for you. Running terraform plan is the easiest way to verify if your changes will work as expected quickly. Next, you can perform some static analysis for your Terraform code without the need to apply it. Unit testing is also an option to verify the normal operation of distinct parts of your system.

Another step would be to integrate a Terraform linter to your CI/CD pipelines and try to catch any possible errors related to Cloud Providers, deprecated syntax, enforce best practices, etc. One step ahead, you can set up some integration tests by spinning up a replica sandbox environment, applying your plan there, verifying that everything works as expected, collecting results, destroying the sandbox, and moving forward by applying it to production.

There are a lot of tools out there that can help you with testing your Terraform code. I will list some of them in the “Helper Tools” section below.

11. Enable debug/troubleshooting

When issues arise, we have to be quick and effective in gathering all the necessary information to solve them. You might find it helpful to set the Terraform log level to debug in these cases.

TF_LOG=DEBUG <terraform command>

Another thing that you might find helpful is to persist logs in a file by setting the TF_LOG_PATH environment variable. Check out this Terraform Debug & Troubleshoot tutorial for some hands-on examples.

12. Leverage Helper tools to make your life easier

Terraform is one of the most used and loved IaC tools out there, and naturally, there is a big community around it. Along with this community, many helper tools are being constantly created to help us through our Terraform journey. Picking up and adopting the right tools for our workflows isn’t always straightforward and usually involves an experimentation phase. Here you can find a list of tools that I find helpful based on my experience, but it is definitely not an exhaustive list.

tflint – Terraform linter for errors that the plan can’t catch.
tfenv – Terraform version manager
checkov – Terraform static analysis tool
terratest – Go library that helps you with automated tests for Terraform
pre-commit-terraform – Pre-commit git hooks for automation
terraform-docs – Quickly generate docs from modules
spacelift – Collaborative Infrastructure Delivery Platform
atlantis – Workflow for collaborating on Terraform projects
terraform-cost-estimation – Free cost estimation service for your plans. Check out also this list that includes a lot more awesome terraform tools.

Key Points

We have explored many different best practices for Terraform and Infrastructure as Code, analyzed various options for handling and structuring our Terraform projects, and saw how adopting helper tools could make our life easier.

Remember, this isn’t a recipe that you have to follow blindly but a guide that aims to provide pointers and cues and trigger you to build your own optimal Terraform workflows and projects.

Thank you for reading, and I hope you enjoyed this “Terraform Best Practices” article as much as I did.