Forem: Omkar Halankar

Atlantis : The DevOps Tool You Need for Cloud Automation

Omkar Halankar — Sun, 26 Mar 2023 08:15:34 +0000

Atlantis is actually a DevOps tool that helps organizations to automate their infrastructure management and deployment processes. It is an open-source tool that streamlines the management of infrastructure as code (IaC) and helps teams collaborate more effectively on large-scale infrastructure projects.

At its core, Atlantis is designed to work with Terraform, a popular infrastructure provisioning tool. It provides a centralized workflow for managing Terraform configuration files, allowing teams to review and approve changes before they are applied to production environments.

One of the key benefits of Atlantis is that it helps to reduce the risk of errors and misconfigurations in infrastructure deployments. By providing a standardized workflow and approval process, it ensures that all changes are thoroughly tested and reviewed before they are applied.

Additionally, Atlantis integrates with many popular DevOps tools, including GitHub, Bitbucket, and GitLab, making it easy to incorporate into existing workflows. It also provides detailed reporting and notifications, allowing teams to stay on top of infrastructure changes and quickly identify any issues that arise.

Overall, Atlantis is a powerful tool for organizations looking to streamline their infrastructure management and deployment processes. By automating many of the tedious and error-prone tasks associated with infrastructure management, it allows teams to focus on delivering high-quality products and services to their customers.

Getting started with Atlantis...

**
To read more about Atlantis go to https://www.runatlantis.io/.

With Atlantis, developers are able to write and apply Terraform safely. They submit pull requests, can run atlantis plan until their change looks good and then get approval from Ops to apply.

Installation Guide
There are multiple ways to install/deploy Atlantis ": Installation Guide

Helm-chart seems pretty straight forward to be defined in values.yml https://www.runatlantis.io/docs/deployment.html#kubernetes-helm-chart

Based on the Version Control System : Github / Enterprise , GitLab, .. setup the Webhook as described in WebHook Configuration

Atlantis Configuration
Three methods for configuring Atlantis:

Passing flags to the atlantis server command
Creating a server-side repo config file and using the --repo-config flag
Placing an atlantis.yaml file at the root of your Terraform repositories

Details on Configuring Atlantis

Workflow Hooks
As part of Workflow setup:
Pre Workflow Hooks
Post Workflow hooks

For Atlantis also supports Terragrunt apart from Terraform but not natively.
Atlantis supports with help of Custom Worksflows

Using-Atlantis

Once Atlantis is Installed, Configured and Workflows are in-place Atlantis will be in action with supported commands Atlantis in Action

How Atlantis Works

There are different aspects that need to be understood in detail while using Atlantis

Locking the environment/workspace while provisioning infra in collaboration
Autoplan how the infrastructure changes should be configured with any change in repo.
Automerge how tge changes should be merged with successful PR plan.

More Details

More robust setup

with the above deployment and configuration, you are all set to use Atlantis but there are few more things that can be put in place for a better and a safer utilization of Atlantis.

The atlantis.yaml file actually requires Project setup which has to be modified manually in case any folders are added ex. in GitHub which has to be monitored by Atlantis. If there are few resources to be provisioned and monitored than this looks fine to be managed manually but in most of the cases provisioning is not limited and anything that involves manual intervention is prone to error.

This can be automated by using another open-source tool by Transcend.io terragrunt-atlantis-config

This generates the project section automatically based on the changes done and raised during PR.

Adding Code Analysis Tool

Can we ever imagine sitting back and manually reading each line of code to find flaws? To ease our work, several types of static analysis tools are available in the market which helps to analyze the code during the development and detect fatal defects early in the SDLC phase.

Some tools that have been proven to be the best-fit in IAC for Terraform are:

The good thing about these Static Code Analysis tools are they can be used in conjunction with each other which increases the code coverage perimeter.

Pre-commit Hooks for Terraform

Enforcing Terraform Linting and Security Standards Using Pre-Commit Hooks

We can use Pre-Commit Hooks as an on-demand tool to integrate and use these tools before checking code into source control.

Installation, Hooks, Usage all described in detail at https://github.com/antonbabenko/pre-commit-terraform#readme

Datree - Automate quality checks for Kubernetes YAMLs

Omkar Halankar — Mon, 16 Jan 2023 04:56:20 +0000

So now-a-days we hear yet another fancy term ... Gitops.
In simple terms Git is used as a single source of truth for applications and infrastructure. So the team can focus on Application development rather than Operational Tasks (Ops).

GitOps provides features like

CI/CD Automation pipelines
Entire Audit History - Version Control, History, Peer Review and Rollback.
Git serves as Single Source of Truth
Webhooks to trigger pipelines
PR driven approach

So now as part of GitOps where the entire Industry is moving towards, we'll have to update the YAML manifests for Kubernetes and then go ahead with the deployment to specific environment - Dev , Stage , Preprod , Production as the situation demands.

Any changes to the Manifest files will raise questions about its success/failure. With human intervention, the process cannot be error-free. And the important question here would be "Can a process be put in place to avoid pushing the changes to next level ?"

And the answer to the above question would be "Yes!".
There is a way to validate the YAML manifest files before applying them in any region post modification.

What's special about Datree tool

Datree provides possibilities to avoid passing any Kubernetes Misconfigurations from reaching Production with its automated policy checks in the CI pipeline.
Its Open-Source CLI tool that enables us to write Manifest files that meet project standards or follow best practices.
CLI runs the policy check on your system, your files and their contents are not transferred to their backend. To their backend, which is used to show your policy check history on your dashboard, the tool only sends metadata.
It has an offline mode as well. That means Datree also does not need to be connected to the cluster.

Installation and general usage

https://hub.datree.io/cli/getting-started#1-install-the-datree-cli

Just run the following command:

curl https://get.datree.io | /bin/bash

Verification post installation To verify the installation process has been successful, run the below command:

datree test ~/.datree/k8s-demo.yaml

When Datree usually starts its check , it goes through 3 main stages:

YAML validation;
checking Kubernetes charts;
checking Kubernetes policies.

The output will be similar to:

$ datree test ~/.datree/k8s-demo.yaml
>>  File: .datree/k8s-demo.yaml

[V] YAML validation
[V] Kubernetes schema validation

[X] Policy check

❌  Ensure each container image has a pinned (tag) version  [1 occurrence]
    - metadata.name: rss-site (kind: Deployment)
💡  Incorrect value for key `image` - specify an image version to avoid unpleasant "version surprises" in the future

❌  Ensure each container has a configured liveness probe  [1 occurrence]
    - metadata.name: rss-site (kind: Deployment)
💡  Missing property object `livenessProbe` - add a properly configured livenessProbe to catch possible deadlocks

❌  Ensure each container has a configured memory limit  [1 occurrence]
    - metadata.name: rss-site (kind: Deployment)
💡  Missing property object `limits.memory` - value should be within the accepted boundaries recommended by the organization

❌  Ensure workload has valid label values  [1 occurrence]
    - metadata.name: rss-site (kind: Deployment)
💡  Incorrect value for key(s) under `labels` - the vales syntax is not valid so the Kubernetes engine will not accept it


(Summary)

- Passing YAML validation: 1/1

- Passing Kubernetes (1.20.0) schema validation: 1/1

- Passing policy check: 0/1

+-----------------------------------+-----------------------+
| Enabled rules in policy "Default" | 21                    |
| Configs tested against policy     | 1                     |
| Total rules evaluated             | 21                    |
| Total rules skipped               | 0                     |
| Total rules failed                | 4                     |
| Total rules passed                | 17                    |
| See all rules in policy           | https://app.datree.io |
+-----------------------------------+-----------------------+

The detailed information that is made available helps know about the violations in the manifest. This also proves to be helpful while fixing the violations.

Each Datree policy check is performed using the default policy, which includes 50+ built-in rules.

CI Intgration

Datree helps in the Left Shift movement. The sooner the errors are identified in the CICD cycle its better for the application life cycle.

To integrate Datree into CI/CD, you can follow the example below. You need to follow these steps:

Get your account token (you can find it in the dashboard's Settings).
Set DATREE_TOKEN as secret/environment variable
Add Datree to your CI flow via token as shown (i.e., for GitHub).

GitHub Repo with Sample Application

Below is an example of Datree Integration with GitHub Actions

# Controls when the workflow will run
on:
  # Triggers the workflow on push or pull request events but only for the main branch
  push:
    branches: [ feature* ]
#   pull_request:
#     branches: [ main ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

env:
  DATREE_TOKEN: ${{ secrets.DATREE_TOKEN }} 

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
      - uses: actions/checkout@v2

      - name: Install Datree
        run: curl https://get.datree.io | /bin/bash

      - name: Test k8s file
        run: datree test deployment/deployment.yaml --no-record --ignore-missing-schemas
        continue-on-error: false

You can install VSCode Extension locally