Forem: Ekin Öcalan

WILw Reading Software Architecture: Cohesion

Ekin Öcalan — Sun, 31 Jan 2021 21:32:49 +0000

In a software module, cohesion defines how well the individual parts are related to each other. When those parts are highly connected, then we can say that module is highly cohesive. If they're not operating or depending on each other, then the module has low cohesion. It is also significantly linked to coupling, as can be seen from a quote by Larry Constantine who developed those concepts:

Attempting to divide a cohesive module would only result in increased coupling and decreased readability.

Cohesion types from worst to best

When we're talking about cohesion, we generally prefer high cohesion (and low coupling). However, it's also true that not all types of cohesion are benefiting the architecture and the codebase. It's a good practice to investigate how to achieve better cohesion and high cohesion — although it depends on a case-by-case basis.

To understand cohesion, we can benefit from Object-Oriented Programming and borrow the Class concept as our module example.

Coincidental cohesion — the worst

Functions in a class are not related at all. They've just been grouped arbitrarily like in a Utility class.

Logical cohesion

Class functions are logically related, although operationally different. Think of a class for sending a notification, for example. One function handles push notifications, the next one sends emails, and the third sends text messages. They're logically doing the same thing (sending notifications), but what they do is entirely different.

Temporal cohesion

Functions in a class are related based on when they're run. Whenever an exception occurs in one of the functions, another function is called to do the cleanup. They're doing two different things, so they're only temporally cohesive.

Procedural cohesion

If you can properly run a specific function only after a different function is run, then those two functions are procedurally cohesive. Think of a function that is checking user permissions. Only if the user is authorized, the target function is run. Then these two functions have procedural cohesion.

Communicational cohesion

When two functions are operating on the same set of data, they're communicationally cohesive. Let's say we have a class that is handling the payments. One of the functions writes the successful payment information to the database. Then the other function reads that data to start the shipping process. In this case, these two functions have communicational cohesion.

Sequential cohesion

If one function's output is another one's input, then those two functions are sequentially cohesive. Think of a function that is creating a user profile. It accepts certain user information as well as a username. Instead of asking the user for a username, another function creates a slug from the user's first and last names. In this case, these two functions have sequential cohesion.

Functional cohesion

When functions of a class all contribute to a single task, then they are functionally cohesive. Let's say we have a class implementing the builder pattern. The class is named CarBuilder, and all of its functions are used to build a car. The class has functional cohesion.

Atomic cohesion — the best

Although functional cohesion can also be defined as the perfect cohesion, atomic cohesion moves the needle one more step. When the class achieves functional cohesion and doesn't have anything besides the bare essentials, it has atomic cohesion.

Cover photo by Boba Jaglicic

[1] WILw: What I Learned while
[2] Types of Cohesion

What I've Learned Learning Terraform: Part 8

Ekin Öcalan — Mon, 30 Nov 2020 13:18:13 +0000

Terraform Series

Part 1: Introduction
Part 2: Creating the Server
Part 3: Provisioning the Server
Part 4: Managing Terraform State
Part 5: Cleaner Code with Terraform Modules
Part 6: Loops with Terraform
Part 7: Conditionals with Terraform
Part 8: Testing Terraform Code

We conclude the series by putting a cherry on top: Testing Terraform code.

Manual Testing vs. Automated Testing

Testing is a crucial part of CI/CD cycles. Since the DevOps movement is fundamentally for delivering software, what could be more important than testing infrastructure-as-code?

Here we are going to mention two strategies. Manual testing will help us test our infrastructure code by running it on the cloud infrastructure. Automated testing will allow us to automate these manual tests by using a Go-based library: Terratest.

One thing to keep in mind is that there is no local environment for testing Terraform code since it's mostly dependent on third party services like AWS, GCP, DigitalOcean, etc. That's why there is always be resource creation & deletion in the cloud.

Manual Testing

The following statement might seem like a no-brainer, but we've already done many manual testing throughout this series. Running terraform plan and terraform apply consecutively were merely manual tests. We write the code, and then we apply it to the cloud. If everything goes well and the changes are made, our manual testing was a success.

There is a rule of thumb to make our codes more testable, though. Take a look at this resource creation example from Part 2 below. Its purpose is to declare a droplet (server):

resource "digitalocean_droplet" "web" {
  image  = "ubuntu-20-04-x64"
  name   = "terraform-sandbox"
  region = "ams3"
  size   = "s-1vcpu-1gb"
}

The declarative code we see above is very much in a frozen state. No matter how you want to test it, it's going to create a single CPU machine with 1 GB of RAM running Ubuntu 20.04 in Amsterdam's AMS3 region. However, it would be better if we could test this piece of code with different inputs.

For that, we could apply our knowledge from Part 5 and make them variables like below. Thus, we'll be able to test our code with different inputs manually.

resource "digitalocean_droplet" "web" {
  image  = var.droplet_web_image
  name   = var.droplet_web_name
  region = var.droplet_web_region
  size   = var.droplet_web_size
}

variable "var.droplet_web_image" {
  description = "Web droplet's image"
  type        = string
}

variable "var.droplet_web_name" {
  description = "Web droplet's name"
  type        = string
}

variable "var.droplet_web_region" {
  description = "Web droplet's region"
  type        = string
}

variable "var.droplet_web_size" {
  description = "Web droplet's size"
  type        = string
}

Note that it's a better idea to encapsulate your values in a different file, as we talked about in Part 5. The example above is only a simplified version.

Automated Testing

Manual testing is a useful and important part of testing infrastructure-as-code (IaC). However, a test should be run as frequently as possible. That's why an automated test is much more beneficial in the long run.

Automated testing of the Terraform code is nothing but a collection of examples of manual testing. There is an excellent open-source tool called Terratest written in Go, which helps you do exactly that. You write examples for your Terraform code, and then you use Terratest to initialize and apply your IaC. You can learn more about how to start using the tool Terratest via their documentation, but here, I'd like to show you how an example test looks like.

The example below is not directly related to DigitalOcean, because Terratest support for DigitalOcean is still yet to come at the time of this writing. However, you can always use Terratest extensively for generic Terraform code as well as other parts by making use of outputs.

Very briefly from official Terratest docs, let's say we have this output resource in the main.tf file under the examples directory:

output "hello_world" {
  value = "Hello World!"
}

Then, a simple automated test written with Terratest looks like this:

package test

import (
    "testing"

    "github.com/gruntwork-io/terratest/modules/terraform"
    "github.com/stretchr/testify/assert"
)

func TestTerraformHelloWorldExample(t *testing.T) {
    // retryable errors in terraform testing.
    terraformOptions := terraform.WithDefaultRetryableErrors(t, &terraform.Options{
        TerraformDir: "../examples",
    })

    defer terraform.Destroy(t, terraformOptions)

    terraform.InitAndApply(t, terraformOptions)

    output := terraform.Output(t, terraformOptions, "hello_world")
    assert.Equal(t, "Hello World!", output)
}

Terratest follows the same testing guidelines as Go. Hence, it's a good idea to save your test file as *_test.go and name your test functions with Test prefixes like in our example above: TestTerraformHelloWorldExample. Then you can run go test to start your test suite.

Let's go through the test code now:

terraformOptions := terraform.WithDefaultRetryableErrors(t, &terraform.Options{TerraformDir: "../examples",})

Since Terraform is all about dealing with third-party providers, we should also consider network errors. If a test fails on one occasion due to a network error, it doesn't mean it will fail the second time. That's why retrying tests automatically helps a lot. Terratest makes it easy to do that with WithDefaultRetryableErrors. We enable the retrying and also define the directory for our Terraform code above.

defer terraform.Destroy(t, terraformOptions)

While testing the Terraform code, we create real resources on the cloud. When the test is done, we should destroy them to avoid unnecessary costs and future test dependencies. Go's defer keyword here helps us to run this destroy keyword at the end. By starting our code with the destroy directive, we make sure Go will destroy any resources created; because even if we face errors during the test, Go will tear it down since it already has the deferred directive.

terraform.InitAndApply(t, terraformOptions)

Basically, this directive initializes the resources defined in the Terraform code on a temporary directory and then applies it.

output := terraform.Output(t, terraformOptions, "hello_world")
assert.Equal(t, "Hello World!", output)

The last part of our test code is the assertion of the expected and actual values. Our expected output resource is "Hello World!" while we get the actual value directly from the Terraform output by using terraform.Output.

Cover photo by Chris Liverani

Part 7...................................................................................................................

What I've Learned Learning Terraform: Part 7

Ekin Öcalan — Tue, 08 Sep 2020 14:28:46 +0000

Terraform Series

Part 1: Introduction
Part 2: Creating the Server
Part 3: Provisioning the Server
Part 4: Managing Terraform State
Part 5: Cleaner Code with Terraform Modules
Part 6: Loops with Terraform
Part 7: Conditionals with Terraform
Part 8: Testing Terraform Code

We continue to learn more about the basic Terraform features.

Conditionals

We've already used some sort of conditionals in the previous parts of this series. However, it makes sense to categorize the usage of conditionals in Terraform. Just like with the loops, we use a different approach to use conditionals since Terraform is a declarative language.

Conditional expressions

Although we don't have a procedural way of implementing if-else statements, Terraform supports the ternary syntax. Let's say you have two environments: Production and staging. You'd like to have a production environment with more computing power, but you also want to cut costs on staging and have less configuration there. How do you declare your condition?

Let's say you create a variable to determine whether the environment is production or not:

variable "is_production" {
  description = "If the environment is production or not"
  type        = bool
  default     = false
}

output "is_staging" {
  value = ! var.is_production
}

The code above is only a demonstration of a pair of helpers. One of them is a variable, and the other one is an output. To keep the example simple, we are not going to populate the variable programmatically, but make it is_production=false.

Now, let's say you are going to create one server for each environment. The production server will have a more powerful configuration.

resource "digitalocean_droplet" "server" {
  image  = "ubuntu-20-04-x64"
  name   = "example-droplet"
  region = "ams3"
  size   = var.is_production ? "s-8vcpu-32gb" : "s-1vcpu-1gb"
}

Notice how we used the ternary syntax to declare the size of the droplet. Now if you run terraform plan, you'll see that Terraform plans to create a droplet with the size of s-1vcpu-1gb because, by default, it's not the production environment:

Terraform will perform the following actions:

  # digitalocean_droplet.server will be created
  + resource "digitalocean_droplet" "server" {
      + backups              = false
      + created_at           = (known after apply)
      + disk                 = (known after apply)
      + id                   = (known after apply)
      + image                = "ubuntu-20-04-x64"
      + ipv4_address         = (known after apply)
      + ipv4_address_private = (known after apply)
      + ipv6                 = false
      + ipv6_address         = (known after apply)
      + ipv6_address_private = (known after apply)
      + locked               = (known after apply)
      + memory               = (known after apply)
      + monitoring           = false
      + name                 = "example-droplet"
      + price_hourly         = (known after apply)
      + price_monthly        = (known after apply)
      + private_networking   = (known after apply)
      + region               = "ams3"
      + resize_disk          = true
      + size                 = "s-1vcpu-1gb"
      + status               = (known after apply)
      + urn                  = (known after apply)
      + vcpus                = (known after apply)
      + volume_ids           = (known after apply)
      + vpc_uuid             = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Creating resources conditionally

Resources are the backbone of IaC. We declare resources, and Terraform finds a way to create the defined resources. It's easy to make a resource when you know that it's necessary. But what if you don't know if you're going to need that resource?

Moving on from the previous example of production and staging environments, let's move one step up and create a resource conditionally. Add this resource declaration to the file:

resource "digitalocean_domain" "domain" {
  count      = var.is_production ? 1 : 0
  name       = "example-domain.com"
  ip_address = digitalocean_droplet.server.ipv4_address
}

Domain declaration requires name and the IP address attributes. count parameter is the one that allows us to create this resource conditionally. I should mention that this is a hack to simulate a conditional statement on a resource-level. If the environment is production, Terraform will create one domain with this configuration. If it's staging, then it's not going to create at all.

If you run terraform plan, you'll see that nothing changes in the plan because our environment is staging by default:

Terraform will perform the following actions:

  # digitalocean_droplet.server will be created
  + resource "digitalocean_droplet" "server" {
      + backups              = false
      + created_at           = (known after apply)
      + disk                 = (known after apply)
      + id                   = (known after apply)
      + image                = "ubuntu-20-04-x64"
      + ipv4_address         = (known after apply)
      + ipv4_address_private = (known after apply)
      + ipv6                 = false
      + ipv6_address         = (known after apply)
      + ipv6_address_private = (known after apply)
      + locked               = (known after apply)
      + memory               = (known after apply)
      + monitoring           = false
      + name                 = "example-droplet"
      + price_hourly         = (known after apply)
      + price_monthly        = (known after apply)
      + private_networking   = (known after apply)
      + region               = "ams3"
      + resize_disk          = true
      + size                 = "s-1vcpu-1gb"
      + status               = (known after apply)
      + urn                  = (known after apply)
      + vcpus                = (known after apply)
      + volume_ids           = (known after apply)
      + vpc_uuid             = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Now let's go ahead and change the default value for is_production variable to true:

variable "is_production" {
  description = "If the environment is production or not"
  type        = bool
  default     = true
}

If you run the terraform plan command now, you'll see two additional changes on the plan. One for the variable change, and the other for the domain resource creation:

Terraform will perform the following actions:

  # digitalocean_domain.domain[0] will be created
  + resource "digitalocean_domain" "domain" {
      + id         = (known after apply)
      + ip_address = (known after apply)
      + name       = "example-domain.com"
      + urn        = (known after apply)
    }

  # digitalocean_droplet.server will be created
  + resource "digitalocean_droplet" "server" {
      + backups              = false
      + created_at           = (known after apply)
      + disk                 = (known after apply)
      + id                   = (known after apply)
      + image                = "ubuntu-20-04-x64"
      + ipv4_address         = (known after apply)
      + ipv4_address_private = (known after apply)
      + ipv6                 = false
      + ipv6_address         = (known after apply)
      + ipv6_address_private = (known after apply)
      + locked               = (known after apply)
      + memory               = (known after apply)
      + monitoring           = false
      + name                 = "example-droplet"
      + price_hourly         = (known after apply)
      + price_monthly        = (known after apply)
      + private_networking   = (known after apply)
      + region               = "ams3"
      + resize_disk          = true
      + size                 = "s-8vcpu-32gb"
      + status               = (known after apply)
      + urn                  = (known after apply)
      + vcpus                = (known after apply)
      + volume_ids           = (known after apply)
      + vpc_uuid             = (known after apply)
    }

Plan: 2 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  ~ is_staging = true -> false

There is a drawback with this hack, though. As you can see from the plan, the domain resource will be created as part of a list: digitalocean_domain.domain[0]. That's because we use the count parameter within the resource.

By the way, starting from Terraform 0.13, you can do this hack with modules as well.

Setting values conditionally (Interpolation)

There is one more feature for conditionals on Terraform, which is the interpolation.

In computer programming, string interpolation (or variable interpolation, variable substitution, or variable expansion) is the process of evaluating a string literal containing one or more placeholders, yielding a result in which the placeholders are replaced with their corresponding values. 1

Interpolation in Terraform is perhaps the closest thing to a conditional in other programming languages due to its direct if-else statement usage.

Let's demonstrate this by setting the droplet name based on the environment:

resource "digitalocean_droplet" "server" {
  image  = "ubuntu-20-04-x64"
  name   = "%{if var.is_production}${"production-droplet"}%{else}${"staging-droplet"}%{endif}"
  region = "ams3"
  size   = var.is_production ? "s-8vcpu-32gb" : "s-1vcpu-1gb"
}

Notice how we used if-else statements inside the name attribute. You can even use variables here to define your interpolation with placeholders. Running the terraform plan command will give us the plan below:

  # digitalocean_droplet.server will be created
  + resource "digitalocean_droplet" "server" {
      + backups              = false
      + created_at           = (known after apply)
      + disk                 = (known after apply)
      + id                   = (known after apply)
      + image                = "ubuntu-20-04-x64"
      + ipv4_address         = (known after apply)
      + ipv4_address_private = (known after apply)
      + ipv6                 = false
      + ipv6_address         = (known after apply)
      + ipv6_address_private = (known after apply)
      + locked               = (known after apply)
      + memory               = (known after apply)
      + monitoring           = false
      + name                 = "production-droplet"
      + price_hourly         = (known after apply)
      + price_monthly        = (known after apply)
      + private_networking   = (known after apply)
      + region               = "ams3"
      + resize_disk          = true
      + size                 = "s-8vcpu-32gb"
      + status               = (known after apply)
      + urn                  = (known after apply)
      + vcpus                = (known after apply)
      + volume_ids           = (known after apply)
      + vpc_uuid             = (known after apply)
    }

You see that the name is production-droplet since we set our environment to production in the last step. Now you have a conditional value setting.

Cover photo by Mario Dobelmann

Part 6.........................................................................................................Part 8

What I've Learned Learning Terraform: Part 6

Ekin Öcalan — Thu, 27 Aug 2020 10:36:44 +0000

Terraform Series

Part 1: Introduction
Part 2: Creating the Server
Part 3: Provisioning the Server
Part 4: Managing Terraform State
Part 5: Cleaner Code with Terraform Modules
Part 6: Loops with Terraform
Part 7: Conditionals with Terraform
Part 8: Testing Terraform Code

We're going to continue on the same path as we left off. I'm going to do a deep dive into Terraform with the help of some DigitalOcean resources. Instead of focusing on resources, though, I'll explain what we can do with those resources using some of the basic Terraform features.

Loops

The Terraform language is declarative, describing an intended goal rather than the steps to reach that goal. 1

Since we need to declare the end goal in Terraform, it shouldn't come as a surprise when I say that Terraform does not have loops. But there are three different features that we can emulate loops.

`count` parameter to create multiple resources

Let's imagine you want to create multiple instances from the same resource type. How do you achieve that? By copying the resource declaration. What if you need a lot of them? It's inefficient to repeat the same logic on a single resource over and over and over.

Terraform's count parameter is the oldest way of encapsulating repeating the resources. It's got supercharged by Terraform's 0.13 version and is now also available in module blocks as well.

Let's imagine you want to create tags on DigitalOcean, later to be used on droplets. We'd like to tag droplets by the amount of RAM it's provided with.

resource "digitalocean_tag" "ram_8" {
  name = "ram_8"
}

Now we can use this resource for tagging droplets with 8 GB of RAM. But RAM amounts are variable. So we need to create a variety of tags starting from 1 GB and going up until maybe 64 GB. Let's use the count parameter to make all of them.

resource "digitalocean_tag" "ram" {
  count = 64
  name = "ram_${count.index + 1}"
}

When you run terraform plan, you'll see a list of tags that Terraform will create:

Terraform will perform the following actions:

  # digitalocean_tag.ram[0] will be created
  + resource "digitalocean_tag" "ram" {
      + databases_count        = (known after apply)
      + droplets_count         = (known after apply)
      + id                     = (known after apply)
      + images_count           = (known after apply)
      + name                   = "ram_1"
      + total_resource_count   = (known after apply)
      + volume_snapshots_count = (known after apply)
      + volumes_count          = (known after apply)
    }

  # digitalocean_tag.ram[1] will be created
  + resource "digitalocean_tag" "ram" {
      + databases_count        = (known after apply)
      + droplets_count         = (known after apply)
      + id                     = (known after apply)
      + images_count           = (known after apply)
      + name                   = "ram_2"
      + total_resource_count   = (known after apply)
      + volume_snapshots_count = (known after apply)
      + volumes_count          = (known after apply)
    }

  # digitalocean_tag.ram[2] will be created
  + resource "digitalocean_tag" "ram" {
      + databases_count        = (known after apply)
      + droplets_count         = (known after apply)
      + id                     = (known after apply)
      + images_count           = (known after apply)
      + name                   = "ram_3"
      + total_resource_count   = (known after apply)
      + volume_snapshots_count = (known after apply)
      + volumes_count          = (known after apply)
    }

...
...
...

  # digitalocean_tag.ram[63] will be created
  + resource "digitalocean_tag" "ram" {
      + databases_count        = (known after apply)
      + droplets_count         = (known after apply)
      + id                     = (known after apply)
      + images_count           = (known after apply)
      + name                   = "ram_64"
      + total_resource_count   = (known after apply)
      + volume_snapshots_count = (known after apply)
      + volumes_count          = (known after apply)
    }

Plan: 64 to add, 0 to change, 0 to destroy.

The *_count parameters inside the tag resources are not related to our own count parameter to let us create multiple resources. And as you can see from our Terraform code above, we'll be able to access the index of the count parameter and convert it to a 1-based index before naming our tag.

One other thing to note here is the internal name of the resource. When we did not use the count parameter, our tag was named ram_8, so we could access it with digitalocean_tag.ram_8. However, now that we have a list of digitalocean_tag resources, we'll access our resources like they're inside an array: digitalocean_tag.ram[0].

`for_each` parameter to iterate over a data structure

count parameter was helpful to create numeric resources. But what if you need to iterate over a data structure like a list, a set, or a map? Let's continue from our earlier example, but this time create tags for operating systems.

Let's first create a variable to list our available operating systems:

variable "operating_systems" {
  description = "Operating systems available on droplets"
  type        = list(string)
  default     = ["ubuntu", "centos", "debian", "fedora", "freebsd"]
}

Notice how we defined our type as the list of strings. We'll come back to type usages at a later point. Now let's create our tags by using the for_each parameter:

resource "digitalocean_tag" "os" {
  for_each = toset(var.operating_systems)
  name     = each.value
}

The reason we are using toset on the operating_systems variable is that for_each supports a set or a map of strings. Otherwise, it would complain:

The given "for_each" argument value is unsuitable: the "for_each" argument must be a map or set of strings, and you have provided a value of type list of string.

Also, note how we used each.value to get the value from each iteration. Now when we run terraform plan, we'll see our tags in the execution plan:

Terraform will perform the following actions:

  # digitalocean_tag.os["centos"] will be created
  + resource "digitalocean_tag" "os" {
      + databases_count        = (known after apply)
      + droplets_count         = (known after apply)
      + id                     = (known after apply)
      + images_count           = (known after apply)
      + name                   = "centos"
      + total_resource_count   = (known after apply)
      + volume_snapshots_count = (known after apply)
      + volumes_count          = (known after apply)
    }

  # digitalocean_tag.os["debian"] will be created
  + resource "digitalocean_tag" "os" {
      + databases_count        = (known after apply)
      + droplets_count         = (known after apply)
      + id                     = (known after apply)
      + images_count           = (known after apply)
      + name                   = "debian"
      + total_resource_count   = (known after apply)
      + volume_snapshots_count = (known after apply)
      + volumes_count          = (known after apply)
    }

  # digitalocean_tag.os["fedora"] will be created
  + resource "digitalocean_tag" "os" {
      + databases_count        = (known after apply)
      + droplets_count         = (known after apply)
      + id                     = (known after apply)
      + images_count           = (known after apply)
      + name                   = "fedora"
      + total_resource_count   = (known after apply)
      + volume_snapshots_count = (known after apply)
      + volumes_count          = (known after apply)
    }

  # digitalocean_tag.os["freebsd"] will be created
  + resource "digitalocean_tag" "os" {
      + databases_count        = (known after apply)
      + droplets_count         = (known after apply)
      + id                     = (known after apply)
      + images_count           = (known after apply)
      + name                   = "freebsd"
      + total_resource_count   = (known after apply)
      + volume_snapshots_count = (known after apply)
      + volumes_count          = (known after apply)
    }

  # digitalocean_tag.os["ubuntu"] will be created
  + resource "digitalocean_tag" "os" {
      + databases_count        = (known after apply)
      + droplets_count         = (known after apply)
      + id                     = (known after apply)
      + images_count           = (known after apply)
      + name                   = "ubuntu"
      + total_resource_count   = (known after apply)
      + volume_snapshots_count = (known after apply)
      + volumes_count          = (known after apply)
    }

Plan: 5 to add, 0 to change, 0 to destroy.

`for` expression to map & filter

Another good loop-like behavior Terraform provides us with is the for expression. It resembles Python's comprehensions.

Let's define a variable for RAM requirements of operating systems:

variable "os_requirements" {
  description = "Which operating system requires how much RAM"
  type        = map(number)
  default = {
    ubuntu  = 4
    centos  = 2
    debian  = 3
    fedora  = 2
    freebsd = 1
  }
}

The numbers are entirely arbitrary, of course. :-) But note how we defined the type and assigned the RAM amounts to different operating systems here. Each of the OS described in the variable is the key in the map. RAM amounts are the values in the map.

Now we are going to filter & map within the same for expression.

output "usable_operating_systems" {
  value = [for os, ram in var.os_requirements : upper(os) if ram <= 2]
}

I always liked this type of one-liner comprehension. Let's see what we have achieved with this one-liner:

First, we extract both keys (os) and values (ram) from the map (var.os_requirements).
Then, we filter all map elements by their RAM amount. It should be smaller than or equal to 2.
For every filtered operating system, we map its name to uppercase by upper(os)/
Finally, we map the whole result into a list with the surrounding brackets [].

That's one type of filtering and two types of mapping in a one-liner.

Now we need to use terraform apply instead of terraform plan here because our Terraform code will not create any resources. To see the output, let's run it:

$ terraform apply

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:

Terraform will perform the following actions:

Plan: 0 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes


Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

usable_operating_systems = [
  "CENTOS",
  "FEDORA",
  "FREEBSD",
]

Perfect, now we know which operating systems we can use with the amount of RAM we have. :-)

Cover photo by Lysander Yuen

Part 5..........................................................................................................Part 7

What I've Learned Learning Terraform: Part 5

Ekin Öcalan — Mon, 24 Aug 2020 11:20:52 +0000

Terraform Series

Part 1: Introduction
Part 2: Creating the Server
Part 3: Provisioning the Server
Part 4: Managing Terraform State
Part 5: Cleaner Code with Terraform Modules
Part 6: Loops with Terraform
Part 7: Conditionals with Terraform
Part 8: Testing Terraform Code

Up until now, we focused on the functionality Terraform provided. Creating servers, adjusting network settings, configuring domain names, running bash scripts remotely, creating storage units, and moving Terraform state to shared space.

In this post, we're going to take a break and see how we can write cleaner, DRYer, and reusable Terraform code. The main keyword here is "reusable" because this post is mostly about Terraform modules.

Terraform Modules

Modules are the key ingredient to writing reusable, maintainable, and testable Terraform code. [1]

Having a module in Terraform easy, because any Terraform code stored under a directory is considered within the same module. For the same reasons, we can also say that modules in Terraform are implicit, and that's why creating a module is a bit of magic.

Let's take a look at our Terraform files we have created so far:

provider.tf: Defines the SSH variables Terraform uses to connect to our droplet. By the way, provider.tf is a pretty lousy name for this file. :-) We'll refactor it.
domain.tf: Points our domain to our droplet and defines its CNAME record.
space.tf: Declares our storage unit. It's a bucket in AWS and a space in DigitalOcean lingo.
main.tf: Configures pretty much what's left, including the provider version, remote backend, and our beloved droplet.

We created all of these files under the same directory. That's why they all belong to the same Terraform module. Now let's break it down so that we can reuse our code to create multiple droplets that be accessed by different domain names. This way, we can have an infrastructure supporting both production and staging environments. We are going to leave the buckets and remote state files out of this article to make things simple.

What we are aiming for is to create an infrastructure with both production and staging environments. We are going to create them with reusable Terraform modules. After we convert our codebase, our directory structure will look like as follows:

Let me get one thing out of the way first: versions.tf. I'm using Terraform v0.13 and you need to define your provider requirements starting with this version. So all my versions.tf files are the same:

terraform {
  required_providers {
    digitalocean = {
      source = "terraform-providers/digitalocean"
    }
  }
  required_version = ">= 0.13"
}

Directory structure with modules

As I mentioned earlier, each directory behaves as a separate module. I decided to divide my infrastructure into two modules: domain and server. Each of my environments will have a different server and a different domain pointing to that server. That's why I created a directory for each module under modules.

Let's start with the domain module. I copied the code from domain.tf into modules/domain/main.tf:

resource "digitalocean_domain" "domain" {
  name       = var.domain_name
  ip_address = var.server_ipv4
}

resource "digitalocean_record" "cname_www" {
  domain = digitalocean_domain.domain.name
  type   = "CNAME"
  name   = "www"
  value  = "@"
}

You should notice a change inside the digitalocean_domain resource: The values of name and ip_address are not hard coded anymore.

Module inputs

So I created a vars.tf file to define my module variables:

variable "domain_name" {
  description = "Domain name like yourdomain.com"
  type        = string
}

variable "server_ipv4" {
  description = "Server's IP address where the domain should point to"
  type        = string
}

Now I'm able to use these variables as inputs in my domain module. Whenever we use the domain module, we will have to provide both variables to create this resource. That's how we'll be able to define a domain with different domain names and separate IP addresses pointing to different servers.

Now let's move on to the server module. Again, I copied my code from my old main.tf file to here:

resource "digitalocean_droplet" "server" {
  image  = "ubuntu-20-04-x64"
  name   = var.server_name
  region = "ams3"
  size   = var.server_size
  ssh_keys = [
    var.ssh_fingerprint
  ]

  connection {
    host        = self.ipv4_address
    user        = "root"
    type        = "ssh"
    private_key = file(var.ssh_private_key)
    timeout     = "2m"
  }

  provisioner "remote-exec" {
    inline = [
      "export PATH=$PATH:/usr/bin",
      # install nginx
      "sudo apt-get update",
      "sudo apt-get -y install nginx"
    ]
  }
}

I created a vars.tf file for my server module as well:

variable "server_name" {
  description = "The name of the server"
  type        = string
}

variable "server_size" {
  description = "The size of the server"
  type        = string
}

variable "ssh_fingerprint" {
  description = "Fingerprint of the SSH key that is allowed to connect to the server"
  type = string
}

variable "ssh_private_key" {
  description = "Private key of the SSH key that is allowed to connect to the server"
  type = string
}

If we pass the SSH variables, you will see that we are now able to configure the server name and the server size in this module. That's how we'll be able to create a production server with a bigger size while keeping the staging server at a smaller size.

We now have both the domain and server modules to create ourselves in an environment. Let's start with production. Here is what it looks like to create an environment with our modules at production/main.tf:

module "server" {
  source = "../modules/server"

  server_name     = "terraform-sandbox"
  server_size     = "s-1vcpu-1gb"
  ssh_fingerprint = var.ssh_fingerprint
  ssh_private_key = var.ssh_private_key
}

module "domain" {
  source = "../modules/domain"

  domain_name = "productiondomain.com"
  server_ipv4 = module.server.server_ipv4
}

I'm going to pass the SSH variables here as we're going to provide them as command-line arguments. As you can see, we declare the name and size for the server module. Similarly, we give the name for our domain module. All hardcoded now. However, server_ipv4 argument for the domain module looks a bit strange, isn't it? :-)

Module outputs

What you see as module.server.server_ipv4 is the usage of module outputs. We have isolated our domain and server modules, but the domain configuration requires the server's IP address. We can access a module's values by defining an output in that module. Here is the content of modules/server/outs.tf:

output "server_ipv4" {
  value = digitalocean_droplet.server.ipv4_address
}

By defining the server_ipv4 output, we grant the module user access to the digitalocean_droplet resource's ipv4_address argument within the server instance. Similarly, we access the output by following this structure:

module.MODULE_NAME.OUTPUT_NAME

In our case, this becomes:

module.server.server_ipv4

This way, Terraform will create the server first, and then use its IP address to configure our domain.

Module locals

Apart from inputs and outputs, Terraform provides another data structure to make our codebase DRY. Instead of inputs and outputs, we don't use locals across modules. Their usage is limited to encapsulate local values, much like the constants in programming languages, but only within the same module.

For example, instead of hardcoding our server image and region, let's encapsulate them within modules/server/vars.tf:

locals {
  server_image = "ubuntu-20-04-x64"
  server_region = "ams3"
}

Then, we can go ahead and use them in our modules/server/main.tf file:

resource "digitalocean_droplet" "server" {
  image  = local.server_image
  name   = var.server_name
  region = local.server_region
  size   = var.server_size
  ssh_keys = [
    var.ssh_fingerprint
  ]

# ...

A note on inputs, outputs, and locals

I just wanted to take a small note here and say that all three structures are not unique to modules. As you can imagine, modules are implicit structures. In a theoretical sense, it would be true if we say that we can also use inputs, outputs, and locals outside of the context of the module. Although each of the usages will practically fall under the module usage since each directory in Terraform is a module.

[1]: Terraform Up & Running: Writing Infrastructure as Code by Yevgeniy Brikman (2nd edition)

Cover photo by Andrej Lišakov

Part 4.........................................................................................................Part 6

What I've Learned Learning Terraform: Part 4

Ekin Öcalan — Thu, 20 Aug 2020 10:56:14 +0000

Terraform Series

Part 1: Introduction
Part 2: Creating the Server
Part 3: Provisioning the Server
Part 4: Managing Terraform State
Part 5: Cleaner Code with Terraform Modules
Part 6: Loops with Terraform
Part 7: Conditionals with Terraform
Part 8: Testing Terraform Code

Every time you run Terraform, it records information about what infrastructure it created in a Terraform state file. [1]

Managing Terraform State

Terraform state files are the backbone of your infrastructure. Terraform can only operate on the infrastructure it knows, and state files are the only way it can know about them.

Limiting Terraform by state files is very good in the context of isolation. You may have other resources you created manually in the same provider account. Terraform doesn't know them. Thus, you cannot accidentally change or destroy those resources.

On the other hand, it means that state files are the source of truth, and thus critical to your infrastructure-as-code. If you lose or corrupt your state file, you'll no longer have the ability to maintain your resources. That's why you need some sort of backup of your state against data loss. Additionally, you need versioning against data corruption.

A version control system might be the first possible solution that comes to mind. It indeed can be a remedy to both loss and corruption. You should use version control for your Terraform code; however, you should not put your state files to your version control for two reasons:

If multiple people are working on the same codebase, your state can get out-of-date because getting the latest changes requires manual pulls from the version control.
State files contain secrets as plain-text. It's a bad idea to store them unencrypted.

Using remote backend instead of local

Fortunately, Terraform offers two ways to store state files. The first one is storing locally, and that's the default way. The second one is storing state files remotely.

There are several advantages to store state files remotely. If you use version control for your Terraform codebase and remote backend for your state files, you no longer depend on your local environment. Even if you lose your machine, you can still work your infrastructure with Terraform.

It also allows collaboration as well. With some of the remote backends, you can set up a lock mechanism to prevent race conditions.

Most remote backends support encryption by default. Thus, your secrets are stored encrypted. You can also set access rules, so only your codebase is allowed to access the state programmatically, and thus restricting any 3rd party access.

Moving state files to DigitalOcean Spaces

DigitalOcean Spaces is an object storage product similar to AWS S3. In fact, it's S3-compatible [2]. That's why, even though Terraform's standard remote backends do not list Spaces as a possible backend type, we can use it as an S3 type.

Creating a Space

While it's alright to create a Space manually to store state files and use its configuration, let's create it with Terraform for this article's sake. Create a space.tf file and populate it with the code below:

resource "digitalocean_spaces_bucket" "tf_state" {
  name   = "terraform-sandbox"
  region = "ams3"
  acl    = "private"
  versioning {
    enabled = true
  }
}

Here we create a digitalocean_spaces_bucket resource. In AWS lingo, a bucket is a standalone storage unit. "tf_state" is the internal name of this resource within the codebase. Let's also take a look at the configuration parameters:

name: The bucket/space name. It should be unique within the region.
region: One of the datacenters listed at Regional Availability Matrix.
acl: Access control list. public for public file listing, private for restriction. Since we are going to use this space to store state files, we use private.
versioning: Enabling the versioning will allow us to restore the state file(s) to an older but working version in case of data corruption or human error.

Run terraform init and terraform apply. You'll see an execution plan like this:

  # digitalocean_spaces_bucket.tf_state will be created
  + resource "digitalocean_spaces_bucket" "tf_state" {
      + acl                = "private"
      + bucket_domain_name = (known after apply)
      + force_destroy      = false
      + id                 = (known after apply)
      + name               = "terraform-sandbox"
      + region             = "ams3"
      + urn                = (known after apply)

      + versioning {
          + enabled = true
        }
    }

To apply this plan, you are going to need an access key and a secret key for Spaces. Go to API on DigitalOcean to create a pair. Then, on command line, you can export these environment variables for Terraform to pick up the credentials for Spaces:

$ export SPACES_ACCESS_KEY_ID=11111111111111111111
$ export SPACES_SECRET_ACCESS_KEY=AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

Configuring Terraform to use remote state on the Space

Now that we have a Space on DigitalOcean to store our state files, let's configure Terraform to use the state file there. While configuring, Terraform will also ask us if we want to transfer our local state to the remote or start from scratch. Since we already have some resources created, we are going to pick the transfer option to avoid multiple resource creation.

Open main.tf and find the terraform block:

terraform {
  required_providers {
    digitalocean = {
      source  = "digitalocean/digitalocean"
      version = "1.22.1"
    }
  }
}

What we're doing now is directly related to how Terraform will behave. That's why we are defining our remote backend in this block:

terraform {
  required_providers {
    digitalocean = {
      source  = "digitalocean/digitalocean"
      version = "1.22.1"
    }
  }

  backend "s3" {
    endpoint                    = "ams3.digitaloceanspaces.com"
    bucket                      = "terraform-sandbox"
    key                         = "terraform.tfstate"
    region                      = "ams3"
    access_key                  = "11111111111111111111"
    secret_key                  = "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
    skip_credentials_validation = true
    skip_metadata_api_check     = true
    skip_region_validation      = true
  }
}

Notice how we defined our backend as s3. Our Space on DigitalOcean is S3-compatible, and that's why we are defining our remote backend as it was S3 storage. Let's go over the configuration parameters:

endpoint: You can find this at your Spaces page. At the time of this writing, the structure is region.digitaloceanspaces.com.
bucket: The name of your Space.
key: The full path to the state file on remote backend. This will be your state file.
region: Just put your Space region. We'll ask Terraform to not use this value as it's going to look at AWS regions here, and not DigitalOcean ones.
access_key: Your Spaces access key.
secret_key: Your Spaces secret key.
skip_credentials_validation: We are not using AWS credentials, that's why we skip.
skip_metadata_api_check: We are not using AWS EC2, that's why we skip.
skip_region_validation: We are not using AWS, that's why we skip the region validation. Since region is a required parameter, Terraform will complain about our DigitalOcean region if we don't skip this validation.

Now when you run terraform init, you'll see a question:

Initializing the backend...
Do you want to copy existing state to the new backend? Pre-existing state was found while migrating the previous "local" backend to the newly configured "s3" backend. No existing state was found in the newly configured "s3" backend. Do you want to copy this state to the new "s3" backend? Enter "yes" to copy and "no" to start with an empty state.

  Enter a value:

As I mentioned before, we'll say yes. Then, Terraform will transfer our local state file to Spaces, and configure itself to use the remote state:

Successfully configured the backend "s3"! Terraform will automatically use this backend unless the backend configuration changes.

From now on, Terraform will read state values from remote, and write updated values to the remote.

A note on the Terraform configuration values

If you noticed, we used plain-text sensitive values (access key and secret key) when configuring our remote backend. That is because Terraform doesn't allow variables inside the Terraform configuration. For the same reason, we could not populate bucket, key, and region parameters from our digitalocean_spaces_bucket.tf_state resource definition.

One way of improving this process is to use partial configuration. You can remove access_key and secret_key from the configuration, and then supply them as backend configs on the initialization process:

$ terraform init -backend-config="access_key= 11111111111111111111" -backend-config="secret_key= AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"

[1]: Terraform Up & Running: Writing Infrastructure as Code by Yevgeniy Brikman (2nd edition)
[2]: S3-Compatibility on Spaces Features

Cover photo by Lÿv Jaan

Part 3.........................................................................................................Part 5

What I've Learned Learning Terraform: Part 3

Ekin Öcalan — Mon, 17 Aug 2020 10:27:03 +0000

Terraform Series

Part 1: Introduction
Part 2: Creating the Server
Part 3: Provisioning the Server
Part 4: Managing Terraform State
Part 5: Cleaner Code with Terraform Modules
Part 6: Loops with Terraform
Part 7: Conditionals with Terraform
Part 8: Testing Terraform Code

Now that we could create a server, it's time to configure it a bit. It's no use to us if it sits right there. So let's install nginx and then have one of our domains point to the server to see if it's working or not.

Provisioning the server

It's only to install nginx for now, but it's still a server provision. We are going to use remote-exec provisioner to achieve this simple install.

Within digitalocean_droplet resource, add the provisioner below:

resource "digitalocean_droplet" "web" {
  image  = "ubuntu-20-04-x64"
  name   = "terraform-sandbox"
  region = "ams3"
  size   = "s-1vcpu-1gb"

  provisioner "remote-exec" {
    inline = [
      "export PATH=$PATH:/usr/bin",
      # install nginx
      "sudo apt-get update",
      "sudo apt-get -y install nginx"
    ]
  }
}

There are better ways to provision a server, but we'll get to that later. For our simple nginx install, this would do the trick.

Still, there is one thing missing here. How is Terraform going to connect to our server to run the bash script above?

Granting Terraform the SSH access

You first need to go to your Account/Security page on DigitalOcean and add your computer's SSH key there. If you don't know how to do it, DigitalOcean helps you to create and use an SSH key on that page. You're going to need the SSH key fingerprint shown on that page.

Now we are going to create a new Terraform file and call it variables.tf.

variable "public_key" {}
variable "private_key" {}
variable "ssh_fingerprint" {}

You see how we create variables in Terraform. Typically, you can give them default values as well. However, since they contain sensitive data, we deliberately left them empty. This way, Terraform will ask us to fill them while planning or applying.

We head back to our main.tf now, and add the ssh_keys parameter inside our droplet resource.

resource "digitalocean_droplet" "web" {
  image  = "ubuntu-20-04-x64"
  name   = "terraform-sandbox"
  region = "ams3"
  size   = "s-1vcpu-1gb"
  ssh_keys = [
    var.ssh_fingerprint
  ]

  provisioner "remote-exec" {
    inline = [
      "export PATH=$PATH:/usr/bin",
      # install nginx
      "sudo apt-get update",
      "sudo apt-get -y install nginx"
    ]
  }
}

This addition will grant the SSH key holder to access our droplet. We provided a list of ssh_keys. It means that you can add more than one. Additionally, it also shows how we can access variables defined within the Terraform files. Hint: var.ssh_fingerprint, var.public_key, etc.

Lastly, we are going to define a connection within the droplet resource.

resource "digitalocean_droplet" "web" {
  image  = "ubuntu-20-04-x64"
  name   = "terraform-sandbox"
  region = "ams3"
  size   = "s-1vcpu-1gb"
  ssh_keys = [
    var.ssh_fingerprint
  ]

  connection {
    host        = self.ipv4_address
    user        = "root"
    type        = "ssh"
    private_key = file(var.private_key)
    timeout     = "2m"
  }

  provisioner "remote-exec" {
    inline = [
      "export PATH=$PATH:/usr/bin",
      # install nginx
      "sudo apt-get update",
      "sudo apt-get -y install nginx"
    ]
  }
}

This connection declaration defines an SSH connection. The host is the IP address of our droplet. Notice how we used the self object there. It's going to connect with the root user, using the private key of our computer.

Our main.tf is ready to create a droplet, and then make a connection to it to install nginx. Now, instead of a plain terraform apply, we need to do it with populating the variables as well:

$ terraform apply \
  -var "public_key=$HOME/.ssh/id_rsa.pub" \
  -var "private_key=$HOME/.ssh/id_rsa" \
  -var "ssh_fingerprint=YOUR_FINGERPRINT"

You need to use your SSH key fingerprint instead of YOUR_FINGERPRINT. And please also note that the public_key and private_key values may be different based on how you created them.

When you run the command, it's going to show you the plan and ask for your approval. You won't see the provisioner in the plan. But when you apply itthe plan, you'll see how Terraform makes the connection to the droplet, and you'll see the logs of the provisioner script.

Pointing a domain to the server

We have a server, and we have nginx running on it. You can visit the IP address of your server to see it with your eyes. However, let's move one step ahead and use a domain instead of the IP address.

Create a domain.tf file and populate it with:

resource "digitalocean_domain" "yourdomain" {
   name = "yourdomain.com"
   ip_address = digitalocean_droplet.web.ipv4_address
}

resource "digitalocean_record" "cname_www" {
  domain = digitalocean_domain.yourdomain.name
  type = "CNAME"
  name = "www"
  value = "@"
}

First, we create a digitalocean_domain resource. You need to update the name with your domain address. Notice how we used the ipv4_address attribute to point our domain to a specific IP address. You can see the full list of attributes reference on Terraform Registry. The same way we access the IP address here, you can access any attribute from digitalocean_droplet.web.

Second, we create a digitalocean_record resource to point all www subdomain requests to the main one. Thus, both yourdomain.com and www.yourdomain.com will lead to our droplet. See how we set the domain parameter inside to digitalocean_domain.yourdomain.name. That's how Terraform will understand under which domain it should create this record.

Resource order is not important

One interesting fact with Terraform we can mention here is that Terraform doesn't care about the order of the resources. Even if you declare digitalocean_record resource before the digitalocean_domain one, it would still work. That's because Terraform is declarative, and it creates its dependency graph based on what you declared.

Back to the domain creation... Now if you apply your plan again:

$ terraform apply \
  -var "public_key=$HOME/.ssh/id_rsa.pub" \
  -var "private_key=$HOME/.ssh/id_rsa" \
  -var "ssh_fingerprint=YOUR_FINGERPRINT"

You'll see a plan just like this:

Terraform will perform the following actions:

  # digitalocean_domain.yourdomain will be created
  + resource "digitalocean_domain" "yourdomain" {
      + id         = (known after apply)
      + ip_address = "111.111.111.111"
      + name       = "yourdomain.com"
      + urn        = (known after apply)
    }

  # digitalocean_record.cname_www will be created
  + resource "digitalocean_record" "cname_www" {
      + domain = "yourdomain.com"
      + fqdn   = (known after apply)
      + id     = (known after apply)
      + name   = "www"
      + ttl    = (known after apply)
      + type   = "CNAME"
      + value  = "@"
    }

Plan: 2 to add, 0 to change, 0 to destroy.

Apply the plan, and that's all. Now if you visit your domain, you'll see your that nginx's default site configuration will catch your request and show you its default page:

PS: Cover photo by tian kuan

Part 2.........................................................................................................Part 4

What I've Learned Learning Terraform: Part 2

Ekin Öcalan — Thu, 13 Aug 2020 19:08:56 +0000

Terraform Series

Part 1: Introduction
Part 2: Creating the Server
Part 3: Provisioning the Server
Part 4: Managing Terraform State
Part 5: Cleaner Code with Terraform Modules
Part 6: Loops with Terraform
Part 7: Conditionals with Terraform
Part 8: Testing Terraform Code

Choosing a Provider

Terraform is using HashiCorp Configuration Language (HCL). Although HCL is unifying how Terraform should be coded, different providers allow you to achieve different goals with different scopes. That's why it's not the same to create a server with AWS provider and GCP provider (although similar).

I have a few resources I use to learn and practice Terraform. One of them is the book Terraform: Up & Running: Writing Infrastructure as Code by Yevgeniy Brikman. It's a beautiful book in terms of how simple it goes through the basics. All examples are for AWS. My day-to-day job, on the other hand, requires me to work with GCP. That's why I chose to practice with something different which is also my favorite cloud provider: DigitalOcean. :-)

You can use my referral link to get $100 DigitalOcean credits: https://m.do.co/c/4916561afc69. It is more than enough to run through all practices in this series several times.

Why DigitalOcean? Because it's not as big and as complex as AWS or GCP. I love how DigitalOcean focuses on a limited set of assets like servers, managed DBs, and spaces. If you're not building for an enterprise, DigitalOcean provides more than enough with excellent UX.

Creating a Server aka Droplet

We are going to create a droplet, and then configure it step by step. First, create a main.tf file and put the content below:

terraform {
  required_providers {
    digitalocean = {
      source  = "digitalocean/digitalocean"
      version = "1.22.1"
    }
  }
}

1.22.1 is the latest version of the DigitalOcean provider at the time of this writing. You can check the registry to find the newest version.

Next, we are going to create a resource to declare the droplet. In the same main.tf file, add this to the bottom:

resource "digitalocean_droplet" "web" {
  image  = "ubuntu-20-04-x64"
  name   = "terraform-sandbox"
  region = "ams3"
  size   = "s-1vcpu-1gb"
}

Here the name digitalocean_droplet identifies the resource as defined by the provider. web, on the other hand, is a name given by us to use this resource within Terraform later. Let me also list the configuration parameters below:

image: The OS image.
name: The server's name.
region: The datacenter's region. You can check Regional Availability Matrix to find yourself a suitable region.
size: The server type DigitalOcean provides. You can check the list of size slugs.

Image and size options are available via DO's API, but we'll talk about them later on. The example values given above are enough to continue our demo.

The code inside the main.tf file is sufficient to create a droplet. Now let's head to the terminal and run the commands below from the directory the main.tf file resides.

Export your DigitalOcean access token as env

There are multiple ways to provide your access token to Terraform so that it can communicate with the provider using its API. For the sake of simplicity, we are going to export it on the command line, and Terraform will pick it up as long as we keep that command-line session open.

export DIGITALOCEAN_ACCESS_TOKEN=cbb3b23bf9281232bd7cc60d0b281f1483d82b45cd067e4ca8a401cbbf44df3e

The access token above is fake, of course. You need to visit the API page on your DigitalOcean account to create one.

Initialize Terraform

Terraform has hundreds of providers, and those providers don't come out of the package. By initializing, Terraform prepares your working directory and installs the required provider plugins. You need to rerun this command whenever you set or change modules or backend configuration for Terraform.

$ terraform init

The above command will do what I described above and return a long success message which eventually comes down to this:

Terraform has been successfully initialized!

Plan the changes

Terraform has a very nice feature where you can audit the changes it will make before applying anything.

$ terraform plan

The above command will generate an execution plan and show it to you. In our case, it's going to look something like this:

Terraform will perform the following actions:

  # digitalocean_droplet.web will be created
  + resource "digitalocean_droplet" "web" {
      + backups              = false
      + created_at           = (known after apply)
      + disk                 = (known after apply)
      + id                   = (known after apply)
      + image                = "ubuntu-20-04-x64"
      + ipv4_address         = (known after apply)
      + ipv4_address_private = (known after apply)
      + ipv6                 = false
      + ipv6_address         = (known after apply)
      + ipv6_address_private = (known after apply)
      + locked               = (known after apply)
      + memory               = (known after apply)
      + monitoring           = false
      + name                 = "terraform-sandbox"
      + price_hourly         = (known after apply)
      + price_monthly        = (known after apply)
      + private_networking   = (known after apply)
      + region               = "ams3"
      + resize_disk          = true
      + size                 = "s-1vcpu-1gb"
      + status               = (known after apply)
      + urn                  = (known after apply)
      + vcpus                = (known after apply)
      + volume_ids           = (known after apply)
      + vpc_uuid             = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.

You can already see the configuration values we've provided in the plan.

Apply the execution plan

Our final step to create the server is to apply the plan. You can ignore the terraform plan command after some time because Terraform is going to require your approval one more time at this step.

$ terraform apply

This command will show you the execution plan we've been shown above and ask you to approve:

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value:

When you enter yes, it's going to go ahead and create your droplet:

  Enter a value: yes

digitalocean_droplet.web: Creating...
digitalocean_droplet.web: Still creating... [10s elapsed]
digitalocean_droplet.web: Still creating... [20s elapsed]
digitalocean_droplet.web: Still creating... [30s elapsed]
digitalocean_droplet.web: Creation complete after 34s [id=111111111]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Now you can go ahead and check your newly created droplet on your account:

PS: Cover photo by Aron Visuals

Part 1..........................................................................................................Part 3

What I've Learned Learning Terraform: Part 1

Ekin Öcalan — Mon, 10 Aug 2020 19:05:47 +0000

Software isn't done until you deliver it to the user. [1]

We've been discussing what done means at my Scrum team lately. There were a lot of suggestions. One of us said that when the code is ready for review, at least the implementation part is done. Others said that we could not specify a task as done until we test it on production and see it working as expected. "Done" means that we don't have anything left to do with it. That's why the above quote explains what "Done" means in terms of software simply.

You have a small feature to implement. You write the code. Someone from your team reviews it and gives feedback. You take your code back, polish it, maybe refactor it to meet your team's criteria. Then you move on to the QA. Oh, something just doesn't look right. You fix it and push it again. Cool! Now your work passes the QA. You merge your PR and deploy it to production.

The scenario above touches two main points at the end:

Continuous Integration (CI) where you integrate your work as frequently and as soon as possible. Having small changes in your work helps, because this way, your code does not get too different from the main branch.
Continuous Deployment (CD) [2] where you not only keep your integrated work deployable at all times, but you also deploy your main branch to production with each integration automatically.

CI and CD approach aims to deliver software faster. They pretty much succeed at this job. However, your software isn't the only thing that is delivered quickly and frequently. Your third party libraries get updated regularly. Your cloud provider issues updates to its infrastructure. Your data gets bigger by the day. Your network needs may change as you expand. So, basically, how do you scale?

The Devops movement encourages software developers, operations staff, and everyone else involved in delivery to work together - avoiding hand-offs that add delays and brittleness. Infrastructure-As-Code takes advantage of our cloud age ability to rapidly deploy and provision new servers. [3]

Infrastructure-As-Code (IaC) lets you define your infrastructure with code so that you can configure your provisions and replicate it at a later point. With version control, you can reverse your infra to an older state if something goes wrong. It makes your operations reliable and scalable. You don't have to worry about upgrading or maintaining a single server you have.

CI is up to me. I need to integrate my work as frequently as possible. CD is up to our deployment rules. I want to deploy my work as often as possible. Now with IaC, I also get control over where my code gets deployed. That is the main reason why I decided to learn more about Terraform [4]. And that is why I want to summarize what I've learned while practicing Terraform.

Terraform Series

[1]: Terraform Up & Running: Writing Infrastructure as Code by Yevgeniy Brikman (2nd edition)
[2]: Not to be confused with Continuous Delivery where you keep your work deployable at all times but make the deployments manually from time to time.
[3]: Software Delivery Guide by Martin Fowler
[4]: Terraform is one of the most popular IaC tools. Popularity and adoption are key to keep the IaC tool up-to-date with various providers.

Cover picture by Lucas Myers

PS: Also checkout Terraforming on Wikipedia!

...................................................................................................................Part 2

Forem: Ekin Öcalan

WILw Reading Software Architecture: Cohesion

Cohesion types from worst to best

Coincidental cohesion — the worst

Logical cohesion

Temporal cohesion

Procedural cohesion

Communicational cohesion

Sequential cohesion

Functional cohesion

Atomic cohesion — the best

What I've Learned Learning Terraform: Part 8

Terraform Series

Manual Testing vs. Automated Testing

Manual Testing

Automated Testing

What I've Learned Learning Terraform: Part 7

Terraform Series

Conditionals

Conditional expressions

Creating resources conditionally

Setting values conditionally (Interpolation)

What I've Learned Learning Terraform: Part 6

Terraform Series

Loops

count parameter to create multiple resources

for_each parameter to iterate over a data structure

for expression to map & filter

What I've Learned Learning Terraform: Part 5

Terraform Series

Terraform Modules

Directory structure with modules

Module inputs

Module outputs

Module locals

A note on inputs, outputs, and locals

What I've Learned Learning Terraform: Part 4

Terraform Series

Managing Terraform State

Using remote backend instead of local

Moving state files to DigitalOcean Spaces

Creating a Space

Configuring Terraform to use remote state on the Space

A note on the Terraform configuration values

What I've Learned Learning Terraform: Part 3

Terraform Series

Provisioning the server

Granting Terraform the SSH access

Pointing a domain to the server

Resource order is not important

What I've Learned Learning Terraform: Part 2

Terraform Series

Choosing a Provider

Creating a Server aka Droplet

Export your DigitalOcean access token as env

Initialize Terraform

Plan the changes

Apply the execution plan

What I've Learned Learning Terraform: Part 1

Terraform Series

`count` parameter to create multiple resources

`for_each` parameter to iterate over a data structure

`for` expression to map & filter