Forem: Klenam Fiah

Working with Workspaces and Backends in Terraform

Klenam Fiah — Fri, 30 Apr 2021 21:02:08 +0000

Terraform manages infrastructure with state files. By default you have a single workspace, default. So when you run terraform plan and terraform apply you are working the default workspace prepared by terraform.
But with workspaces we can have multiple states. What this means, we can have different server setup with different state files managing that server setup(keep records of what has been deployed at every time). Also it cuts down the amount of copy and paste you will be doing.

To set up workspaces in terraform is very easy. The command terraform workspace new <name of workspace> does that. Assuming I needed different workspaces to keep tabs on my deployment for DEV, QA and PROD. This will be simply achieved by running-
terraform workspace new DEV, terraform workspace new QA and terraform workspace new PROD. This creates three different sub-folders under the terraform.tfstate.d folder

Workspace created! How do we run our Terraform code in a specific workspace? Simply run terraform workspace select <name of workspace>. Terraform will throw you a prompt on what workspace you are working in. Go head and run terraform plan and terraform apply

This is all well and good if you are building infrastructure alone because you are an amazing engineer. More often than not, tasks are performed in teams and we need some collaboration to get things done effectively. How do we achieve this with workspaces🤔? We should use some backend to store these state files✨😎.

Using AWS as our provider, I'll demonstrate

We create an S3 bucket to store the state files.

Note: bucket name must be unique because the s3 namespace is a shared one. You get an error like
Error: Error creating S3 bucket: BucketAlreadyExists: The requested bucket name is not available. The bucket namespace is shared by all users of the system. Please select a different name and try again. status code: 409, request id: EMBTK255FN7K3QDT, host id: 20vAPtmPGrMG/2tec=
if the bucket name has been taken.

Now we need some way to lock our file when we are deploying to avoid your team from walking over each other. This way just one person can deploy at a time. We will use AWS DynamoDB to achieve this

The table must have a primary key named "LockID" and type "S" (type string). You get an error if this isn't so. Error below:
Error: Error locking destination state: Error acquiring the state lock: 2 errors occurred: * ValidationException: One or more parameter values were invalid: Missing the key Locker in the item status code: 400, request id: 7BRN5V5Q92K0393GO5AEMVJF66Q9ASUAAJG * ValidationException: The provided key element does not match the schema status code: 400, request id: KDJ19AI5UB2TJBSQ9ASUAAJG

With all this in place we can now terraform init -> terraform plan -> terraform apply.

The final step to achieve our objective will be to declare our backend in terraform.

Again we need to initialize. terraform init
Done correctly Terraform will seek permission to make of a copy of your state to the backend.



Initializing the backend...
Acquiring state lock. This may take a few moments...
Do you want to copy existing state to the new backend?
Pre-existing state was found while migrating the previous "local" backend to the newly configured "s3" backend. No existing state was found in the newly configured "s3" backend. Do you want to copy this state to the new "s3" backend? Enter "yes" to copy and "no" to start with an empty state.

Enter a value:

Real easy, huh?!🙃

If you go into your S3 bucket you should see a path env:/ with all your workspaces listed with each it's own state file.

The terraform has good documentation on this.

+++Header Photo credit: Pixabay's mozlase__+++

                   Happy Computing!!💻✌️

Building a two tier infrastructure with Terraform in AWS

Klenam Fiah — Fri, 26 Mar 2021 15:01:36 +0000

During the lockdown I had an intriguing conversation with a close buddy of mine from the university. During our usual undirected and wild conversations I asked what's the newest tech he's been dabbling in. He mentioned Terraform and he spoke about how awesome it was . I was totally intrigued so set myself on this the path of discovering this amazing IaC (Infrastructure as Code) tool.

I didn't know what to expect and I feared I wouldn't grasp the concepts because this was all new to me and I hadn't ever taken a swim in the DevOps pool, yada yada yada 😁. Down the rabbit hole I went.

In this project I deployed a FastAPI in my first tier and a PostgreSQL and Ansible machines as my second tier. Could have designed a proper frontend but I am horrible with frontend😅 and all my pals had a ton of things to work on as well. I'll be focusing on the deployment of the infrastructure into AWS in this write-up.

Let's get down to it!😎

To get started we need to prepare:
*get yourself a AWS free tier account. this will give you limited but enough access to all AWS has to offer for a year

get AWS cli configured. Useful AWS doc to do this
get the Terraform CLI
get FastAPI installed in a virtual environment. Getting FastAPI installed (not necessary here so you could ignore)

Since the focus of this article isn't going to be on FastAPI, I'll won't be showing how to put it together. The FastAPI docs are one of the best and will give you a good guide on how to get something going in no time.

To get the ball rolling, in AWS you need to create a VPC. What's a VPC you might be asking😳? A VPC(virtual private cloud) is your personal isolated corner within the AWS cloud. Read more?

You might want to ignore the data source defined above the resource definition. To read more about Data Sources. It's beyond the scope. I'll love to define all the attributes but I'll only explain attributes that were a hell for me, mostly because I can be slow in the head sometimes🤣. Terraform docs for providers(AWS in this case) explain these attributes so well.

The next thing to do do will be to divide this VPC CIDR block into smaller portions for our private and public subnets. The DB needs to be associated with a subnet. In here, I associate it with both subnets.

With the subnet provisioned we have to create security groups. Security groups are rules that govern what comes in and out of our instance.(An instance is virtual server in the AWS cloud).

I have three Security groups to manage my instances -

aws_security_group.backend_security_groups - to manage my FastAPI backend
aws_security_group.db_security_groups - to manage my PostgreSQL server
aws_security_group.only_ssh_bastion - to manage my Bastion server(What's this weird thing😭? A Bastion server? Will explain soon)

We need now need an internet gateway to communicate with the outside world

With that done we need a way to direct traffic within our VPC. In walks routing tables🚶‍♂️. By default when you create a VPC it comes with a default route table but I will advise you create custom route tables. This helps in troubleshooting issues.

We will create route tables that point to the Internet Gateway, VPC Internet Gateway and one between our FastAPI and Database instances.

Here I'll explain the attributes the best way I understand them.

Taking the aws_route_table.backend_route_table:

vpc_id: this associates the route table to our VPC

route:

cidr_block = "0.0.0.0/0" .What this is saying is, when you see any IP that doesn't belong within our VPC IP range direct this towards the internet gateway. Same logic applies to aws_route_table.nat_route_table.

We then need to associate these routes to subnets. So we associate them to the public and private subnets

We then need a way for our instances in the private subnet to communicate with the internet but not the internet getting to the them. A VPC NAT Network Address Translation Gateway will serve that purpose. This helps when we need our servers to pull updates and patches.

What next? This is taking forever to put together😡. I agree but remember when I said I went down the rabbit hole🤪?

To be able to reach instances and resources in our public subnet from outside our VPC, we need to assign Elastic IP Addresses to them. AWS gives you five eips when you use the free tier and it's yours till you release them(destroy the build). Read more

We are almost there. Hold your focus for 10 more minutes!

We need our instances now. AWS instances are best described by me as our virtual servers. We will need three (for the FastAPI backend, Bastion and Ansible control node). In the snips below you will see some other stuff like provisioner and connection. You can ignore these since they only help us perform some provisioning on our instances when we are spinning them up. It does this by using SSH. Not our focus for now.

Now to answer some questions👩‍🏫. "Chris, what's a Bastion Server?". A Bastion Server is a special server that reduces the attack vector of your infrastructure by limiting access to the restricted part of your setup. Some call it a jump sever. So to get to the instances in the private network(in this case my ansible control node) I need to SSH into the Bastion server then hop onto my Ansible control node by SSH from the Bastion. Super, right?🧑‍🚀

We do the same for our database but amazon has special instances called RDS(Relational Database Server) that can be provisioned on build automatically.

Now to get it over with… I left this very important one for last to make sure you read to the end.😈. To tell Terraform what Provider you are going to be building your infrastructure we need a provider block.

Phwee! we are done. There are a couple of other nuances you will need to figure out. What's a lesson without an assignment?😂. You will need variables to provide your code with your aws access key, aws secret key, instance type, provide tags, path to private key(for SSH), database name, database password. Notice in the code a lot of var.*, it means I'm calling variable. You will need separate SSH keys for your Bastion Server and Ansible. Why would you use the same SSH key for both? Kinda forfeits the purpose security wise. You can find the codes here.

Happy Computing!✌️💻