Forem: Mark Sta Ana

Adding missing functionality to Terraform

Mark Sta Ana — Sun, 27 Oct 2019 13:37:00 +0000

Photo by Hello I'm Nik 🇬🇧 on Unsplash

I needed to codify the creation of PostgreSQL read replicas, so I did a bit of research around ways I could do this quickly without diving into the Terraform provider.

The quickest way to do this was to:

use the local-exec provisioner to invoke the Azure CLI commands (details to follow)
wrap the code in a module (to allow for reuse and share with the community)

The Azure docs requires the following steps to be carried out to create a read replica using the Azure CLI are:

Enable replication support on the primary server
Restart the primary server (for the changes to take effect)
Create the replica using the primary server as the source

Caveat emptor: as the Terraform docs mention, provisioners are a last resort. A major downside of using this method to add missing functionality is that there's no state tracking, i.e. if you make a change to the resource, Terraform won't know about it.

Here's the essence of the code (I've omitted certain details for brevity the full code is on GitHub):

resource "null_resource" "postgresql-read-replica" {
  triggers = {
    resource_group_name            = var.resource_group_name
    postgresql_primary_server_name = var.postgresql_primary_server_name
    postgresql_replica_server_name = var.postgresql_replica_server_name
  }

The null_resource is also provisioner is used as a container for the local_exec calls. The triggers block allows the resource to be replaced, i.e. destroyed and recreated when the resource group, the PostgreSQL primary or replica server names changes.

You can already see that modules are just ordinary bits of Terraform code.

  provisioner "local-exec" {
    command = <<ENABLE_REPLICATION
az postgres server configuration set \
...
ENABLE_REPLICATION
  }

  provisioner "local-exec" {
    command = <<RESTART_SERVER
az postgres server restart \
...
RESTART_SERVER
  }

  provisioner "local-exec" {
    command = <<CREATE_REPLICA
az postgres server replica create \
...
CREATE_REPLICA
  }

These three provisioner blocks perform the required actions to create a read replica using the Azure CLI. To avoid having to escape quotes we're using the here doc notation.

  provisioner "local-exec" {
    when = "destroy"
    command = <<DESTROY_REPLICA
az postgres server delete \
  --name ${var.postgresql_replica_server_name} \
  --resource-group ${var.resource_group_name} \
  --yes
DESTROY_REPLICA
  }
}

Finally, we handle when what to do when the replica is destroyed.

I've uploaded the module to the Terraform registry which means the module can be easily referenced just like another resource:

module demo-replica {
  source                         = "booyaa/terraform-azurerm-postgresql-read-replica"
  resource_group_name            = azurerm_resource_group.demo.name
  postgresql_primary_server_name = azurerm_postgresql_server.demo.name
  postgresql_replica_server_name = "${azurerm_postgresql_server.demo.name}-replica"
}

Just like the Data Sources, modules can be used as a reference so we can now apply a firewall rule against the read replica:

resource "azurerm_postgresql_firewall_rule" "demo-replica" {
  name                = "office"
  resource_group_name = azurerm_resource_group.demo.name
  server_name         = module.demo-replica.replica_name
  start_ip_address    = "8.8.8.8"
  end_ip_address      = "8.8.8.8"

  depends_on = [module.demo-replica]
}

Learning about eBPF on macOS

Mark Sta Ana — Sun, 13 Oct 2019 12:21:00 +0000

Photo by Sarah Lee on Unsplash

I've created this is a short post to talk about a new GitHub repo that might be useful to some: vagrant-bcctools.

It's a simple Vagrant box using the latest (at the time of writing) version of Ubuntu (bionic) with the bcc tools package installed.

I needed a way to play around with eBPF on macOS locally. So before embarking on a fool's errand, I did some research. For details about my findings, see the repo's README.md.

I saw there was a Docker image, which doesn't work because I think it expects the underlying Docker host to be Linux base (volume mounts to /lib/modules, /usr/src and /etc/localtime).

The Vagrantfile provided by IO Visor is using such an old version of Ubuntu that a modern version of Vagrant seems to choke on it.

No doubt someone will point me to something that only requires xhyve (at me on Twitter or dev.to if you do know).

Thanks HacktoberFest!

Mark Sta Ana — Mon, 07 Oct 2019 07:55:11 +0000

Just a small diversion before the article begins in earnest...

I'm available for hire! If you want to get in touch contact details are available in this twitter thread.

Alternatively you can either drop me message here or check out my profile for LinkedIn details.

</jobAd>

Photo by Kerstin Wrba on Unsplash

I've participated in a couple of HacktoberFests (for the uninitiated this is a month-long challenge to help make open source projects better. Check out the tag below for more info).

#hacktoberfest Follow

Happy hacking! 🎃

Last year was a special HacktoberFest because I earned my t-shirt by doing my day job! That's right, I was working on Open Source projects for public sector clients and getting credit for HacktoberFest. 💪

This year I intended to carry on as usual, but for some reason (unknown to me) a couple of PRs from an old side project caught my eye. The project was a crate (Rust parlance for a package) called wifiscanner

booyaa / wifiscanner

A crate to list WiFi hotspots in your area

A quick look and I could see there was no reason to not merge them, they were passing their tests in Travis (CI/CD).

This got me thinking about improvements I could make to the project more friendly to contributors:

include a contribution guide.
introduce GitHub actions since it's my current hammer (see exhibits a, b, c and d) to eventually replace Travis.
add a checks target to the Makefile to format, lint and run tests before contributors submit a PR

Whilst all of these tasks could be done by me I did wonder, would other people like to do these tasks? So on a whim, I created issues and labelled them as HacktoberFest. Within a couple of hours, people had claimed the issues and a few days later fixes were issued.

I've been thinking about what were the contributing factors to my success. I think the quality of the issues played a significant part:

there was a brief explanation of the task
an acceptance criteria (with one or more check listed items)
the task could be worked on in isolation
tasks were relatively short (could be completed in a few minutes)

With all this collaboration, I felt energized to work on my side project again. Heck, I even tweeted about it.

// Detect dark theme var iframe = document.getElementById('tweet-1179113691679207424-540'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1179113691679207424&theme=dark" }

// Detect dark theme var iframe = document.getElementById('tweet-1180553738446135296-623'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1180553738446135296&theme=dark" }

Whilst this is great (I have missed working on side projects) there a little voice in my head that's telling me the reason for my new found love for this project is that I'm procrastinating because I'm currently job hunting! 😂

So if you're a remote-friendly company that's hiring for an SRE / DevOps role drop me a message (LinkedIn details are available in my profile)! I'd love to work magic on your systems!

GitHub template for Rust projects

Mark Sta Ana — Sat, 05 Oct 2019 21:38:52 +0000

Photo by Marian Kroell on Unsplash

I've just finished creating a GitHub template that you can use to set up a new Rust project that GitHub Actions for Rust as a CI pipeline.

What this means, is that your project from day one will always perform the following cargo commands against any code changes (PRs, pushes, etc):

check - Check a local package and all of its dependencies for errors
test - Execute all unit and integration tests and build examples of a local package
fmt - This utility formats all bin and lib files of the current crate using rustfmt
clippy - Checks a package to catch common mistakes and improve your Rust code

Here's how to use it

go to my template: https://github.com/booyaa/gh-actions-template-rust
click on the "Use this template" button
fill out the project details using the new project wizard
clone your new project
cd to your project
initialise the project using cargo init --vcs none or copy files from your existing Rust project
commit and push your changes up to GitHub
sit back and relax as GitHub Actions will now perform checks against any code changes submitted to your new Rust project

If you're wondering why I don't include any Rust scaffolding, it's because cargo already does a great job of providing this. I didn't want to perform updates as changes occur to the new project process owned by cargo.

As an added bonus you also get Rust's dual license of MIT and Apache v2!

I'd love to know what you think of this template. :)

GitHub template for Terraform projects

Mark Sta Ana — Mon, 30 Sep 2019 18:59:46 +0000

Photo by Shane McLendon on Unsplash

Hot on the heels of my last template for Rust, I've created a new one for Terraform. As before this gives you a new Terraform ready project that uses GitHub Actions for Terraform as a CI pipeline.

What this means, is that your project from day one will always perform the following Terraform commands against any code changes (PRs, pushes, etc):

fmt - Rewrites all Terraform configuration files to a canonical format.
init - Initialize a new or existing Terraform working directory by creating initial files, loading any remote state, downloading modules, etc.
validate - Validate the configuration files in a directory, referring only to the configuration and not accessing any remote services such as remote state, provider APIs, etc.

Here's how to use it

go to my template: https://github.com/booyaa/gh-actions-template-terraform
click on the "Use this template" button
fill out the project details using the new project wizard
clone your new project
cd to your project
initialise the project i.e. create a main.tf and add resources
commit and push your changes up to GitHub
sit back and relax as GitHub Actions will now perform checks against any code changes submitted to your new Rust project

If you're wondering why I don't include a main.tf, it's rare for terraform projects to have similar infrastructure. Also, I didn't see the point of sticking an empty terraform file for the sake of having one.

You maybe have noticed that plan and apply aren't included, these usually require secrets to be set i.e. API keys, Cloud Vendor access keys and again these are too specific to a given project.

Update: turns out init will be unhappy if there's any remote backends. I will write a new blog post to share my findings of wiring up this workflow to an existing remote backend on Azure Storage.

I'd love to know what you think of this template. :)

GitHub Actions: Rust edition

Mark Sta Ana — Sat, 28 Sep 2019 13:43:27 +0000

Photo by Zsolt Palatinus on Unsplash

I'm on a bit of a GitHub Actions deep dive this weekend. I tried GitHub's starter workflow for Rust but I was disappointed to discover the macOS virtual environment doesn't have the Rust toolchain.

Luckily the folks at action-rs have you covered. They've developed a bunch of actions for Rust's toolchain. All your favourites are there including Clippy!

Why do I need Rust on macOS? I like to test my code across as many platforms as possible. You can use a job strategy in your workflow to achieve this.

There's a handy quickstart guide on the action-rs "meta" GitHub repo.

p.s. Can you tell I'm super excited about GitHub Actions? Doing a lot of squeeing at the moment despite coming across the odd quirk.
p.p.s. There's a rather excellent blog post about the action-rs project by one of the main authors of action-rs: svartalf.info/posts/2019-09-16-github-actions-for-rust

ICYMI GitHub Actions v2 is a breaking change

Mark Sta Ana — Sat, 28 Sep 2019 10:45:18 +0000

Photo by Slim Emcee (UG) the poet Truth_From_Africa_Photography on Unsplash

On the 30th of September, the old HCL format will stop working. There's a handy migration tool you can download to migrate your existing workflows to the new YAML format.

It's ludicrously easy to use, and there's step by step instructions in the following GitHub help page

AWS DevOps Pro Certification Blog Post Series: Exam Time!

Mark Sta Ana — Tue, 09 Jul 2019 13:37:00 +0000

Photo by Ian Kim on Unsplash

Caveat emptor

Using AWS costs money, some of these services may not be part of the AWS Free Tier. You can keep costs down by tearing down anything you've created whilst learning, but it's still possible to run up a hefty bill so pay attention to the instances you setup!

I'm very lucky to be able to use my employer's AWS account. You should ask your place of work if a similar arrangement can be made as part of your study.

Velocius quam asparagi conquantur

The format of the blog posts is liable to change as I try to refine my mental model of each domain, so be sure to revisit the blog posts on a regular basis.

Exam Time!

At the end of June, I sat the AWS DevOps Profession exam and sadly readers I did not pass. I hadn't really expected to pass the first time, but I scored 69% (you need to score 75% to pass)!

After completing the exam you're given an immediate PASS or FAIL result. A few days later you get the actual score and areas that need improving.

Domains where I have demonstrated adequate knowledge

Configuration management and Infrastructure as Code
Monitoring and logging
Policies and Standards Automation

Domains where I need to brush up on

SDLC automation
Incident and Events response
High Availability, Fault tolerance and Disaster recovery

Things I wish I knew beforehand

The main and most obvious thing I should've done was to focus my knowledge around best practices to build infrastructures and utilising AWS services.

Most of the questions were of a similar format to the sample exam and mock exam. They wanted you to pick the best answer to meet the requirements of a customer.

There were so many services that I hadn't had hands-on experience, which meant I spent a lot of time just trying to gain a basic understanding of what each service did and their use cases. Turns out I needed to a deeper understanding of the domains I scored poorly against.

I think the only domain I was surprised at scoring poorly against was SDLC automation, as I do a lot of CI/CD on a daily basis, although this tends to be around non-AWS services: Circle CI, Travis CI and Azure DevOps.

So, how can you find out what the best practices are? There are probably two key documents:

The AWS whitepapers, which often contain solutions that combine various AWS products and services
The service's user or developer guide

My study before retaking exam will focus on this. I only spent the week before the exam skimming through the whitepapers, but this time I plan to make a lot more notes and try to memorise diagrams of solutions provided.

Tips for sitting the exam

Check your gear

The exam centres will require you to place all your belongings in a locker.

Some centres will allow you to bring cups of water to the exam, take two cups if you can carry them.

You'll also be given paper and pen, or something similar to make notes. Check the pen works before you start the exam!

Empty your bladder before you sit the exam, some places require you get the attention of the invigilator which can cost you precious minutes of your exam time.

Timekeeping

Start getting used to scanning questions in 2-minute intervals. This is the max you should spend on reading and answering a question.

In a perfect scenario where you could answer each question fully and not skip any, it would be easy to know the next 2-minute interval. In reality, you will probably skip or answer a question in less than 2 minutes, so whilst this might be obvious to most, I used the following to keep myself on track: as I started the next question I made a note of the current time on the exam countdown timer and worked out how long I had to answer the question, so if timer was at 1 hour and 7 minutes, I would need to answer the question on or before 1 hour and 5 minutes.

Skip and don't dither

Unless you're feeling particularly confident about your capabilities, you'll probably be in a blind panic (like I was). Give yourself up to 30 seconds (better if you can do it in less) to scan the question, if you don't even know where to begin then skip it. That time is better spent on questions where you have a vague notion of what the answer is.

Skipping questions is okay and you will find there's still time to revisit these questions once you've gone through all the questions.

Don't panic if you find you're skipping lots of questions either, it's probably your nerves.

Alternative study aids (new)

Whilst studying for this exam, I've used the following to help absorb the study material:

My AWS DevOps Pro Tinycards deck is a flashcard app by Duolingo. It's a bit rough and ready, but it's helped me retain some of the facts and figures that pop up.
I've also been listening to Last week in AWS which is a podcast by Corey Quinn that covers the ever-changing AWS landscape.

Until next time

I will, of course, let you know when I sit and pass the next exam (see what I did there). I'm aiming for mid-July so expect to hear from me soon!

Unsplash path (what terms I used to get to the cover image): failure

To go to the next part of the series, click on the grey dot below which is next to the current marker (the black dot).

AWS DevOps Pro Certification Blog Post Series: Study Gaps

Mark Sta Ana — Sat, 22 Jun 2019 13:37:00 +0000

Photo by Suad Kamardeen on Unsplash

Caveat emptor

I'm very lucky to be able to use my employer's AWS account. You should ask your place of work if a similar arrangement can be made as part of your study.

Velocius quam asparagi conquantur

The format of the blog posts is liable to change as I try to refine my mental model of each domain, so be sure to revisit the blog posts on a regular basis.

Study Gaps

This section will change a lot as I find new gaps whilst sitting in mock exams.

I've had a go at the sample exam under exam condititions (which before AWS made the exams adaptive would leave you with about 2 mins per question). Here's some items where I need to fill in gaps:

General

Knowing which services are able to use Resource Based Policies:
- Lambda (Configuration Management and Infrastructure as Code).
- ECR (via ECS - Configuration Management and Infrastructure as Code).
- CloudWatch Logs (Monitoring and Logging).
- AWS Secrets Manager (Policies and Standards Automation).

SDLC Automation

Need to read the blue/green whitepaper (SLDC automation). Pssst if you have the time you should read all the DevOps related whitepapers!

Blue/Green Techniques using CloudFormation or manually provisioned i.e. through AWS Console

This is based on the Blue/Green whitepaper.

Update DNS Routing with Amazon Route 53
- Setup
- Route 53 DNS
- Blue/Green Environments
  - Elastic Load Balancer (ELB)
  - Autoscaling group behind xthe ELB
  - Both environments are point to the same database instance (Amazon RDS Multi-AZ)
- Sub patterns
- Classic DNS pattern - Flip alias (live) record from blue to green
- Classic DNS-weighted distribution - Use split to send traffic to different environments
Swap the Auto Scaling Group Behind Elastic Load Balancer
- Setup
- Route 53 DNS
- ELB pointing to
- Blue and Green Auto Scaling Groups
- Both ASGs point to the same database instance (Amazon RDS Multi-AZ)
Update Auto Scaling Group Launch Configurations
- Setup
- Route 53 DNS
- ELB point to
- Auto Scaling Group containing
  - Blue Launch Config (LC)
  - Green Launch Config (LC)
- LCs are point to Amazon DynamoDB, Amazon RDS Multi-AZ or Amazon ElastiCache

There’s patterns for OpsWorks and Elastic Beanstalk, will add if I have time.

Configuration Management and Infrastructure as Code

Lambda
- Deploying new versions
- What triggers are available
- API Gateway
- AWS IoT
- Application Load Balancer
- CloudWatch Events (Monitoring and Logging)
- CloudWatch Logs (Monitoring and Logging)
- CodeCommit (SLDC automation)
- Cognito Sync Trigger
- DynamoDB (High Availability, Fault Tolerance, and Disaster Recovery)
- Kinesis (Incident and Event Response)
- Also doesn't hurt to know the following services are supported: S3,SNS and SQS

Monitoring and Logging

CloudWatch events for the services covered in the exam
- SDLC Automation
  - CodeCommit
  - CodeBuild
  - CodeDeploy
  - CodePipeline
- Configuration Management and Infrastrcuture as Code
  - AWS Config
  - AWS OpsWorks
  - AWS (Lambda) Step Functions
  - AWS ECS
- Monitoring and Logging
  - CloudWatch (scheduled events)
- Policies and Standards Automation
  - Amazon Macie
  - AWS Systems Manager
    - Configuration Compliance
    - Maintenance Windows
    - Parameter Store
  - Trusted Advisor
- Incident and Event Reporting
  - Amazon GuardDuty
- Fault Tolerance, High Availability and Disaster Recovery
  - Amazon EC2 Auto Scaling
CloudWatch Event Rule Targets
- SDLC Automation
- Code Build
- Code Pipeline
- Configuration Management and Infrastructure as Code
- Lambda (and Step) function
- Incident and Events Reporting
- Kinesis
  - Data Streams
  - Data Firehose
- Amazon Inspector
- Policies and Standards Automation
- Systems Manager
  - Run Command
  - Automation
- Nice to knows: SNS and SQS

Fault Tolerance, High Availability and Disaster Recovery

RDS
- snapshots and their use in a DR situation. (High Availability, Fault Tolerance, and Disaster Recovery).
- Understanding Recovery Time Objective (RTO) and Recovery Point Objective (RPO) with DR in mind. (High Availability, Fault Tolerance, and Disaster Recovery).

Policies and Standards Automation

AWS Systems Manager - EC2 patch groups and Patch Manager's baselines (Policies and Standards Automation)
AWS Service Catalogue - how to offer products that provide different tiers (web, web + db) or stacks (.NET or Ruby) (Policies and Standards Automation)

Unsplash path (what terms I used to get to the cover image): gap

To go to the next part of the series, click on the grey dot below which is next to the current marker (the black dot).

AWS DevOps Pro Certification Blog Post Series: Databases

Mark Sta Ana — Fri, 14 Jun 2019 13:37:00 +0000

Photo by Jan Antonin Kolar on Unsplash

Caveat emptor

I'm very lucky to be able to use my employer's AWS account. You should ask your place of work if a similar arrangement can be made as part of your study.

Velocius quam asparagi conquantur

The format of the blog posts is liable to change as I try to refine my mental model of each domain, so be sure to revisit the blog posts on a regular basis.

What?

Amazon RDS is a managed service for Relational Database engines Service (RDS). AWS supports the following engines:

MySQL is an open source database engine
MariaDB is a fork of MySQL when the former was acquired by Oracle (through the acquisition of Sun Systems)
PostgreSQL is an open source database engine
Oracle is a commercial engine by Oracle
SQL Server is a commercial engine by Microsoft
Amazon Aurora is a MySQL / Postgres compatible relational database engine

Amazon DynamoDB is a proprietary NoSQL database service offered by Amazon.

Why?

Managed services for database engines like all managed service remove key concerns:

provisioning/scaling/termination of servers that host the database engines
maintenance of servers (patching)
backups and restores

These concerns often affect the ability to provide a database server that is fault-tolerant, highly available and has a contingency for disaster recovery.

N.B. there's a caveat around storage scaling that it won't be applicable to SQL Server instances. The details can be found in the SQL Server FAQs: Why can’t I scale my storage?.

To understand the difference between Amazon RDS and Amazon DynamoDB, I've provided the following examples:

Relational database engines (sometimes referred to as RDBMS) are tabular in natural, you can think of it visually as a spreadsheet with fields as the column headers and the rows being a single record of data. The term "relational" comes from the ability to link tables through a foreign key, an example of this might be a list of developers and their favourite foods. The link would be a column in the developers table called food_id, which would reference the column called id in the food table.

developer(s) table

id	name	food_id
1	alice	1
2	bob	1
3	carol	2

food(s) table

id	name
1	banana
2	nuts

If you were to delete nuts (id: 2) from the food table you would trigger an error warning there were dependencies in the developer table.

The key takeaway from this example is to remember Relational Databases store records tabularly in rows.

NoSQL database engines are a bit of a catch-all, but in essence, if you don't store your data tabularly you're probably a NoSQL database engine. Amazon DynamoDB is a Key/Value pair and Document store. Key/Value store allows you to store data like a Hash (associative array/dictionary), you provide a key and the value is returned, you may have used one without knowing as they're often referred to as cache servers i.e. Redis. Document stores allow you to data in a structured way common formats are XML and JSON, often these are the database engines most people associate with NoSQL i.e. CouchDB and MongoDB.

When?

Amazon Aurora provides a compatible engine that is 3x faster than PostgreSQL and 5x faster than MySQL. In terms of cost-effectiveness, you need to compare the other RDS engines as a multi-AZ deployment and memory optimised instances.

Things that set it apart from the other RDS offerings in terms of this domain are:

Aurora can failover over to 1 of (up to) 15 read replicas with low impact to the primary instance.
- You can use MySQL replicas instead of Aurora native, but you're limited to 5 replicas and there's a high impact to the primary instance.
- The order which replicas are promoted to primary can be customised.
Data is stored in 10GB chunks with 6 copies replicated across three availability zone.
- Aurora will continue to handle:
- write capability with the loss of 2 copies of data
- read capability with the loss of 3 copies of data
The data blocks and disks are scanned for errors and repaired automatically

Amazon RDS can be part of your disaster recovery strategy by keeping replicas of your production Oracle or SQL Server database servers.

Amazon DynamoDB requires a lot more consideration to make use of its features that make it relevant to this domain. Choosing the wrong scheme for partition keys can see your database starved of I/O.

Things to consider:

You pay for the number of I/O you use (rather than instance size) the size of units is in kilobytes and vary depending on the type of I/O operation:
- Read Capacity Unit (RCU) are measured in 4KB blocks, so an 8KB block of data would consume 2 RCUs
- Write Capacity Unit (WCU) are measured in 1KB blocks, so a 5KB block of data would consume 5 WCUs
- Data in tables are stored as 10GB partitions that can handle 3K RCU and 1K WCU
- As you expand the table into more 10GB partitions, the RCU and WCU are distributed across the partitions i.e. if you 2 partitions then you have a max of 1.5K RCUs and 0.5K WCUs across the 2 partitions.
Terminology
- Table (highest level unit for Dynamo DB)
- Item (a record)
- Attributes (columns or fields of a record)
- Primary Keys can consist of
- just a Partition Key (which is often referred to as a Primary Index and is used to query the table)
- the partition key and sort key this is known as a composite primary key
- Secondary Indexes (allows you to query on a different attribute)
- Local - requires the same partition key, but the sort key can be different
- Global - can have a different attribute for the partition and sort keys
Strategies for write-heavy use cases it's recommended you add a randomly generated number from a predetermined range. e.g. if Partition Key is a composite attribute based on invoice number (1234), then you would suffice the randomly generated number (1) to the end, so the composite key would be: 1234-1.

How?

Amazon Aurora requires you to choose your engine compatibility i.e. MySQL or PostgreSQL after which you define:

Create DB Cluster Parameter Group (parameters to be applied to all instance in a DB cluster)
Create DB Cluster (the group of instances associated with a DB cluster)
Create the Database Instance (adds a new instance to a DB cluster)

The AWS CLI features a cli walkthrough of how to provision a table, store an item, perform a query. It should be noted that as a rule of thumb you would probably use the [AWS SDK][aws_sdk] to store and retrieve data from DynamoDB.

API and CLI features and verbs

Amazon RDS

Features

DB (Cluster) Parameter Group
Db Cluster
DB Instance
DB (Cluster) Snapshot

Verbs (CRUD)

create/copy
describe
modify
delete

Outliers

Not my best work will see if I can optimise this list.

add-option-to-option-group
add-role-to-db-cluster
add-role-to-db-instance
add-source-identifier-to-subscription
add-tags-to-resource
apply-pending-maintenance-action
authorize-db-security-group-ingress
backtrack-db-cluster
copy-option-group
create-db-cluster-endpoint
create-db-instance-read-replica
create-db-security-group
create-db-subnet-group
create-event-subscription
create-global-cluster
create-option-group
delete-db-cluster-endpoint
delete-db-instance-automated-backup
delete-db-parameter-group
delete-db-security-group
delete-db-snapshot
delete-db-subnet-group
delete-event-subscription
delete-global-cluster
delete-option-group
describe-account-attributes
describe-certificates
describe-db-cluster-backtracks
describe-db-cluster-endpoints
describe-db-cluster-parameters
describe-db-cluster-snapshot-attributes
describe-db-engine-versions
describe-db-instance-automated-backups
describe-db-log-files
describe-db-parameter-groups
describe-db-parameters
describe-db-security-groups
describe-db-snapshot-attributes
describe-db-subnet-groups
describe-engine-default-cluster-parameters
describe-engine-default-parameters
describe-event-categories
describe-event-subscriptions
describe-events
describe-global-clusters
describe-option-group-options
describe-option-groups
describe-orderable-db-instance-options
describe-pending-maintenance-actions
describe-reserved-db-instances
describe-reserved-db-instances-offerings
describe-source-regions
describe-valid-db-instance-modifications
download-db-log-file-portion
failover-db-cluster
generate-db-auth-token
list-tags-for-resource
modify-current-db-cluster-capacity
modify-db-cluster-endpoint
modify-db-cluster-snapshot-attribute
modify-db-snapshot-attribute
modify-db-subnet-group
modify-event-subscription
modify-global-cluster
promote-read-replica
promote-read-replica-db-cluster
purchase-reserved-db-instances-offering
reboot-db-instance
remove-from-global-cluster
remove-option-from-option-group
remove-role-from-db-cluster
remove-role-from-db-instance
remove-source-identifier-from-subscription
remove-tags-from-resource
reset-db-cluster-parameter-group
reset-db-parameter-group
restore-db-cluster-from-s3
restore-db-cluster-from-snapshot
restore-db-cluster-to-point-in-time
restore-db-instance-from-db-snapshot
restore-db-instance-from-s3
restore-db-instance-to-point-in-time
revoke-db-security-group-ingress
start-activity-stream
start-db-cluster
start-db-instance
stop-activity-stream
stop-db-cluster
stop-db-instance
wait

Amazon DynamoDB

Features

Item
Backup
(Global) Table

Verbs (CRUD)

create (global table, table and backup)
describe/list (global table, table and backup), get-item, batch-get-item
update (global table, table and backup), put-item, batch-put-item
delete

Outliers

describe-continuous-backups
describe-endpoints
describe-global-table-settings
describe-limits
describe-time-to-live
list-tags-of-resource
query
restore-table-from-backup
restore-table-to-point-in-time
scan
tag-resource
transact-get-items
transact-write-items
untag-resource
update-continuous-backups
update-global-table-settings
update-time-to-live
wait

Unsplash path (what terms I used to get to the cover image): database

To go to the next part of the series, click on the grey dot below which is next to the current marker (the black dot).

AWS DevOps Pro Certification Blog Post Series: Amazon Single Signon, CloudFront, Autoscaling and Route53

Mark Sta Ana — Wed, 12 Jun 2019 13:37:00 +0000

Photo by Todd Quackenbush on Unsplash

Caveat emptor

I'm very lucky to be able to use my employer's AWS account. You should ask your place of work if a similar arrangement can be made as part of your study.

Velocius quam asparagi conquantur

The format of the blog posts is liable to change as I try to refine my mental model of each domain, so be sure to revisit the blog posts on a regular basis.

What?

Amazon Single Sign-On is a managed single sign-on (SSO) service that you can use to simplify access to applications and 3rd party services. If SSO is not a term you're familiar with, if you've ever signed up for a service using your Google, Facebook or Twitter account (instead of using your email address and password specific to that site) then you've used SSO.

Amazon CloudFront is a managed Content Delivery Network (CDN) service, you may have heard of CloudFront's competitors like CloudFlare, Akamai and Fastly. CDN speed up your website performance by strategically placing mirrors of popular content (static files, API or streaming audio/video) at locations nearer to the user accessing your website. These mirrors are referred to as Edge locations popular content for the region (not specific to client) is cached here. In more densely populated areas there are also Regional Caches that hold content for longer than Edge locations.

Amazon Route53 is a managed Domain Name Service (DNS). At its very basic level of functionality DNS servers allow you to connect to servers using friendly domain names i.e. dev.to rather than IP addresses like 151.101.123.4, 151.101.12.34, 151.101.1.234. It's designed to work with other Amazon Web Services that is you can point DNS records directly to Elastic Load Balancer, S3 and EC2 instances.

Autoscaling as we saw in the Domain intro comes in two varieties: AWS Auto Scaling and Amazon EC2 Auto Scaling. The general rule of thumb for use if you want to just autoscale EC2 instances then use EC2 Auto Scaling service, otherwise, AWS Auto Scaling is a better use case for when you want to scale multiple resources (not just EC2) i.e. DynamoDB tables and indexes, ECS tasks.

An important thing to note that to use AWS Auto Scaling, your resource must be created using CloudFormation or Elastic Beanstalk.

For the rest of this post, we'll only be referring to Amazon EC2 Auto Scaling.

Why?

Amazon Single Sign-On or generically any single sign-on (SSO) service is better than managing the administrative overhead of keeping separate logins for each application / service, you reduce the impact on day to day operations should disaster strike (think the number of helpdesk tickets will be raised for DR systems that rarely get used). You'll also get the undying love of your users because it means fewer logins to track, which in turn means they will be less likely to keep a scrap paper lying around their desk with the various logins and passwords written down.

Amazon CloudFront distributes your content geographically rather than storing in a single location or S3 bucket. By careful design (falling back graceful should the backend be unavailable) ensures your website is highly available.

Amazon Route 53 provides the following routing policies whose attributes are suitable for this domain:

Failover routing - used for active-passive failover, a good use case for automated disaster recovery.
Geolocation routing - used to route traffic based on the location of users, a good use case for highly available
Geoproximity routing - similar to Geolocation routing, but also allows you to route to a secondary location. This also makes a good use case for fault tolerance.
Latency-based routing - used to route users to the resources with the best (least) latency
Multivalue answer routing - this is similar to round robin, in that you can randomly pick a route from up to eight healthy resources
Weighted routing - routes traffic to different resources using a percentage split (useful for A/B testing or load balancing). Weights are between 0 to 255.

Amazon EC2 Auto Scaling allows you to launch or terminate a number of EC2 instances by defining conditions when scaling out (increase) or scaling in (decreasing) the number of instances. The condition might be metrics like CPU or Memory utilisation, or health checks. This combined with an elastic load balancer provides a system that can be highly available and fault tolerant.

Terminology to be aware of:

Auto Scaling Group (ASG) - this is a group of EC2 instances associated with one or more scale in/out conditions.
- Minimum size - that the ASG never goes below
- Maximum Size - that the ASG never goes above
- Desired capacity - that the ASG will already try to maintain (unless the condition requires further scaling-in).
- Scaling capacity is between desired capacity and Maximum Size

When?

Amazon Single Sign-On should ideally be implemented as soon as possible, but it's still possible to retrofit into an existing environment. Doing this soon rather than later could mean you're not having to re-organise the team who are responsible for user and access management if the headcount reduces because of efficiency savings through the implementation of SSO.

Amazon CloudFront should be implemented once you have some metrics (via Amazon X-Ray or something similar) to indicate you have customers in regions that are experiencing poor response times because of their proximity in relation to the region where your load balancers, EC2 instances or S3 buckets are hosted.

Amazon Route 53's routing policy provides a lot of desirable features that are relevant for this domain. This combined with the fact that Amazon also offers an [SLA] of 100% availability and the ability to create and modify DNS record programmatically mean the use of Route 53 is a bit of a no-brainer.

Amazon EC2 Auto Scaling has the concept of lifecycle hooks. These allow you to perform custom actions by pausing the instances as the ASG launches (EC2_INSTANCE_LAUNCHING) or terminates (EC2_INSTANCE_TERMINATING) them. Whilst the instance is paused, it is in a wait state until you complete the action by issuing the complete-lifecycle-action action in the CLI/API or the timeout period ends (one hour by default). You can extend the timeout period by either:

set a longer heartbeat timeout period using the put-lifecycle-hook action (CLI/API) when you create the lifecycle hook
restart the timeout period using record-lifecycle-action-heartbeat action (CLI/API)

The maximum time you can place an instance in a wait state is 48 hours or 100 times the heartbeat timeout (whichever is smaller).

How?

Amazon Single Sign-On requires an AWS Organization to exist and then you can enable single sign-on via the AWS Console. The specifics for setting up the service with the AWS Account or Cloud Applications (3rd party services) can be found in the [guide][sso_guide]. There is an option to link to your existing Microsoft Active Directory, but if you don't need this option then the service will use its own directory.

Amazon CloudFront to setup you define a distribution that determines the content origins (S3 bucket or HTTP server), access, security (TLS/SSL/HTTPS), session/object tracking, geo restrictions and logging. The provisioning of CloudFront can take a while as the content is being distributed to edge locations.

I've found the following article in the AWS blog very helpful in terms of an application that I was already familiar with, but also knew the difficulty in optimising for response time: How to accelerate your WordPress site with Amazon CloudFront

Amazon EC2 Auto Scaling terminology:

Auto-scaling group - this is a collection of EC2 instances that will be scaled in or out depending on conditions defined
- There's a minimum size
- Desired capacity
- Max capacity
Autoscaling lifecycle
- starts when an ASG launches an instance
- ends, when you terminate an instance or ASG, takes an instance out of service and terminates it
Cooldowns prevents the ASG from launching or terminating more instances before the previous scaling activity event has taken effect. The default period is 300 seconds (5 minutes)

API and CLI features and verbs

Amazon Single Sign-On

This service has no API/CLI.

Amazon CloudFront

Features

(Streaming) Distributions (this is probably the most important one to be aware of) with or without tags
Field Level Encryption (Config/Profile)
Invalidation (cache)
(CloudFront) Origin Access Identity
Public key

Verbs (CRUD)

create (distrbution/streaming-with-tags)
get/list
update (except invalidation)
delete (except invalidation)

Outliers

get-field-level-encryption-profile-config
get-distribution-config
get-public-key-config
get-cloud-front-origin-access-identity-config
get-streaming-distribution-config
list-distributions-by-web-acl-id
list-tags-for-resource
sign
tag-resource
untag-resource
wait

Amazon Route 53

I've opted for the main API/CLI for Route 53 instead of Domains and Resolvers as I've been using this more on a day to day basis.

Features

Health Check
Hosted Zone
Reusable Delegation Set
Traffic Policy (Instance/Version)
Create Query Logging Config

Verbs (CRUD)

create
get/list
update
delete

Outliers

CreateVPCAssociationAuthorization
AssociateVPCWithHostedZone
DeleteVPCAssociationAuthorization
ChangeResourceRecordSets
ChangeTagsForResource
DisassociateVPCFromHostedZone
GetAccountLimit
GetChange
GetCheckerIpRanges
GetGeoLocation
ListGeoLocations
TestDNSAnswer

EC2 Auto Scaling

Features

The API has a lot of features, but the API actions I've focussed on have been around the Lifecycle Hooks.

Verbs (CRUD)

describe (types)
put
delete

Outliers

CompleteLifecycleAction
RecordLifecycleActionHeartbeat

Unsplash path (what terms I used to get to the cover image): random, miscellany, junkyard, collection

To go to the next part of the series, click on the grey dot below which is next to the current marker (the black dot).

AWS DevOps Pro Certification Blog Post Series: High Availability, Fault Tolerance and Disaster Recovery

Mark Sta Ana — Sat, 08 Jun 2019 10:32:02 +0000

Photo by Emiel Molenaar on Unsplash

What does the exam guide say?

To pass this domain, you'll need to know the following:

Determine appropriate use of multi-AZ versus multi-region architectures
Determine how to implement high availability, scalability, and fault tolerance
Determine the right services based on business needs (e.g., RTO/RPO, cost)
Determine how to design and automate disaster recovery strategies
Evaluate a deployment for points of failure

This domain is 16% of the overall mark for the exam.

What whitepapers are relevant?

According to the AWS Whitepapers page we should look at the following documents:

What services and products covered in this domain?

AWS Single Sign-On is Amazon's managed SSO service allow your users to sign in to AWS and other connected services using your existing Microsoft Active Directory (AD).
Amazon CloudFront is a managed Content Delivery Network (CDN) service.
Autoscaling resources - Amazon has two offerings Amazon Autoscaling and Amazon EC2 Auto Scaling
Amzon Route 53 is a managed Domain Name Service (DNS).
Databases
- Amazon RDS is a managed relational database service with a large choice of engines: Amazon Aurora, PostgreSQL, MySQL, MariaDB, Oracle Database and SQL Server.
- Amazon Aurora is part of the RDS offering but is unique in that it provides compatibility with MySQL and PostgreSQL engines whilst outperforming them considerably (5x for MySQL and 3x for PostgreSQL).
- Amazon DynamoDB is a managed NoSQL (non-relational) database service that can be used for storing key-value pairs or document based records.

What about other types of documentation?

If you have the time, by all means, read the User Guides, but they are usually a couple of hundred pages.

Amazon Single-Sign On
Amazon CloudFront
Amazon Autoscaling and Amazon EC2 Autoscaling
Amazon Route53
Databases

Alternatively, get familiar with the services using the FAQs:

Amazon Single-Sign On
Amazon CloudFront
Amazon Autoscaling and Amazon EC2 Autoscaling
Amazon Route53
Databases

You're all expected to know the APIs

Amazon CloudFront
Amazon Autoscaling and Amazon EC2 Autoscaling
Amazon Route53
Databases
- Amazon RDS
- Amazon Aurora uses the same API as RDS
- Amazon DynamoDB

Before you panic, you'll start to spot a pattern with the API verbs.

And the CLI commands

Amazon CloudFront
Amazon Autoscaling and Amazon EC2 Autoscaling
Amazon Route53 has three subcommands: DNS and Healthchecking, Service Discovery and Domain Registration
Databases
- Amazon RDS
- Amazon Aurora uses the same CLI as RDS
- Amazon DynamoDB has two sub commands: dynamodb and dynamodbstreams

As with the API, there are patterns to the commands.

High Availability, Fault Tolerance and Disaster Recovery, oh my!

Let's the basics out of the way and discuss the core concepts around this domain.

I'm going to use an excellent example provided by Patrick Benson in his blog post: The Difference Between Fault Tolerance, High Availability, & Disaster Recovery

An airplane has multiple engines and can operate with the loss of one or more engines. The design of the airplane has been made it resilient to falling out of the sky because of engine failure. This design is fault tolerant.

In terms of infrastructure, this is likely to be a managed service like RDS, where under the hood the database engine has multiple disks and CPUs to cope with catastrophic failure.

Whereas spare tire in car, isn't fault tolerant i.e. you have to stop change the tire, but having the spare tire in the first place makes the car still highly available. In terms of infrastructure is any type of technology like an autoscaling group.

It's very common for a solution to implement a system that is fault tolerant (resilience) and highly available (scalable).

Finally, ejector seats in Fighter aircraft are disaster recovery (DR) measure. The goal is to preserve the pilot, or in our case, the service after all other measures have failed (Fault Tolerance and HA).

Often in terms of infrastructure, this might be a standby infrastructure or database replica in a different AWS region and using Route 53 to point to the stand by infrastructure. Whilst it's still common for DR strategies to be manual, for this domain we'll be expected to provide an automated solution.

Unsplash path (what terms I used to get to the cover image): airplane

To go to the next part of the series, click on the grey dot below which is next to the current marker (the black dot).