Forem: Jenn

Using Lambda to rewrite URLs for Hugo

Jenn — Tue, 18 Aug 2020 04:38:15 +0000

Hugo by default uses pretty URLs which work out of the box in most hosting environments. Using AWS Cloudfront in front of S3 causes it to not automatically translate pretty urls.

Instead of setting uglyurls = true in your site config, use AWS Lambda to translate for you.

There are example Lambda@Edge functions out there, I used this Medium post and an old AWS Compute Blog to build my function.

'use strict';
const path = require('path')
exports.handler = (event, context, callback) => {
    // Extract the request from the CloudFront event that is sent to Lambda@Edge 
    var request = event.Records[0].cf.request;
    // Extract the URI from the request
    var olduri = request.uri;
    // If last element of old uri doesn't contain a .html extension, assume it's a directory and append a '/'
    if (!olduri.match(/\/$/) && path.basename(olduri).match(/^[^\.]+$/)){
        olduri = olduri + '/';
    }
    // Match any '/' that occurs at the end of a URI. Replace it with a default index
    var newuri = olduri.replace(/\/$/, '\/index.html');
    // Log the URI as received by CloudFront and the new URI to be used to fetch from origin
    console.log("Old URI: " + olduri);
    console.log("New URI: " + newuri);
    // Replace the received URI with the URI that includes the index page
    request.uri = newuri;
    // Return to CloudFront
    return callback(null, request);
};

The main gotcha is you may get this weird looking error when publishing the function if you are using the web interface.

Your function's execution role must be assumable by the edgelambda.amazonaws.com service principal.

Lambda can create an IAM role for you, but you still have to go into the newly created role and edit the Trust relationships policy document to allow Lambda@Edge access.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": [
          "lambda.amazonaws.com",
          "edgelambda.amazonaws.com"
        ]
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

After editing the policy document for trust relationships, REFRESH the tab/window you are publishing the function from to force it to see the new permissions.

Building and deploying a Hugo site with CodeBuild

Jenn — Mon, 17 Aug 2020 01:46:43 +0000

If you follow along in the Hugo documentation there are many way to deploy your site. I already had a CodePipeline and CodeBuild set up for my previous Gatsby blog, so I decided to leverage that instead.

The biggest hurdle in doing this, was figuring out what changes I needed to make to my CodeBuild project to make it work with Hugo instead of Gatsby. I searched online until I found a blog post with an example buildspec.yml for me to work from.

I made a few updates to the buildspec.yml and then deployed my site with the following.

version: 0.2

env:
  variables: {}
  parameter-store: {}

phases:
  install:
    commands:
      - echo Entered the install phase...
      - apt-get -qq update && apt-get -qq install curl
      - apt-get -qq install asciidoctor
      - curl -s -L https://github.com/gohugoio/hugo/releases/download/v0.74.3/hugo_0.74.3_Linux-64bit.deb -o hugo.deb
      - dpkg -i hugo.deb
    finally:
      - echo Installation done
  build:
    commands:
      - echo Entered the build phase ...
      - echo Build started on `date`
      - cd $CODEBUILD_SRC_DIR
      - rm -f buildspec.yml && rm -f .git && rm -f README.md
      - hugo
    finally:
      - echo Building the HTML files finished
  post_build:
    commands:
      - echo Entered the post_build phase...
      - echo Build completed on `date`
      - aws s3 sync public/ s3://bucket-name --region region --acl public-read --delete
      - aws cloudfront create-invalidation --distribution-id XX --paths '/*'
artifacts:
  files: []
secondary-artifacts: {}
cache:
  paths: []

I updated my buildspec.yml to use the current version of Hugo 0.74.3 and due to some troubleshooting issues, made the build of the site not be silent.

The biggest change is that I am leveraging the post_build phase instead of sending an artifact back out into the CodePipeline. I sync the built public/ folder with my S3 bucket, deleting everything in the bucket, copying over the files, and making sure everything is public-read. The last line in the phase does a cache invalidation for my CloudFront distribution.

Now whenever I push to the main branch on my Git repo, the site is built and deployed automatically for me. The only issue left is that S3 does not play nicely with "pretty" URLs and I ended up setting my blog to use ugly URLs until I could make a Lambda function to fix it, which will be my next blog post.

How to convert a simple blog from Gatsby to Hugo

Jenn — Sun, 16 Aug 2020 05:13:28 +0000

I updated my hobby blog to Hugo today. Hugo is a static site generator written in Go and it is very fast. I created that blog with Gatsby several years ago and it needed a refresh.

Switching from Gatsby to Hugo is not bad. Both use the same style of front matter and Markdown so I didn't have rewrite everything. As they are written in different programming languages, I could work on converting everything while leaving Gatsby alone.

Adding Hugo

Backup everything or create a new branch.
Rename the content folder to content1.
Run the following command to generate a new Hugo site in the existing web site folder (it will not work if a content folder already exists).
```
hugo new site . --force
```

Add a theme, I used Hermit

git clone https://github.com/Track3/hermit.git themes/hermit

Add the theme to the site config.
```
echo 'theme = "hermit"' >> config.toml
```
Start the Hugo server.
```
hugo server -D
```
Start moving content.

Moving content

In Gatsby, all my blog post files were named like the following: content\blog\postname\index.md. In Hugo, the postname folder was not needed so I began the slog of moving and renaming my files.

Create the folder posts under content.
Copy a blog post from content1 to the new posts folder.
Rename index.html to the desired slug name for the post.
Repeat.

Writing new content

I was not able to move my About and Equipment pages, as they were written in JavaScript. I had to write new Markdown files for them but writing two pages in Markdown is much better than rewriting a whole site.

Gotchas

I had to also move my static content from src folder to a new top level folder called static.

As my blog content had a lot of embedded tweets, I had to change all of the references over to a shortcode.

Deleting Gatsby

Once I was happy that all my content was moved over and appearing in Hugo, I began deleting all the Gatsby files.

I removed:

gatsby-browser.js
gatsby-config.js
gatsby-node.js
gatsby-ssr.js
package-lock.json
package.json
src/
yarn.lock
content1/

Converting my blog was a bit tedious, but I think it was worth it. I was able to correct spelling mistakes that had been up for years and add tags to the posts.

It only took 8 hours, 20 commits, and 7 builds.

Building an RSS reader in Javascript

Jenn — Thu, 16 Jul 2020 19:25:21 +0000

Blogging never went away and neither did RSS feeds. RSS (Really Simple Syndication) is a web feed to check for updates on sites. It used to be quite popular with several different apps devoted to reading these feeds. In addition, many browsers used to have RSS readers built in. Sadly, RSS has fallen out of popularity probably due to social media and other feeds taking over its role.

But last night on a whim, I decided to build a personal RSS reader with Vanilla JS. Okay, it wasn't a whim, Twitter was on fire and what better way to distract myself from my usual distraction than creating a new web application?

The tools

Setting up the project

Whenever I start a new project, I look for examples of the idea or similar ideas to build from. I did a search for "rss reader javascript" and I came across several older projects and tutorials. Many of them were written in older syntax or used frameworks. I found one tutorial that used the (then new) Fetch API and decided to build on that.

Finding RSS feeds

Finding feeds is harder than it used to be back in 2006. I searched online for example feeds to pull from and decided on dev.to's feed, Codepen's feed, and Mozilla Hacks Blog. I tested that I could reach all the feeds in the browser and was served the appropriate XML.

An example RSS XML document from my personal dev.to feed.

<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0">
  <channel>
    <title>Jenn</title>
    <author>Jenn</author>
    <description>Jenn is a self taught web developer who specializes in usability and accessibility.  She is easily spotted at conferences by her bright lipstick and various code dresses and t-shirts.</description>
    <link>https://dev.to/geekgalgroks</link>
    <language>en</language>
    <item>
    ...
    </item>
  </channel>
</rss>

Database

I wanted a way to easily update my list of feeds without having to update my code. I used Google's Firestore for several other little projects and spun up a new collection called rssfeed. I decided the only thing I needed was the url and added four RSS feed urls to the collection.

Diving straight in

As I had written other little Javascript web apps that used Firestore, I started by copying what I did in that project.

I created a global variable to hold my feeds and queried the database to push the URL values into it.

    const database = firebase.firestore().collection('rssfeed');
    database.get().then((querySnapshot) => {
        querySnapshot.forEach((doc) => {
            feeds.push({
                id: doc.id,
                url: doc.data().url
            });
        });
    });

First problem

I was getting 404 errors in my console. I realized I forgot to set the Firestore database rules to allow reading of the collection.

I copied the rules of a previous collection and after waiting a bit, confirmed they worked.

    match /rssfeed/{feedId} {
        allow read;
        allow write: if request.auth.uid == 'REDACTED';
    }

I was now able to console log the value of the array and confirm everything was working.

Doing too much at once

Spurred on by new success I continued on. I built a function that used Fetch to get the title of a feed. I used a foreach loop on my array and called it.

I got a bunch of odd errors.

CORS and Promises

The first error message that made sense in the console was about CORS.

CORS

CORS stands for Cross Origin Resource Sharing. CORS protects sites from calling assets (Javascript, images, apis, etc) from other websites. Some sites protect all their assets, others explicitly let others use some or all of them.

Some of the feeds were being protected by CORS.

At the time I thought it was all the feeds. I looked up how to add CORS modes to my Fetch call.

// No CORS, this is an "opaque" mode that limits what headers are sent.
fetch(feed, {mode: no-cors});

This didn't help. I started looking at proxies and other solutions.

Searching again

I was frustrated. I searched again for projects. Looking for something newer that might give me insight on how to combat CORS.

I stumbled across CSS-Tricks How to Fetch and Parse RSS Feeds in JavaScript. It had a working example and was written in 2020!

I commented out all of my code and pasted their example in, everything worked. I changed the hardcoded URL from Codepen to my dev.to feed, everything still worked. I wrapped the fetch call in a function and tested again, it worked. I was feeling great. I added back in my database call and using a foreach on my array, called the function.

It didn't work because my array wasn't populated yet, it just held promises.

Promises

Promises are placeholders. Asynchronous functions return promises instead of blocking everything on the page while they work. The function promises to get you a result.

My array was full of promises. Fetch couldn't pull down content from a promised URL, it needed the real thing.

This is where then comes in handy. It waits until the asynchronous function completes and then does the next thing. I removed my global variable (should not have made it a global anyway), moved the return statement up on my database call, and chained in my fetch call.

It worked!

Except I had three results, not four.

CORS strikes again

The Mozilla blog was protected by CORS. Instead of fighting it more, I just removed the url from my database. Some battles are not worth fighting.

The final code

My completed reader can be found on my portfolio site. I have included an HTML snippet and the full javascript file below. CSS is omitted because not everyone loves pastels.

HTML Snippet

<main id="content">
      <h1>Jenn's Glorious RSS Reader</h1>
      <p>Implemented in Vanilla JS because blogging never died.</p>
</main>

JS

function getRssFeed(feed) {
    fetch(feed)
        .then(response => response.text())
        .then(str => new window.DOMParser().parseFromString(str, "text/xml"))
        .then(data => {
            const items = data.querySelectorAll("item");
            let html = ``;
            html += `<h2>${data.querySelector("title").innerHTML}</h2>`;
            html += `<p>${data.querySelector("description").innerHTML}</p>`;
            html += `<div class="feeds">`;
            items.forEach(el => {
                html += `
        <article>
          <h3>
            <a href="${el.querySelector("link").innerHTML}" target="_blank" rel="noopener">
              ${el.querySelector("title").innerHTML}
            </a>
          </h3>
        </article>
      `;
            });
            html += `</div>`;
            document.getElementById('content').insertAdjacentHTML("beforeend", html);
        });
}
function getFeeds() {
    let feeds = [];
    const database = firebase.firestore().collection('rssfeed');
    database.get().then((querySnapshot) => {
        querySnapshot.forEach((doc) => {
            feeds.push({
                id: doc.id,
                url: doc.data().url
            });
        });
        return feeds;
    }).then(function (feeds) {
        displayFeeds(feeds);
    });
}
function displayFeeds(feeds) {
    feeds.forEach(feed => { getRssFeed(feed.url); });
}
getFeeds();

All in all, it took around four hours to write. Much of that time was troubleshooting and research. It probably would have been faster if I wasn't tired and didn't try to do too many things at once in the beginning.

Adventure in restoring WordPress to a Docker container

Jenn — Sun, 28 Jun 2020 21:05:23 +0000

Today's adventure was harrowing, fraught with errors, and shouts of victory!

Backup

First, I downloaded a backup of my site. The backup included the database and items in wp-content folder (e.g. plugins, themes, and uploads). My backup was compressed into a single file, I extracted the files locally on my machine.

Docker-compose

I followed the quickstart to create my containers for WordPress. After typing docker-compose up I completed the install of WordPress. I now had a clean install to restore my site into.

Plugins, themes, and uploads

Restoring everything in the wp-content folder was a snap. I copied everything over from my download making sure the folder structure was correct. WordPress picked up the themes and plugins right away.

Database

Each table in my database was a separate .sql file. Instead of having to keep track of all the files and import them in the correct order I concatenated them together into a single file.

cat *.sql > backup_db.sql

I now had a problem. How could I transfer this file into my MySQL container?

Volumes

To make my life easier, I decided on creating a new volume for my MySQL container.

I created a local folder called "sql" and added the mapping to a new folder "sql-backup" in my docker-compose.yml file.

  db:
    image: mysql:5.7
    volumes:
      - db_data:/var/lib/mysql
      - ./sql:/sql-backup

I chose "sql-backup" for the folder name as I knew it wouldn't already be in my MySQL container and would not conflict with anything. Docker would create it on the next restart and map anything I had in "sql" to it.

I added the backup_db.sql file to my "sql" folder and restarted my containers.

MySQL

I logged into the MySQL container and confidently entered the following command:

mysql -u root -p wordpress < /sql-backup/backup_db.sql

I entered the root password and then it threw an error!

ERROR 1067 (42000) at line 21

I looked it up and realized quickly that I was dealing with new MySQL defaults. I quickly found a blog with reasonable steps.

I grabbed my sql_mode variables and came to a new problem, most Docker images do not have editors installed.

Containers are suppose to be able to be quickly rebuilt and thus shouldn't need an editor. I didn't want to rebuild mine in order to do the import, so I cheated and installed one.

Apt

Thankfully the MySQL image had apt installed, after a quick update I installed vim and was on my way.

apt update
apt install vim

I edited my config file, restarted the service, and tried again only to get a new error.

ERROR 1215 (HY000): Cannot add foreign key constraint

Foreign keys

The good thing about error 1215 is that it is common and there are many blogs and articles on how to fix it. The bad part is there is around 12 different things that can be wrong. It is a very generic error for MySQL.

I logged back into the MySQL container, looked up once again how to switch databases (it is USE DATABASENAME), and ran SHOW TABLES. I got error 1146 which confirmed that the tables foreign keys were referencing were being created in the wrong order.

I opted to disable foreign key checks instead of hunting through all the tables and creating them in the right order. (See #1 in this list for the commands)

And it worked!

Nevertheless, Jenn Coded

Jenn — Sun, 08 Mar 2020 18:17:38 +0000

In college, I applied a position as a teaching assistant for C/C++. My boyfriend at the time said I would never get it as I wasn't in the computer science program.

Nevertheless, Jenn Coded

An imagining lab need instruction guides and example projects for 3D deconvolution. I am not a microbiologist nor know the supercomputers well.

Nevertheless, Jenn Coded

It was the final day to submit a game to a small contest. It is 80s themed, so I make a quick text based game.

Nevertheless, Jenn Coded

My career as a chemist was stalling, I interview for a sysadmin position on a whim.

Nevertheless, Jenn Coded

I move states suddenly and am looking for work, I apply at midnight for a job as a engineer.

Nevertheless, Jenn Coded

I move back and am looking for work. I apply randomly to a job and the interview is my former team lead.

Nevertheless, Jenn Coded

I go to a wedding and sit at my assigned seat, a month later I'm a developer at the local University.

Nevertheless, Jenn Coded

Life have thrown me a lot of curveballs but...

Nevertheless, I coded.

Shattered or How to bomb interviews as a senior developer

Jenn — Fri, 06 Mar 2020 19:30:10 +0000

Yesterday I interviewed for a senior developer position at a large corporation. I was confident, in charge, and ready to rock.

Thirty minutes into the interview I left shattered and destroyed.

What happened?

I bombed.

How I bombed the interview

Bombing an interview can happen at any stage of your career. It can happen if you prep. It can happen if you don't prep.

Bombing isn't always just on you. It can also be because of the team that you are interviewing for, their lack of preparation, and the interview style.

Prepping

I prepped by reviewing everything on the job listing.

Most of the bullet points were about Agile, testing, and leading a project so I reviewed those skills and made sure they were mentioned on my resume.

There was a single line mentioning the MEAN stack so I brushed up on my MongoDB and Express.js.

I felt confident and prepared.

I washed my power suit, picked out my jewelry, and settled on a simple red lipstick and gold eyeshadow look.

The interview

I had been to the location before, so I knew how to get there and the amount of time needed to walk to the building. I arrived a few minutes before my interview to check in and calm myself.

I met one of the interviewers and went to the conference room. He started to explain the the team and project.

And I realized something wasn't right.

He was going into a deep dive of security terms. I rolled with it. I did security years ago so I was able to keep up and add to the banter.

He then talked about the tool the team was making and their stack, Java 8 and Oracle.

They wanted a senior Java engineer who knew a little about front end development not a senior front end developer.

It had been years since I worked with Java. It wasn't even listed on my resume.

They carried on and quizzed me about JavaScript and everything left my brain.

I couldn't remember the base datatypes, hoisting left my mind, first class functions were not there. I scribbled frantically on my scratch paper trying to pull up anything. These were terms I knew, things I just did without thinking, I could not define them anymore. They were just a part of how I coded.

Then I was asked about databases.

I finally asked them what they were looking for. It was obvious I was not it. They agreed and we ended it early.

Afterwards

I calmly walked out and drove home.

I called my recruiter and they apologized to me. They didn't know the team wanted a Java engineer as that is not what they asked for. The recruiter was also confused to why they even interviewed me as my resume has none of the skills they actually wanted.

I bombed because I should not have been interviewed for that position. It still hurts and is frustrating. Remember not every job is for you.

One year later....

Jenn — Sat, 22 Feb 2020 22:05:00 +0000

One year ago, I bought a domain, jenn.dev. I wanted to have to have a space to show off my skills and an excuse to learn new things.

I've used Gatsby, React, various Firebase SDKs and APIs, to add a blog and portfolio of web apps to the site. It is place for me play and learn.

The only thing I wished I had done more of while making this little teal wonderland, is blogging. Sharing the joys and frustrations of various decisions and whims I made along the way. It is my goal for this year to make more tutorials and blog posts about the random little projects I made.

Not everything needs to be polished right away. The process is just as important as the final product.

Why I prefer rem over em

Jenn — Tue, 11 Feb 2020 22:53:12 +0000

I always prefer to use rem units over em units in CSS.

Definitions

Rem represents the root element's font-size. The root element for a HTML document is almost always the html element.

Em represents the current element's font-size. This value can and does change while CSS rules are being applied.

Why I prefer rem

There are two main reasons why I prefer rems:

Rem's value does not change in a CSS rule

Take for example this rule with inherited font-size 18px:

html {
  font-size: 18px;
}
.page {
  font-size: 2em;
  margin: 1em;
}

How many pixels wide is the font-size?
How many pixels wide is the margin?

They are both 36px wide even those the number of ems are different!

This is because the font-size for the element changed and all properties calculated after that will use this new font-size.

In comparison, the same rule with rems instead:

html {
  font-size: 18px;
}
.page {
  font-size: 2rem;
  margin: 1rem;
}

How many pixels is the font-size?
How many pixels wide is the margin?

The font-size is 36px and margins are width 18px.

Rems scale the whole page

If a user sets their browser or personal css file to use a larger base font, it is inherited everywhere!

The browser/personal css is the last file in the cascade (minus anything styled directly on an element).

Don't fight users. They know what font-size works best for their eyes.

Not breaking the flow

Having a website that can work with this kind of resizing requires using other CSS length units like percentages and viewport height/width (vh & vw). Using these other units will help to keep the rhythm and flow of the page similar at vastly different base font-sizes.

It is important to remember not everyone uses super high resolution monitors or the same resolution as you.

Crafting as Code

Jenn — Tue, 04 Feb 2020 19:27:55 +0000

Knitting and crocheting has had a recent comeback. I can be spotted knitting at conferences and meetups. What makes knitting so similar to programming?

The patterns!

Hello World

Squares and rectangles are the "Hello world" of knitting and crochet. These first projects are easy to travel with, often use a small number of stitches, and can be completed quickly enough to give the crafter an idea if they want to keep making things.

A typical dishcloth pattern

Gramma's Dishcloth

Cast on 3 stitches.
Knit one row.
K1, Kf&b, K1.
*K2, yo, knit to end.* (For no holes,*K1, kf&b, knit to end.*)
Repeat from * to * until there are 45 stitches on your needle.
K2, yo, k2tog, knit to end; repeat this row two more times.
(Knit three rows plain for no holes.)
^K1, ssk, yo, k2tog, knit to end.^ (For no holes,^K2, k2tog, K to end.^)
Repeat from ^ to ^ until 5 stitches remain.
K2, k2tog, K1.
K1, k2tog, K1 (3 stitches remain).
Bind off, cut yarn and weave in ends at corners.

Knitting is manipulating arrays

Each row is an array carried on a needle. Stitches are popped, spliced, and shifted as per the pattern.

Knitting patterns are code!

Stitches are array methods

In knitting there are two main stitches: knit and purl. They are made in opposite directions of each other (knit is in front of the needle, purl is in the back). They can be thought of as 1's and 0's.

All the different knitting patterns and stitches are combinations of these two stitches.

The above pattern has five different stitches:

K: knit
Kf&b: knit the front and the back of the same stitch
K2tog: knit two stitches together
Ssk: Slip, slip, knit (this also knits two stitches together)
yo: Yarn over

Knitting is usually completed from left to right and then the work is flipped over. A pair of rows acts like a stack.

A dishcloth is a multidimensional array

In the pattern above, think of each row as an array. Each row is pushed into our parent array and knit stitches will be represented by a 1. All code is in JavaScript.

Casting on is creating our initial array with three stitches.
var dishcloth = [] dishcloth.push(Array(3))

Knit one row: This is filling the first array.

dishcloth[0].fill(1)

The dishcloth isn't very impressive at this point:
[[1, 1, 1,]]

K1, Kf&b, K1: Knit one stitch, knit and insert a stitch into our array, then knit the last stitch.
dishcloth.push([1,1,1,1])

K2, yo, knit to end: This is our first loop. Knit two stitches, insert a null value, and knit to the end.
dishcloth.push([1, 1, null, 1, 1])

Repeat from * to * until there are 45 stitches on your needle: Repeat the previous row until the array length is 45.

Here is an example of code that could be written for the instructions so far.

You may notice that I reversed the odd numbered rows, I did this to represent how a pair of rows in knitting acts like a stack. This is how the dishcloth gets a decorative edge on both sides.

Other similarities to programming

Patterns can require stitches to be knit out of order (cabling), be multithreaded (knitting with multiple colors at the same time), and use different languages (decorative crochet borders).

There are several kinds of debugging. Two popular ones are:

Frogging: Is removing one or more rows (it is called frogging because you "rip it, rip it", knitters love bad puns).
Tinking: is knitting backwards to the bad stitch.

Patterns can be written out or drawn as charts.

Every knitter has their own opinion on the best tools, e.g. wood, acrylic, or metal needles and yarns. It can be compared to the language and editor wars.

Knitting is code, you just have to learn the language first.

Photo by Rebecca Grant on Unsplash

My code review checklist

Jenn — Wed, 04 Dec 2019 23:39:24 +0000

I like reviewing code. I often learn new tips or tricks whenever I review. It doesn't matter if it expert or novice code, I learn from each.

Keeping track of everything can be difficult when doing reviews, so here is my personal list of things I check for.

My reviewing code checklist

Spelling: Spelling things correctly helps others search the code more effectively when debugging, refactoring, or writing enhancements.
Transposed text: If it is a content update, I double check the source material.
Links: Any relative links? Absolute links? SSL? The correct answer depends on the project but I always click through to check that they work.
Overly complex code: I do not like code golf. Writing overly elegant code just makes it more brittle and harder to support. If I can’t understand it at 3am during an incident, I want it refactored or documented thoroughly.
Does it meet code standards? I like code standards. If everyone knows what the requirements are, the more likely code will be accepted. Also known as, "Argue once over tabs versus spaces, decide as a team, and WRITE IT DOWN."
Comments: Format and length depends on the team (see code standards). Some teams prefer commit messages or READMEs over comments in code. This step could be renamed, "Is it complex or messy but documented?"
Security: Is this resource only for admins or editors? How is that checked? Are inputs sanitized?

Actions

Run the linter: Sometime people forget.
Run the tests: Yes, even if there is automated testing.

Testing

Run the code: Did the change work?
Check existing behavior: Does this change affect something else unrelated? I usually check only two or three key paths.

If everything looks good, I approve the pull request.

Reviewing code shouldn't be a chore. It is a great learning opportunity for everyone.

Demystifying Docker Compose

Jenn — Thu, 21 Nov 2019 23:45:28 +0000

My latest project used Rails and Postgres. After fighting my Mac and the Postgres install for an hour, I gave up and used Docker instead.

Docker has always seemed kinda mystical to me and I really wanted to understand my setup this time instead of just copying a quick start. And since there is no better way to learn something than to teach it, I will share it with you!

There are three key files:

Dockerfile
docker-compose.yml
database.yml

And two main commands:

docker-compose build
docker-compose up

You will need to have Docker installed.

I prefer to use the Docker extension for VSCode to interact with my running containers, so there are more commands you may need if you are not using that extension.

Dockerfile

The Dockerfile is where the dependencies for your container are defined. I'll go through mine line by line.

FROM ruby:2.6.3
This is from what image it is starting from, in this case the Ruby 2.6.3 image.

RUN apt-get update -qq && apt-get install -y nodejs postgresql-client RUN gem install bundler
Docker containers are VERY bare bones and the next lines were installing Nodejs, a Postgres client, and Bundler.

WORKDIR /my-api
This should be the folder where all your application code is in. It is setting the working directory so the next commands are easier.

COPY Gemfile /my-api/Gemfile COPY Gemfile.lock /my-api/Gemfile.lock
Copying the Gemfiles to the working space in the container.

RUN bundle install
This will run bundler in the container.

COPY . /my-api
Copy all the other files into the working space.

COPY Docker/database.yml config/database.yml
This line can be skipped. The project was shared with others who had local installations of Postgres and I didn't want to override the config/database.yml in the repository. I created a Docker folder for my specific database.yml file. This command copies my database.yml file to the standard location for the container.

EXPOSE 3000
Only open the ports you need to have open.

# Start the main process. CMD ["rails", "server", "-b", "0.0.0.0"]
Start rails.

So for this Rails and Postgres project, my full Dockerfile looked like this:

FROM ruby:2.6.3
RUN apt-get update -qq && apt-get install -y nodejs postgresql-client 
RUN gem install bundler
WORKDIR /my-api
COPY Gemfile /my-api/Gemfile
COPY Gemfile.lock /my-api/Gemfile.lock
RUN bundle install
COPY . /my-api
COPY Docker/database.yml config/database.yml 
EXPOSE 3000

# Start the main process.
CMD ["rails", "server", "-b", "0.0.0.0"]

docker-compose.yml

This file connects our newly defined Rails container to a Postgres database.

version: '3'
All docker-compose files start with this. Don't change the version, just leave it.

services:
This defines the services (aka containers) to be built and how they are connect.

db: image: postgres
The first service is a Postgres database. This is pulling the base Postgres image. I do not need any add-ons or customizations for my database so a Dockerfile for this service isn't necessary.
If you want data to persist, you will have to add a volume.
volumes: - ./tmp/db:/var/lib/postgresql/data
I didn't need persistent storage for this project so I didn't use a volume.

web:
The next service is the Rails web server.

`build: .`

Build the image from our newly defined Dockerfile.

command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
Run this bash command in our new service container. This command is removing the pid file and telling rails what ports to use.

`ports:
  - "3000:3000"`

Mapping the Rails service port 3000 to my local machine's 3000.

`depends_on:
  - db`

This service will not start until the database service has started.

Once again, a volumes line could be here but I didn't need it.

All together my docker-compose.yml looks like this.

version: '3'
services:
  db:
    image: postgres
  web:
    build: .
    command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
    ports:
      - "3000:3000"
    depends_on:
      - db

database.yml

This is the database connection file.

default: &default
  adapter: postgresql
  encoding: unicode
  host: db
  username: postgres
  password:
  pool: 5

development:
  <<: *default
  database: my-api_development


test:
  <<: *default
  database: my-api_test

docker-compose build

This command builds the containers. With my set up, anytime I update the Gemfile I have to rebuild.

Building can take some time. Each line in the docker-compose.yml is a step in the building process that can be cached so each build afterwards is faster.

Example output from a quick rebuild.

$ docker-compose build
db uses an image, skipping
Building web
Step 1/11 : FROM ruby:2.6.3
 ---> d529acb9f124
Step 2/11 : RUN apt-get update -qq && apt-get install -y nodejs postgresql-client
 ---> Using cache
 ---> 32e807d05964
Step 3/11 : RUN gem install bundler
 ---> Using cache
 ---> 0c1c757c6905
Step 4/11 : WORKDIR /my-api
 ---> Using cache
 ---> b305131f9c6b
Step 5/11 : COPY Gemfile /my-api/Gemfile
 ---> Using cache
 ---> 657fc337d0c4
Step 6/11 : COPY Gemfile.lock /my-api/Gemfile.lock
 ---> Using cache
 ---> 09154df316d8
Step 7/11 : RUN bundle install
 ---> Using cache
 ---> 7915826ca085
Step 8/11 : COPY . /my-api
 ---> 4ff482944c8e
Step 9/11 : COPY Docker/database.yml config/database.yml
 ---> 101f814ba0bc
Step 10/11 : EXPOSE 3000
 ---> Running in 45f91676a8f3
Removing intermediate container 45f91676a8f3
 ---> 9426b9097b78
Step 11/11 : CMD ["rails", "server", "-b", "0.0.0.0"]
 ---> Running in 3f262f9679a2
Removing intermediate container 3f262f9679a2
 ---> a056b93d4bc8
Successfully built a056b93d4bc8

docker-compose up

This command starts the containers.

$ docker-compose up
Starting my-api_db_1 ... done
Recreating my-api_web_1 ... done
Attaching to my-api_db_1, my-api_web_1

Then visit localhost:3000 in a browser to confirm it is up and running.

For more information, check out the official quickstart guide for Rails.