Forem: Luke Babich

Create and Seed Your Postgres Database in Minutes with Docker

Luke Babich — Wed, 14 Dec 2022 09:38:05 +0000

adventure time Golang Gopher character growing out of ground, chaotic composition, photorealistic - Created with Midjourney

What is Database Seeding and Why?
Building Your Postgres Docker Container
- Prerequisites
- Creating the Docker Container
- Connecting to our DB
Structuring Your DB with Go Migrate
- Generating Our Migration Scripts
- Running our Scripts
Populating Your DB with Go Faker
- Generating Fake Data
- Inserting Fake Data into the Database
Summary

This article help us understand the importance of creating silo'ed development environments that allow us to quickly iterate, create scenarios and test. In this article we will focus on using Go as our language to programatically seed our data. However there are tools in other languages to achieve the same results. Whatever your preference, this article just hopes to achieve bringing light to how you can go about it.

What is Database Seeding and why

Database seeding is the process of adding initial data to a database. This is often done for testing or development purposes, as it allows developers to work with a sample dataset without having to worry about corrupting real data. Database seeding can be useful for testing the functionality of a database, as well as for experimenting with different queries and data structures. In general, seeding a database with fake data can help ensure that the database is working properly and can be useful for a variety of purposes.

One important aspect we want to ensure is that it is quick and easy to reboot and create these scenarios or get us back to a stable point.

1) Building Your Postgres Docker Container

Before we can start seeding our database we first have to get our container up and running.

Prerequisites

Before we begin with creating a container we need to ensure that we have Docker installed and running.

Now you will not need to worry about getting too advanced with Docker features. So if you are not very well versed in Docker do not worry.

In short we are going to use a predefined Docker image for Postgres and then modify and put some data into it.

Creating the Docker Container

So to start we want to create our bare bone Postgres container:



docker run --name my-db -e POSTGRES_PASSWORD=pwd -e POSTGRES_DB=database -d -p 5432:5432 postgres

This will create a new Docker container named "my-db" using the latest version of the Postgres image from Docker Hub.

The -e flag is used to set the password for the default user (default is postgres) to mysecretpassword as well as creating us a database inside our container called 'database' (default is postgres).

The -d flag specifies that the container should run in detached mode, running in the background.

Then the -p flag is used to expose the port of our container and map it to our db, in this case it is a mapping of 5432 -> 5432

To confirm it is running we can then run:



docker ps

Connecting to our DB

So now that our container is up and running you may be asking yourself how do I connect to it or how do I know it is working?

We will now connect to our new Postgres container to then confirm that it is as we expect:



docker container exec -it my-db bash

Once connected to the container we can confirm all is running fine with:



psql -u {user} {db}

NOTE: to exit this psql \q and hit enter

So if you have followed the above you can swap out user for postgres and db for database or whatever you have named your instance.

Now you are able to execute and insert data into your new database.

If you prefer to just connect to the container and run queries direct you can modify the above:



psql -U {user} {db} -c "{SQL_QUERY}"

2) Structuring Your DB with Go Migrate

Now that we have got our database running we want to now populate it with some tables.

This involves specifying the schema, which defines the structure and organization of the data, as well as any constraints or rules for the data. To do this we are going to use Go Migrate.

Tools like Go migrate are typically used in database development to manage changes to the database schema over time. In our case we can make use of this tool to define our initial DB schema which will we then populate with the data we want.

Generating Our Migration Scripts

Using the Go Migrate CLI we will first create our migration files in a folder of choice in our project.



migrate create -ext sql -dir db/schema -seq schema_init

This will create an up and down file. I recommend reading into how migration tools work. In short however. Up has to do with doing some form of change to your DB. Whereas Down has to do with reverting the changes.

When using the CLI this will add some numbers to the front. This is used for internal tracking of what SQL scripts have been executed and what has not. So make sure to keep these.

Once you have generated your migration files we can then add our SQL scripts to structure our DB:



-- schema_init.up.sql

BEGIN;

-- Create the users table
CREATE TABLE users
(
    id       SERIAL PRIMARY KEY,
    username VARCHAR(255) NOT NULL,
    handle   VARCHAR(255) NOT NULL
);

COMMIT;

This will create 2 tables called rooms and users. You can create your structure however you want but this is just to show as an example.

Running our Scripts

You can use the CLI to run a migration script while connecting to the DB directly. However in this example I am going to show you the code version.

TLDR; this is the end code:



package main  

import (  
   "database/sql"  
   "fmt"   "github.com/golang-migrate/migrate/v4"   "github.com/golang-migrate/migrate/v4/database/postgres"   _ "github.com/golang-migrate/migrate/v4/source/file"  
   "github.com/joho/godotenv"   _ "github.com/lib/pq"  
   "log"   "os")  

func main() {  
   db, err := sql.Open("postgres", "postgres://postgres:mysecretpassword@localhost:5432/database?sslmode=disable")  

   if err != nil {  
      panic(err)  
   }  

   driver, err := postgres.WithInstance(db, &postgres.Config{})  

   if err != nil {  
      panic(err)  
   }  

   // NOTE: relative path to folder  
   m, err := migrate.NewWithDatabaseInstance(  
      "file://./db/dev",  
      "postgres", driver)  

   if err != nil {  
      panic(err)  
   }  

   err = m.Up()  

   if err != nil {  
      fmt.Println(err)  
   }  
}

Over here I have my main package that I can run. Using the database/sql and golang-migrate packages.

First we open a connection to our container DB, which includes the username, password, host, and port of the database, as well as the database name and SSL mode. We then create a new instance of the Postgres driver and uses it to connect to the database instance.



   db, err := sql.Open("postgres", "postgres://postgres:mysecretpassword@localhost:5432/database?sslmode=disable") 

   driver, err := postgres.WithInstance(db, &postgres.Config{})

Then we create a new instance of the migrate package, using the path to the migration files and the Postgres driver instance (In my case this is relative path to where I am executing this script). This allows the migrate package to access and manage the database schema using the provided migration files.

Finally, the code calls the Up() method on the migrate instance, which will apply any pending migration scripts to the database, updating the schema as necessary.



   // NOTE: relative path to folder  
   m, err := migrate.NewWithDatabaseInstance(  
      "file://./db/dev",  
      "postgres", driver)  

   if err != nil {  
      panic(err)  
   }  

   err = m.Up()

After this your running Postgres Container will now have its relevant schema set up. If you are wanting to change this at all. Create a Down script that removes these tables or simply delete you docker container and recreate it.



docker rm -f {container}

Populating Your DB with Go Faker

Now that we have our DB and our Schema for our DB set and in place we now want to populate it with some fake data using Go Faker.

Generating Fake Data

Go Faker is a handy little library that allows us to define a Struct, add some tags and it will then randomly generate data based on the tags we define.

In the example above we defined a users table. Now for simplicity we are going to generate ourselves a user and then insert that user into our DB.

First let's go ahead and define a user Struct:



type User struct {
    Username string
    Handle   string
}

As you can see here there is nothing crazy going on. Just a simple struct with 2 properties. We can then dive into how faker can work in 2 different ways. Depending on what your needs are you may want to generate the data yourself or use aspects of faker to do it for us.

Let's modify our User model slightly by adding a faker tag to generate a username:



type User struct {
    Username string `faker:"username"`
    Handle   string
}

The above has now defined that when we use Faker to generate our data that we want to use it's inbuilt generation to create a random username for us. There are a few other ways that you can ensure that it is unique. But there are many tags you can use.

To then generate using Faker we then can add the following piece of code:



func createUser() User {
    user := User{}

    err := faker.FakeData(&user)

    if err != nil {
        fmt.Println(err)
    }

    return user
}

This method over here will now generate our user with a random username and a random string for their handle (if you do not define a tag it will generate randomly based on the type).

The alternative to the above is:



func createUser() User {  
   user := internal.User{}  
   username := faker.Username()
   return user  
}

With the above approach there is no need for the tag.

Inserting Fake Data into the Database

We now have a way of generating a user and inserting them into our DB. However how do we tie all this in together?

TLDR;



package main  

import (  
   "database/sql"  
   "fmt"   "github.com/golang-migrate/migrate/v4"   "github.com/golang-migrate/migrate/v4/database/postgres"   _ "github.com/golang-migrate/migrate/v4/source/file"  
   "github.com/joho/godotenv"   _ "github.com/lib/pq"  
   "log"   "os")  

func main() {  
   db, err := sql.Open("postgres", "postgres://postgres:mysecretpassword@localhost:5432/database?sslmode=disable")  

   if err != nil {  
      panic(err)  
   }  

   driver, err := postgres.WithInstance(db, &postgres.Config{})  

   if err != nil {  
      panic(err)  
   }  

   // NOTE: relative path to folder  
   m, err := migrate.NewWithDatabaseInstance(  
      "file://./db/dev",  
      "postgres", driver)  

   if err != nil {  
      panic(err)  
   }  

   err = m.Up()  

   if err != nil {  
      fmt.Println(err)  
   }  

    user := createUser()
    insertUser(db, user)
}

func createUser() User {
    user := User{}

    err := faker.FakeData(&user)

    if err != nil {
        fmt.Println(err)
    }

    return user
}

func insertUser(db *sql.DB, user internal.User) int {  
   sqlStatement := "INSERT INTO users (username, handle) VALUES ($1, $2) RETURNING id;"  

   lastInsertId := 0  
   err := db.QueryRow(sqlStatement, user.Username, user.Handle).Scan(&lastInsertId)  

   if err != nil {  
      panic(err)  
   }  

   return lastInsertId  
}

Lets break down the new sections we just added to the above code:



user := createUser()
insertUser(db, user)

In the main we then called our createUser method and then handed this over to another method who will then insert that user into our DB.



func insertUser(db *sql.DB, user internal.User) int {  
   sqlStatement := "INSERT INTO users (username, handle) VALUES ($1, $2) RETURNING id;"  

   lastInsertId := 0  
   err := db.QueryRow(sqlStatement, user.Username, user.Handle).Scan(&lastInsertId)  

   if err != nil {  
      panic(err)  
   }  

   return lastInsertId  
}

The above is just taking our original connection when we did our schema setup and then executing a SQL query to insert into our users table.

This proceeds to then return the index that was last inserted in case we wanted to perform another action after.

Summary

Hopefully all the above makes sense. To summarize everything:

We created a bare bones Postgres Docker container
Created scripts to structure our DB for us (Users table) using Go Migrate
We then generated a random user with fake data using Go Faker
Inserted this new user and returned the insertion ID

The example is not overly complicated however you can expand and add to it as you wish. Now as I said this is not something just for Go. Other languages have their own versions of these tools. The main takeaway is understanding their use cases and how they can be used together to create a development environment that is easy to modify and simple to reset should you need to! As always these things can be as complicated as you require.

If you liked this article be sure to follow or check out some of my socials. I also stream on Twitch every Tuesday + Thursday at GMT+1.

Dependency Injection in under 2minutes

Luke Babich — Thu, 19 May 2022 08:53:34 +0000

Hello Devs! Lets talk DI or dependency injection.

The easy to digest analogy:

You work in a car factory. Each part of the car making process requires either multiple pieces of specialized machinery or a person to put the car together.

Stage 1 of production requires a machine that cuts the frame, and another to lift the frame to stage 2.

In stage 2 it then puts assembles the frame the car puts it together. It then uses the same machine to lift and take it to stage 3.

In stage 3 we may have a person who comes and inspects the final product and then sends it on its way.

The summary:

The above analogy can be thought of as you have a code base or group of functionality (the car factory) that is made up of different services or functionality (machines or people).

These pieces of functionality could be used in one or multiple places in your code base (the stages of production).

Dependency injection is about providing your functionality to your different pieces of code. Without that code directly importing or using that piece of code (this is the responsibility of something called an injector to provide it).

This makes it easy to swap it out and only have to make sure that whatever you are swapping it with conforms to the interface that piece of code requires (it is about making sure the contract of functionality is met).

The benefit of dependency injection is:

Decreased coupling between pieces of code
Increased flexibility in swapping functionality (what happens if you need a new machine to lift or move assemble the car?)
Ease of testability of your code
Less boilerplate
It allows concurrent development. Multiple developers can work on different classes that use each other, while only needing to know the interface the classes will communicate through.

I hope this has helped!

How to identify and avoid burnout as a software developer

Luke Babich — Mon, 06 Sep 2021 13:23:54 +0000

Recently me and some friends have been working on a side project. Since we had been hanging out together a lot more you start to get some insight into how they manage their time, as well as see how they burnout.

This started to get me thinking that in the current tech space there is this constant need for validation and push towards always learning. However it is not talked about how on top of your regular job these extra learnings, side projects and general life start to add up and contribute to burnout.

I recently made a video that outlines the issues and how to deal with burnout.

However I will also discuss it here.

The Problem

I will focus on one main point within this article and that has to do with how developers (the ones in the frontend space in particular) are swamped with a so much information, tools, frameworks etc.

This is not a bad thing. However I am sure there are people that have reached a point where they have been reading articles or been in software long enough to see the infamous

x framework vs y framework,
Top 10 tools this week,
What you need to know to be a good developer.

Once again these are not necessarily bad. I am more focusing on the mindset. The feeling newer developers start to experience when either coming into the industry or just starting. I know I felt it when I first started and that was when half the current tools didn't exist yet. This feeling of a massive amount of learning.

If I had to list every single tool or framework I have worked with or learnt, either in my free time or work the list would be quite a nice chunky read. The frontend ecosystem is always growing and evolving. However there is always pressure on people to try and learn it all as quick as they can. There is a reason there is a meme behind "Have 3 years of experience in X framework" when it has only been around for a year. This pressure to learn the latest and newest things as fast as possible, causes a ripple effect across the industry.

What does this result in? New and existing developers feeling pressure to dedicate what little time they have and this can lead to burnout and lifestyle imbalances.

The Controversy

This take might seem somewhat double edged. On one hand its important to grow your skills. However on the other it is also important to not just wither away learning something that may not even be relevant in a few years time. Or rather, just to live a little.

Some might say that learning is what they enjoy. That is fine and maybe then this article is not something that would really appeal to you. The idea behind this is to identify unhealthy habits that some people have when it comes to learning in their free time, on top of a job, side project and just a general busy schedule.

So what's the solution?

The solution is not to stop learning. The solution is to schedule your time accordingly.

As an example I dedicate 30min in the morning and evening (this is more a time to wind down and relax) to reading. During my day job, at lunch I use the gym to stay in shape and ensure I stay healthy.

After work it becomes a choice, dedicate an hour or two to studies/articles/videos or on my side project. The rest of my time is dedicated to what I enjoy, playing games with my friends or watching some TV.

By scheduling this time it ensures that I am getting the right amount of sleep and balance. This balance allows not only for me to grow as a developer but also to grow as a person and enjoy the things I want.

I just want to re-iterate. If what you find as fun is learning and doing little code projects this is 100% fine. The point behind the solution is not to stop coding and doing those things. It is about managing your time in order to ensure that you get sleep/exercise/eat well and stay hydrated. What you do for fun is up to you. However that fun should not make you feel pressured or induce anxiety and stress to do it. That fun time should be the stuff that allows you to unwind and reset mentally in order to avoid a prolonged or new burnout.

I hope this was helpful to some as I have found this is what works best for me. Feel free to leave any questions and if you did check out that video that is much appreciated as well!

JavaScript: File Naming Conventions

Luke Babich — Mon, 10 Feb 2020 10:09:14 +0000

So first a little disclaimer, I don't feel how I write files to be better than the next person. This is more of an open forum to discuss how people go about naming theirs and how the manage files as a system grows.

So what exactly do I mean?

So you are working in your company, personal project or whatever it may be and it starts to scale. You get lots of files, loads of components etc.

So in general how do you make sure that you understand what something is? Coming from starting in Angular and reading Clean Code by Robert Martin I am very used to the approach of naming my files according to the thing that it is:

Component
Service
Router
Util

(communication.service.ts or button.component as an example)

The list goes on. Each has a reason for the name, each has a difference in its responsibility. Just like a function would have its single responsibility.

Now coming into React/Vue most components stick to UpperPascal for their naming and a lot of people just name their files without saying what it is but sometimes what it does. Its just a different approach to file splitting.

So the question comes in how do you go about naming and scaling your code base? What approaches did you take to ensure you don't get overwhelmed by files with hidden meaning behind their names?