Forem: MBV

Building a Saas in five days

MBV — Thu, 12 Sep 2024 12:35:15 +0000

I love myself a good challenge once in a while. Not the cringy kind you find on MrBeast's channel where they exploit the ever-growing economic inequality. But a simple "Build X thing in Y amount of time" kind of challenge.

I've suffered from wantreprenurship for a long time. I've written about it and attempted a similar challenge before to cure it. So far, I have not had much success to report on.

So, doubling down on another challenge, I wanted to take my saas starter template for a spin and see how well it worked out in real life. I also wanted to get better at creating videos on YouTube, so to add even more work to what is already a tight timeline, I decided to create a video about each day of the challenge.

I decided to try and create a micro-saas in 5 days.

What can you build in five days?

Given my aforementioned affliction, I've long subscribed to all the newsletters that focused on "business ideas" that I could get my grubby little hands on.

I have previously also tried to get my creative juices flowing by writing down 10 ideas every day that lasted for a long enough time, to have a decent list of not-so-decent ideas. The whole idea that ideas are important was busted a long time ago. What matters, like really matters, is the execution and actually doing something. Anything.

I've tried to take this to heart. Prioritize getting things out into the wild and see what happens. I'm not going to try and argue that doing some market research beforehand and choosing the most promising one is going to be a waste of time. But when just starting out, you probably shouldn't be spending much time here as you still (like me) lack the gut feel for where to focus your attention. Look around, and see what other people are doing and making money on. Pick something that matches your interest and skillset, try to create a competitor and take it from there.

Anyway, back to my long list of genius ideas. I did spend some time filtering the ideas based on how much they spoke to me and whether I thought they were feasible to do in 5 working days. The time constraint did help me cut a lot of ideas, many of which would involve acquiring lots of data for the AI overloads to gobble up. So what could I, realistically, build and ship a scrappy version of in five days?

A no-code blog.

Game Plan

I've written and rewritten my own blog many times now. Started out in next.js back when I was huge into JavaScript/TypeScript and had my "portfolio" website with some light blogging content, to learning and rewriting it in Rust to finally just settling on Go. The language I've been working in for the last many years. I think it's one of the best ways for developers to get a sense of how to build and ship software, especially early on, as it takes you through all of the steps that happen in building web apps.

Anyway, I have a fair bit of experience writing blogging software.

I had recently begun to use Traefik in favour of Caddy for reverse proxying my applications. Combined with a docker plugin called rollout you get zero-downtime deployments basically for free. This sparked an idea for how I easily could spin up blogs for customers, keeping them updated and restarted when needed.

I would simply have a template file that would parse some variables, create the file and store it in a specific place and call docker-compose pointing to this file. Easy. This ended up looking something like this:

services:
  blog--id-{{.BlogID}}:
    image: mbvofdocker/the-bloggening:{{.Version}}
    environment:
      - ENV_VAR_1={{.EnvVarOne}}
      - ENV_VAR_2={{.EnvVarTwo}}
    labels:
      - "traefik.enable=true"
      - "traefik.http.services.blog-{{.BlogID}}.loadbalancer.server.port=3333"

I've left labels and fields out for brevity but this should get the idea across.

Traefik would take care of discovery and routing traffic to the individual blogs, generating SSL certs and taking the health status of each container.

Next, there needs to be some way to spin up blogs quickly with various designs, color themes, layouts etc. You might have guessed that this is what happens in this part of the above docker-compose file: mbvofdocker/the-bloggeing:{{.Version}} and you're right.

I went with only having one design and two colour themes, dark and light, to begin with. Basically, I'm just re-creating this blog you're reading this article on, but on an automatic basis.

Then, I needed a way to take in some information about the blog like a name, title, description etc. Maybe even some socials and have it be dynamically updated whenever the user chooses to do so.

If you've spent any time in the tech space lately you probably heard about the hype of SQLite and Turso. Turso has open-sourced an upgraded version of SQLite called libsql that allows you to talk to your database using HTTP requests and have embedded read replicas. The combination of these two was exactly what I was looking for.

So the components and plan were in place. I would use my starter template, create a docker-compose file for each blog a customer created, route traffic to it using traefik and sync/provide data through Turso. But, as a wise man once said:

Everybody has a plan until they get punched in the mouth

Where are my databases?

I was putting the above together fast, like fast fast. I could definitely make this in those 5 days, no problem. I had a working demo and wanted to have my wife try and create her own blog, so I enthusiastically forced her to drop whatever she was doing to sit down and be an equally enthusiastic test user.

She went through the signup flow, with no issues, same for the onboarding. I had taken a very liberal approach to error handling at this point, which basically meant I only logged when stuff errored. But, nothing happened. It was reporting overall success and I could see the container image was running and healthy. So we went onto her blog that was being served locally on mortens-wifes-blog.localhost, aaaaand nothing.

Not wanting her to think that the financial success I had practically guaranteed from this project was a pipe dream, I frantically looked for why. Turns on, that since I was on the free version of Turso they would sometimes put my "cluster" to sleep. However, their API didn't return any errors or notice that the database hadn't been created. A bit of a bummer, but again, free software so I can't really complain much. Also, I realized that as long as I used my own database to hold the information I could just as easily sync the data with each blog through an endpoint staying with regular sqlite.

I made the changes, had her sign up again and everything worked. Great. Then, my next (and much bigger headache): exit code 18.

Calling docker from within docker

I love Docker. Having your application containerised makes a lot of sense. In this case, it ended up causing a lot of frustration.

Everything was working locally, but I develop outside of docker on my machine and packaged it up and ship when I push to production. So calling docker from my application was not an issue until I had to do it from with the docker image that doesn't automatically have access to resources on the host system. It was a fairly easy fix, just add a volume that contained the docker socket and you're good to go.

volume:
  - /var/run/docker.sock:/var/run/docker.sock

However, I kept getting an exit error from the system that I couldn't for the life of me figure out what meant. Nothing came from googling and asking chatgippity. I even tried Stackoverflow to no avail. I then replicated the entire setup on my local machine but with everything running in docker, where it worked, which left me with more questions than answers.

Turned out, that I hadn't pulled the correct version of the docker image on the server so it couldn't start. Normally, when an image is not available, docker will try and pull it from some hub. I, rather naively, assumed the same would happen here which it of course didn't since permissions are different.

This would cause so much delay that I was super tight on time to create the onboarding experience for which I would spend all of day 5.

In the end, I had a version live that allowed users to sign up, create a blog and see something live, e.g. mortens-blog.mbvlabs.com. But, since I was so strapped for time on day 5 I didn't get to implement the functionality that would allow users to upload or write blog posts. Saying I finished without that feature is probably pushing it but I got super close and have since added it to the setup. The product still needs work but pushing through those 5 days gave it a quick foundation on which to stand. Retrospectively, I probably wouldn't do this and have to create a video about it for each day. Just mentally having to build a thing and also make it work in a story that shows progression is quite tough. You can't just speed through the building process. You need to get relevant clips and snippets of videos throughout the entire process. But it was super fun.

But, what's the takeaway?

Urgency is good

Having publically, even to a rather small audience, announced an end date was incredibly beneficial in cutting into the core of what the product is and what was needed. I realized multiple times during that week that I was spending time on things or adding stuff that wasn't really necessary at the core of what I wanted to do: let users quickly spin up a good-looking blog and add content.

I think that, in general, urgency is a good thing in software. It's super easy to get into a long drawn-out analysis of what is needed right now vs what is needed in the future. Spend time only focusing on the now and you might have a hard time evolving the thing in the future. Spend time preparing for the future and it will be easier to deal with future requirements. If you're right about those future requirements. And that's the core of it, you need to be correct in your assumptions of the future which most people rarely are. I think optimizing for the now will let you get something out in the real world that users can test and give feedback on. As long as you follow good practices such as the single responsibility principle, separation of concern and YAGNI, dealing with future requirements most likely wouldn't be a huge problem. And, should it turn out to be so and you need to rewrite a lot of code, isn't that a good thing? This means you learned something about the domain in which you are working and that is ultimately what should drive the software.

Stop overthinking, ship some things and see what happens.

How to create a blog using Golang

MBV — Wed, 03 Apr 2024 08:21:03 +0000

After getting fed up with React, SPAs, and Javascript around 2021 I decided to re-write my personal webpage in Rust and wrote an article on how you could build a simple blog, purely using Rust. It ended up becoming one of my most popular articles and for good reason; Rust is exciting, fun to write, and blazingly fast. After a while though, I started to feel frustrated with the development process for adding new features to my site: the feedback loop was simply too long.

I've always been interested in solo entrepreneurship and technology. But, as I'm getting older, I realize that I might have been more interested in trying out new technologies. Great for learning and growing as an engineer, bad for shipping projects, and starting to see that sweet MRR grow. I decided to re-re-write my site once again, this time in Go for multiple reasons:

I write Go for a living
Simple language, with a decent type system and fast performance
Blazingly fast compile times

In this post, I will show how you can create your own personal blog using Go. I'll assume you're familiar with Go, and know how to configure a router/database/server, and so on. Should you not, feel free to grab a clone of my Go starter template Grafto that has lots of things configured for you out of the box.

Available on Video

I made this article into a video series that you can follow here:

How To Create A Blog Using Golang: Part 1 - YouTube

This is the first episode in a mini-series showing how to create a personal blog using only Go. The tech stack we'll end up using consists of: Go, sqlite, go...

youtube.com

Foundations

I write all my stuff in markdown; if you're a developer who also wants to start blogging, chances are you also are quite familiar with it. After Go 1.16 where we got the embed package included in the standard library most of our work is already done. We basically only need to have a way of storing some filenames, associating them with an ID or a slug, grabbing the file, and serving it to the user. Pretty simple.

Whether you've created your own setup or grabbed a copy of Grafto, create a new directory in the root of your project called posts and in there, create a file called posts.go. Open it and add the following:

package posts

// imports omitted

//go:embed *.md
var Assets embed.FS

Now, any file with the .md extension will get included in the binary that ultimately gets built once we run go run build. We can simply grab the files from our global Assets variable using Assets.ReadFile(name-of-file) to handle any error that might occur or return the file as a string, e.g:

file, err := Assets.ReadFile("my-post.md")
if err != nil {
    return err
}

return string(file)

It won't be pretty (we'll fix that later), but it gets the basic idea out, which we can build on.

We have a way to include our blog posts in the binary; let's add a way to associate them with a slug. You could use an ID here, but it just looks better to have URLs like: "https://acme.com/my-first-blog-post" compared to "http://acme.com/44a2530b-567d-472f-9495-e2ee64e7ae6d". So, assuming you have a database up and running, add this table:

create table posts (
    id uuid primary key,
    created_at timestamp not null,
    updated_at timestamp not null,
    title varchar(255) not null,
    filename varchar(255) not null,
    slug varchar(255) not null
);

Not much exciting going on here. Basically, for the unaware readers, the slug column above will be the title but URL-friendly. So if you have a post with the title "My First Blog Post" the slug equivalent would be "my-first-blog-post" easy.

Lastly, we need to be able to serve this to readers. That flow would typically involve them hitting a landing page showing a list of articles they can choose from, which links to the article.

The implementation of this will depend a bit on your setup, but let us implement the handler to deal with grabbing a specific article using Echo as our router. Assuming you have a route like /posts/:slug, create the following:

type ArticleStorage interface {
    GetPostBySlug(slug string) (Post, error)
}

func ArticleHandler(ctx echo.Context, storage ArticleStorage) error {
    postSlug := ctx.Param("postSlug")

    postModel, err := storage.GetPostBySlug(postSlug)
    if err != nil {
        return err
    }

    postContent, err := posts.Assets.ReadFile(postModel.Filename)
    if err != nil {
        return err
    }

    return ctx.String(http.StatusOK, string(postContent))
}

I'm putting some decisions on your here in terms of implementing the ArticleStorage. We just need something that grabs the data on the post from the DB, based on the slug.

This is the foundation of what we need...but it's not pretty let's fix that by letting the server do what it was always supposed to do: return HTML.

Enter templ

If you've spent time in the Go ecosystem, chances are you've probably heard about templ. It lets you write HTML templates as Go packages and it's just such a pleasant way of building out a UI. Add some HTMX and alpine.js and you've at least 95% of what you get with SPAs, with the added complexity.

It's good practice to have a base template that wraps around your other templates, so we have a single point for adding things like stylesheets, javascript, metadata, etc. Create a directory in root called views and add the following to a file called base.templ.

package views

templ base() {
    <!DOCTYPE html>
    <html lang="en">
        <nav>
            <a href="/">MBV Labs</a>
        </nav>
        <body>
            { children... }
            <footer>
                <aside>
                    <p>Copyright ©2024 </p>
                    <p>All right reserved by MBV Labs</p>
                </aside>
            </footer>
        </body>
    </html>
}

For this to work, you'll need to install templ and run templ generate which will produce a file called base_templ.go that we can then import into other templates to wrap around them. For the sake of brevity, we'll only create the template to show the actual article. Create a file called article.templ, and add the following:

type ArticlePageData struct {
    Title             string
    Content           string
}

templ ArticlePage(data ArticlePageData) {
    @layouts.Base() {
        <div>
            <div>
                <h1>{ data.Title }</h1>
            </div>
            <article>
                @unsafe(data.Content)
            </article>
        </div>
    }
}

and run templ generate once again.

We can now go back and update our ArticleHandler handler:

func ArticleHandler(ctx echo.Context, storage ArticleStorage) error {
    postSlug := ctx.Param("postSlug")

    postModel, err := storage.GetPostBySlug(postSlug)
    if err != nil {
        return err
    }

    postContent, err := posts.Assets.ReadFile(postModel.Filename)
    if err != nil {
        return err
    }

    return views.ArticlePage(
        views.ArticlePagedata{
            Title: postModel.Title,
            Content: postContent,
        },
    ).Render(
        ctx.Request().Context(), 
        ctx.Response().Writer,
    )
}

If you run your application now and visit a valid URL, you should see a (rather ugly) page showing the markdown of your article but this time, with some sweet hypertext markup.

Making things (slightly) less ugly

In terms of styling things, throwing some tailwind or vanilla CSS at what we have now will get you a long way. But, we still show raw markdown to the user when they visit our articles. Additionally, we might want to show some nicely formatted code snippets in our articles. Let's fix this now.

For this, we need something that can transform the markdown into HTML components e.g

## Some sub header

into

<h2>Some sub header</h2>

Luckily, there already is a create library for this: Goldmark. So let's refactor the posts/posts.go file to parse the content we store using embed.

//go:embed *.md
var assets embed.FS // unexport assets

type Manager struct {
    posts embed.FS
    markdownParser goldmark.Markdown
}

func NewManager() Manager {
    md := goldmark.New(
        goldmark.WithParserOptions(
            parser.WithAutoHeadingID(),
            parser.WithAttribute(),
        ),
        goldmark.WithRendererOptions(
            html.WithHardWraps(),
            html.WithXHTML(),
            html.WithUnsafe(),
        ),
    )

    return Manager{
        posts:           assets,
        markdownHandler: md,
    }
}

func (m *Manager) Parse(name string) (string, error) {
    source, err := m.posts.ReadFile(name)
    if err != nil {
        return "", err
    }

    // Parse Markdown content
    var htmlOutput bytes.Buffer
    if err := m.markdownHandler.Convert(source, &htmlOutput); err != nil {
        return "", err
    }

    return htmlOutput.String(), nil
}

Lastly, update the ArticleHandler to use the Manager:

func ArticleHandler(
    ctx echo.Context, 
    storage ArticleStorage,
    postManager posts.Manager
) error {
    postSlug := ctx.Param("postSlug")

    postModel, err := storage.GetPostBySlug(postSlug)
    if err != nil {
        return err
    }

    postContent, err := postManager.Parse(postModel.Filename)
    if err != nil {
        return err
    }


    return views.ArticlePage(
        views.ArticlePagedata{
            Title: postModel.Title,
            Content: postContent,
        },
    ).Render(
        ctx.Request().Context(), 
        ctx.Response().Writer,
    )
}

Try and edit your article by adding some code blocks and they will now get nicely formatted. You can add custom themes to the parser so your code snippets will be shown with your favorite theme.

Wantrepreneurship & getting out of tutorial hell

MBV — Sun, 31 Mar 2024 15:57:02 +0000

New year, new me bullshit is on everybody's mind right now as we're kicking off the first week of the new year. This is
not going to be one of those "I'm going to start exercising 5 times per week" posts, but thoughts about reaching a
goal of mine, which I've had for multiple years. The idea is to provide my thoughts on the subject of entrepreneurship
and then review it after the month of January 2023, is over. Alright, here we go!

Create something, put it on the internet and have people pay me money for it

For the last few years, my main goal has been to create something using software and have people on the internet pay
me for it. I've yet to accomplish that goal, spending the majority of my time reading and studying entrepreneurship.
To be honest, I've spent probably the last 5+ years preparing for my entrepreneurial journey; reading books, taking
courses, watching videos, participating in forums about indiehacking and bootstrap, which is absolutely crazy way to
approach this. Some of those years were spent getting into software engineering, so not a total loss, but still way too
long to put off something that has excited me ever since I was a kid. I remember reading one of Mark Manson's books
where he talks about wanting to be a rock star but didn't want to spend the long hours practicing guitar, taking on shitty
gigs. Mark never became a rock star, what he really wanted was the fame and people idolizing him, not all the other stuff
which is necessary for you to become a rock star. He did, though, became an absolute beast of a writer by, you guessed it,
doing a ton of writing on his blog and eventually turning into a best-selling author of multiple books.

And that lessons is something I seemed to, almost willfully, forget when it comes to being an entrepreneur. This game
requires you to have a predisposition towards doing, a shoot-first ask question later approach, if you will. It's
comfortable, and it definitely feels productive to be reading books, watching courses, following thoughtful
twitter arguments, from a successful entrepreneur. But in the end, that should take up a minuscule amount of your time spent
playing this game. Even science agrees on this.
While it's definitely a great business to be selling shovels in a gold rush, digging for gold is way more fun. But you
have to be digging, not read books about how to best use a shovel. This effect of gravitating towards action is something
I also find to be very true in software engineering. You want to have a little upfront planning and postulating about
the problem at hand, and then start cracking away at it. If you over-do the upfront planning, you more often than not
end up with overengineered systems that the problem or domain has to fit into, and not the other way around. If you do
nothing upfront, it tends to lead to a lot of wasted time because you didn't spend time gathering observation or analyzing
what it is, you're actually trying to do. Figuring out the root cause, the absolute minimum that needs to be in place
to solve a problem and the feasibility of it is the task at hand.

Therefore, one of my plans for the coming month is to get a process down for ideation and analyzing the outcome of the
ideation process (market analysis, problem area and feasibility etc.), by actual doing it. The next part will be a review
of some of the most common practices and advice I see, when trying to learn about this topic. Should make for a fun
re-visit a month or two in the future.

The indiehacking/entrepreneurial process

Warning: there is a massive amount of postulation here, all of this is to be tested over the coming months.

For a long time, my biggest problem was actually having ideas. I knew I was supposed to look for problems, in my personal
life or professional, and draw experience from that. But, and maybe I'm super lucky and don't have lots of problems and
the authors of such advice have a very trying life, it didn't really lead me anywhere. Luckily, over the past couple
of years, this advice has become more nuanced as multiple people have contributed. I love Jakob Greenfeld's piece
on it, where he illuminates multiple viewpoints from different authors, which actually gave me some sort of success in
terms of generating ideas. Because, let's be honest, most ideas are shit, so you need to have lots of them. One way to
look at this is through the creativity faucet where you treat creativity
as a process of dumping your brain into the keyboard, so you get as much of the bad stuff away, to make room for the
good stuff. Generally, the advice seems to be to use some sort of prompt in combination with brainstorming to generate
lots of ideas. Getting into the habit of this, you start making lots of connections that will lead to more interesting
thoughts over time. I've already had some success with this; Jacob Lang's Step One covers this great with lots of
examples and different prompts, which gets you far and wide in terms of potential ideas. I don't expect much to change here
except picking out the prompts that seem to trigger the most interesting ideas.

A lot of the literature focuses on problem/idea validation as an integral part of overall processes,
as if you do X steps the idea is validated, and all that is left to do is just to build it and money will flow[0]. I was a
stout believer in this for many years (having no actual experience doing it, of course), until I read Jason Fried's
Validation is a mirage. What are we, after all, validating?

Ideally, we want to validate that we are not wasting our time on a product that solves a problem that nobody has and
exists in a market where we can actually compete. But aren't this what market analysis is for? We wouldn't actually
know if our product solves a problem well-enough until we put it out there and charge money for it. That is the first
order of business for an entrepreneur; make a product that competes well enough to make money. But that's can only occur
once the product is out and in front of people, everything before that is hypothetical. This makes me think that we should
instead, spend our time looking for signals that this might be a good problem space to spend our time in. Just because
you can drive traffic to a landing page does not necessarily mean you have validated your product. I've read quite a
few threads from people who have built a subscriber list leading up to their product launch only to be celebrating
with crickets after they release the actual product.

I guess it might be a question of mindset, if you're convinced you've solved the problem in a valuable way because you
validated the idea using traditional steps, you might miss obvious red signs that are right in front of you.
It's the "assume your wrong and try to be less wrong vs assume your right and try to be more right" situation. One can
lead to over-confidence, while the other leads to critical questions and analysis. You want to be in the latter category
when embarking upon a journey of unknown-unknowns, we don't know what we don't know. So, do more
market analysis, tests hypothesis like "can I compete on keywords in a financial sustainable way" or "can I
convert people that land on my page", how is the competition and is the market big enough or me to enter or is it too
big that you can't compete given your current resources. Add the score up in the end and see if you think it's worth
pursuing, but only from there can you start validating your product; by building it and putting it in front of people.

The turnover rate of how fast you can do all of the above is what determines your success; getting as many quality
tickets in the raffle as possible, so you increase your chance of getting lucky. Because you need luck to play it's part
as well before you start seeing those tasty internet dollars rolling in.

Resources:

How to Validate Your Idea & Launch to $7k in Recurring Revenue

One Month One Dollar - Part Two

MBV — Sun, 31 Mar 2024 15:55:42 +0000

It's currently 14:30, February the 28th, 2023, while I'm writing this update to the challenge. I'm going to make it short.
We didn't make it. We actually didn't even get anywhere close to making it. It was a complete failure of a challenge.

Okay, okay, that's properly a bit too gloomy. It was kind of expected that we wouldn't make any money after the month
was over. But what we didn't received in terms of monetary gains we got in (quoting the great (scam) entrepreneur Tai Lopez)
knowledge. We also decided to make some adjustments to the challenge
to keep it going, let's break it down.

What went wrong

First thing first, if you attempt to do this challenge, don't select the shortest month of the year to do a
1 Month 1 Dollar challenge. That's just shooting yourself in the foot. Additionally, don't pick the same month where you've
planned holidays with your girlfriend, thinking you can just do a little bit of work. Especially, don't plan two holidays
in the shortest month of the year for this challenge. I'm only half joking here, but seriously, we didn't give this
challenge the focus and energy it required. We should have picked another month when we both could focus, at least
40 hours a week fully into the project.

Next up, and this is the biggest one, we started outreach too late. We did get some early promising result
by doing (personalized) outreach on LinkedIn. I think we might have reached out to around 50 people in total which
resulted in ~8 getting back to us, and out of those ~8 people, 1 wanted to jump on a call with us, showing some
interest in the product. The excitement quickly died down when we discovered that the person didn't really get
our value proposition. Both good and bad, we have new information we can react to, and it helps points us in the right
direction. However, we didn't get any new calls or follow-ups after this. It could be due to people just being busy
but I would guess more that we didn't pursue this aggressively enough.

What did we learn

There is something about learning a new skill where it only really manifest itself in your mind, when you try to
actually do that skill. Most of this advice is already flooded on the internet, in start-up/bootstrapping forums etc.
But, basically, it can be boiled down to:

Get a landing page out there quickly | forces you to describe your product and makes it more concrete
Write down your assumptions from the beginning | continuously test and adjust them to new information
Reach out to people even when you have a half-baked notion of your product

The first two points are pretty self-explanatory, if you're not sure what is meant by point 2, check out the original post.
I do think the third point is the main learning from this challenge. You got to focus on what actually moves the needle
and pushes you towards a profitable product, which means getting people to give you money. It's extremely comfortable to
just build build build (especially as software engineers) while having wet dreams about all the monthly reoccurring
revenue you're going to receive in the end. But that is, on average, not really a productive strategy even though it
might feel like it. Sure, all your assumptions could be spot on and your big-bang release is going to be a raving
success. Most are just not. So speak to more people, look up people in relevant positions/groups on LinkedIn, find
people following your competitions on twitter, scourer those facebook groups and write to people. Also, read make sure
to read The Mom Test.

A blank piece of paper

Was it all bad and with no successful experiences? Not at all, one process that really helped us kick this off was the
habit of continuously writing down ideas each day.

There is something strange about the blank page effect; it can stare about at you for hours on end, almost paralyzing
your ability to get started. Should I start with X? Should I include Y? But what about that incredibly important Z? It's
seriously frustrating, and it was something that both Rasmus, and I were feeling a bit anxious about before starting the
project. For the past ~2 months, we had been meeting up once a week to discuss ideas, problems, businesses, everything
under the sun related to bootstrapping companies. I myself had been trying to (and somewhat succeeding) write down
10 ideas a day and you quickly come to realize, ideas are not that hard to get. It's definitely a muscle that can be
trained with consistency just like, well, pretty much anything. Most of that comes out is dog shit, like my brilliant idea
of uber but for walking dogs - list where you want to go, and a dog will walk there with you, not exactly a Zero to One
idea.

It did give us a list of 200+ ideas to start from, which means, 200+ ideas that can be combined, modified,
thrown-away or lead you down new paths to even better ideas. The tiniest bit of exploration sparked new ideas which led
us to ask new questions that we haven't considered before.

You do see this a lot on forums like indiehackers where people, on a quite consistent basis,
will be asking how people generate ideas. And every time, you will see the same (more or less) generic advice:

solve your own problems
look for problems at work/with clients or what issues are your friends having

These never worked well for me, mainly because I forgot to actually think about these things while doing my work. So it
felt like nothing was actually occurring to me, and I was left with no viable product ideas and therefore didn't really
build anything. Making a habit out of writing down 10 ideas a day did work, however, so if you're struggling with this
give it a try. You'll soon start thinking about potential products consistently.

Change of direction

Is it over? No, but we're making some adjustments. It will no longer be 1 Month 1 Dollar challenge but instead:
1 Year 1 Noodle.

This is basically just to say that the challenge will continue in the sense that we will keep trying to get our hands on
that sweet magical internet money. And then post an update once in a while.

We'll continue working on the product for the foreseeable future until we either gets some traction or kill it, it's
definitely too early to kill it. If you want to check it out, it can be found here databayz.

Outside of the challenge, I have also been working on another project to help do customer discovery on reddit:
Townsquare Digest. The process for this is going to be roughly the same as described
above, outreach outreach outreach. But more on that in a later post.

One Month One Dollar - Part One

MBV — Sun, 31 Mar 2024 15:54:33 +0000

High-stakes cash games in Vegas

When I was a younger and slightly dumber man I was captivated by the allure of online poker and high-stakes No-limit Hold em.

Going to Vegas and battling it out for million of dollars with all the high-rollers, was a big dream of mine. I would watch all the tutorials I could get my hands on, read whatever poker book was available to me, and, binge every poker show out there.

Now, this was a few years after the golden age of online poker which was kicked off by Chris, I
shit you not, Moneymaker. An accountant from Nashville and hobby poker player whom is said to be responsible for online
poker exploding, after he won the WSOP Main Event, gaining entry based on a satellite tournament he played online. For
the unfamiliar with online poker, a satellite tournament is basically a (often much) lower priced tournament that gives
the winner(s) entry to a higher-stakes tournament. I'm getting a bit sided-tracked here, anyway. In this post-chris-money
maker-winning-wsop-main-event environment a lot of young people (i.e nerdy white guys, try and guess why it was
appealing to me) started printing money online. This lay the basis for a show called 2 Months 2 Million which was
centered around this bunch of young poker pros who pooled their bankrolls (money) and gave themselves two months to make
a net profit of 2 million dollars. They failed. Well, sort of failed, they made a bunch of money just not the two million
dollars.

I never made it into professional poker nor did I go to Las Vegas. I did, however, go into Software Engineering and
happily forgot about poker and that show. That is, until I stumped across this article about a guy who challenged himself
to start from zero and, after one month, have built a product that some stranger on the internet would pay money for. I
loved the idea and was immediately reminded of that show long time ago. The difference now being that I actually have
acquired a skill set that would allow me to have a real shot at completing this challenge. Even better, I also have a
business partner to do the challenge with me. So, from the 1st of February 2023, Rasmus
and I will be attempting at starting completely from scratch and after 30 days of work, have a product that will have
made more than 1 dollar. A million seemed to bit too ambitious.

Isn't this just 12 month 12 start-ups but shorter

Maybe, yes, kind of, the chance that an idea is completely original is absolutely minuscule. I've tried to (mentally,
at least) do the challenge made famous by Pieter Levels but never really got it kicked-off. I started
the preparations, loaded up on some ideas, picked a date but then excuses got in the way (work, family, friends take your
pick) and it didn't go anywhere. Taking potentially 12 months off, or built 12 new projects on the side while working
full-time just seems really exhaustive to me. But, dedicating one month to try and kick-start a project is way more
manageable; both financially and risk-of-burn-out wise. I used to build a lot of side-projects in my spare time when I
just started as a software engineer, which was incredible fun, but as I've gotten older I just don't want to spend my
time coding for 12+ hours a day. This limited time scope seems to be a much better fit.

Wait, but why?

Wantrepreneur: A person who aspires to be an entrepreneur, especially one who never realizes this ambition

At the beginning of January, I wrote about Wantrepreneurship
and how I've been wanting to break into this for quite some time now. I must have read every single book under the sun
about entrepreneurship, creating micro-saas businesses and start-ups. Yet, I haven't even launched anything always
holding back because I lack a good idea or because it's not the right time or insert-other-common-excuse (actually,
it's not entirely true. I did launch something called called ByGourmand intended to provide subscribers with a wide
variety of recipes. If you're currently thinking "so, a shittier version of HelloFresh?", you've got the right idea).
But, I seemed to have forgot an important lesson; one I learned straight after uni having made the jump into software:
you don't really learn anything until you take action. I.e you need to do that thing you're consistently talking about
and you will get better at it. I've spend long evenings on YouTube learning to program but equally long evenings building
and writing software. And the last step, that's what actually makes the difference. One of my favorite twitch streamers,
theprimeagen, has a perfect acronym for this: TITS (time in the saddle, Elon Musk
would love it - spend more time doing
that thing.

Having set an optimistic deadline while also made a public announcement is hopefully going to be the combination that
really pushes this into gear. Additionally, teaming up with Rasmus for this project, I think it creates a situation
that will push me to take a real stab at this.

So, what's the plan? I'm glad you asked.

The Plan

This is quite an ambitions deadline, so we're going to be cutting quite a few corners wherever possible. Since we're
starting from zero we can't really have a already researched idea ready to go from day one, we need to start out doing
some good old fashioned brainstorming. No more than 2 days have been put aside for this, along with preliminary
research into potential customers and the market. On the third day we're taking a pick, putting maybe one or two ideas
in reserve, and then starting to creating a landing page and doing outreach. Based on the feedback, we will start building,
do more outreach, do more building. Rinse and repeat until we either run out of time, or, make our first dollar.

Honestly it's a fairly simple approach but given that this is mostly a learning experience, I would argue it's the ideal
way forward. The thing about planning for an activity, that you are unfamiliar with in practice, is that it quickly
turns into a big soup of postulates. In the famous words of Mark Hanna:

Fugayzi, fugazi. It's a whazy. It's a woozie. It's fairy dust.

It's not fucking real, but two people can quickly convince themselves about X hypothesis being very real because the world
would be so much sweeter if it was. The plan can be summed up to be hypothesis-driven entrepreneurship if you will. We
have a problem, think up potential experiments that could solve said problem, run them and evaluate. Continue what works
and stop what doesn't. It's worked for old Greeks, I'm sure it can work for a couple of danish guys.

Speaking for myself here, reaching out to strangers on the internet have always been an Achilles heel for me. I'm not
sure why, the thought of it has always made me imagine myself as the human form of a spam email. I fucking hate spam;
Escobar had burner phones, I have burner emails. But I've also slowly come to accept the fact that this is going to a
necessity, if I ever want to move forward in this process. My first job in Uni was as a phone sales man where I made a
sell in my very first phone call, and then never made one again. Hopefully, this month goes a bit better.

Likely Outcome

Most likely, this project will not make much money within this month. A firm rule is that we cannot beg friends and
family to buy the product/service, so we unfortunately have to take the long route. Additionally, we could go the cash
grab route and do AI profile pictures but that just seems boring at this point. Saas businesses have never been easy
and even though we might get lucky and strike gold, the takeaway here is hopefully going to be a lot of learnings that will push both of us into the next step in our entrepreneurial journey.

Top level internal is unnecessary

MBV — Sat, 16 Mar 2024 14:31:03 +0000

I recently got into a discussion on twitter; I suggested that root level internal directories doesn't make much sense when you're creating applications that are not a library. It's conventional wisdom in the Go community, to suggest putting any code you do not want to expose to other applications inside the internal directory. And it's true, if you're creating something that is meant to important into other peoples code. The truth is, however, that most people are likely not working on other libraries but rather applications (apis, web servers, etc). Take this layout, for example:

- app
    - cmd
    - app
        - main.go
    - internal
    - services
    - repository
    .git
    go.mod
    go.sum

A standard repository on github, gitlab etc.

What does internal protect against here? Literally any other package in this application will be on the same level as the internal directory, all will be able to import from it. The only situation, where this would not hold true is if you were to take this entire code and copy it into another repository. And, honestly, when was the last time you did that?

So, yes, it's technically correct (best kind) that using root-level internal directories provides encapsulation (which is good). But, in practice, you don't really get any of the benefits of internal. If we were to add a view package to our application, and wanted to start grouping different views together in a sub-package, an internal directory suddenly starts to make sense. Let me illustrate:

- app
    ...omitted
    - views
    layouts.go
    - home
        home.go
    - about
        about.go

When does internal make sense?

If we had some layout components (base, dashboard etc) in the layouts.go file that aren't exported, home.go and about.go wouldn't be able to import it as both are no longer part of the view package. If we export it, any package in the application can import it, obviously not desirable. But, we can add an internaldirectory, export the layout components and encapsulate them from the rest of the application, like so:

- app
    ...omitted
    - views
    - internal
        - layouts
        base.go
        dashboard.go
    - home
        home.go
    - about
        about.go

This is kind of a pet peeve of mine; it isn't really that important now that we have gotten modules (pre modules was one of the reasons for internal which I will get into in a bit) and as long as you stay consistent about what goes into internal, it's all good. I personally prefer to use pkg, as that to me at least better communicate what the directory is for. And I've seen quite a few Go applications in my time now, they either use internal or pkg and most often contain the same kind of code. Auxiliary packages that are needed for the application as a whole but does not contain core business logic. Code to interact with aws, wrappers around a library to send mails, a queue etc. To me, for non-library code, this is the most important thing to get right, what do you expect to find in the directory not as much the same (again, you don't get any of the benefits from internal if you place it in the root).

Why do we have internal?

To answer that question, we have to go quite a big back. All the way back to the time before modules, where the creators of Go thought it a great idea to force everyone into their (Google's) preferred way of structuring code. You'd be forced into something like this:

- $GOPATH
    - src
    - github.com
        - user
        - repo
            - cmd
            - internal
            - pkg

Before Go 1.4, you could only have local and global components which could lead to frustrating situations where you'd like to have code that is only used within the module its defined, and not part of the public repository. And in Go 1.4, the solution to this was introduced with the introduction of the internal directory. The Go compiler will verify that the package doing the import is within the same module as the package being imported.

With the above structure, using internal as the default makes a ton of sense as it's quite easy to accidentally expose something that otherwise should've been kept private. But with the introduction of modules, this doesn't really apply. Reading internal makes me thing of something that's internal to a package not the application as a hole.

Does this really matter

No.

I tend to pick this "fight" just because I think its a bit funny how some people will go out of their way to argue that internal is the only correct way to do it.

As argued above, if you're building something that is not meant to be imported into other peoples code, this is good advice. If you're building an application, consider using pkg. But most importantly, be consistent. We spend too much time in software focusing on right and wrong as black and white, but it all depends. If the code that I'm arguing should go into a pkg directory feels more natural for you and your team to be in an internal directory, then by all means, do that.

Just be consistent.

Pulumi vs Terraform

MBV — Mon, 12 Feb 2024 13:21:38 +0000

I've started a new contract, where I'm tasked with consolidating the company's infrastructure under aws. They (the company) are starting to see some serious growth, have a team in-place currently using a PaaS and various other cloud providers. Infrastructure as code tools have been popular for a long time; I've mostly been a bit skeptical about using them given a small team size, often the ui is more than enough. But, when you need to start spinning up multiple environments (dev, staging, prod, demo etc), this start to become a lot of overhead. An issue I've often seen is that it becomes the responsibility of one or two developers to maintain this, as the rest of the team is not familiar with the tool and language syntax. Obviously, this is not ideal, so I wanted to research the space to see if there was a middle way where you can get the benefits of IaC, without the overhead of having to learn a new language and tool.

I've previously used terraform in other companies I've done work for but never from scratch and (full disclosure) always found the experience rather frustrating. So, I began to search for terraform alternatives and stumbled upon pulumi, which lets you provision resources using a number of programming languages (including Go) and pulumi's aws provider support is excellent.

But, as in any other situation where you are considering marrying yourself to a new tool, it's very worthwhile to do some research and preferably build some small proof of concept before you commit. I've had some interesting discoveries through this process, why I choose pulumi over terraform and when I would consider terraform over pulumi.

Short intro to terraform

I think it's fair to say terraform has been the big dog, in the IaC space for a long time. They've support for the majority of cloud providers, been battle-tested at scale and has a bunch of great tutorials. They also used to have a open source license, using Mozilla Public License v2, but decided recently to switch to what is known as the Business License, BSL, which can be considered as "source-available". I will not go into a longer debate about why this is not ideal, if it's fair or not, but it does pose a threat to the development of third-party tools and your options in terms of making tweeks if needed.

For the unfamiliar, terraform is written in their own "DSL" known as HCL. It's a declarative language, so you tell it the desired state you want your infrastructure to be in, and terraform figures out how to get your infra to that state.

A short example of how you would provision a S3 bucket would look like this:

resource "aws_s3_bucket" "my_bucket" {
  bucket = "my-bucket"
  acl    = "private"
}

Which would be written in a file with the .tf extension. As we'll see later on, you have lots of options in terms of structuring your code into modules, different types of options for code-reuse: data blocks, variables, outputs etc.

Short intro to pulumi

Pulumi is just a very strange name. Growing up, I had a friend we for some reason called pummi (or maybe it was bummy, can't recall for sure) and all I can see when reading the name, is his face. They're also rather new in the IaC space but have made some significant advances. Some years ago, I worked at a place who utilized AWS's CDK, and I imagine people would feel pulumi to be quite similar.

You've a range of choices in terms of the language you want to use, JS/TS, python, Go etc. All having the same API, so switching shouldn't be the biggest issue. I write all my stuff in Go these days, so the same example as above written using Go and pulumi, would look like this:

func main() {
    pulumi.Run(func(ctx *pulumi.Context) error {
        _, err := s3.NewBucket(ctx, "my-bucket", &s3.BucketArgs{
            Bucket: pulumi.String("my-bucket"),
            Acl:    pulumi.String("private"),
        })
        if err != nil {
            return err
        }
        return nil
    })
}

To anyone that has ever written Go, this should immediately make sense, even without knowing the pulumi api or the aws ecosystem. And this is one of the main benefits we will get back to, again and again, over the course of this post.

Pulumi is also "true open source", in the sense that they operate with an Apache 2.0 license, which is a lot more permissive than the BSL license terraform uses. Which is great for the community and adoption rate, but, so was terraform at one point. For now, it's a huge plus.

However, it isn't all sunshine and rainbows. Pulumi is still young, so documentation is still not great when actually writing code. Their general purpose docs do make up for this, but it can be frustrating having to jump in and out of the editor to look up what you're currently working on.

Building the POC

All of the applications are containerized being we can take full advantage of aws's elastic container service, so the POC was a simple hello world api packed in a docker image, served using fargate and a load balancer in front. I've been aware of ECS and fargate (plus, vpc, subnets, security groups etc), but never used them, so there was also some learning involved.

The first thing that became clear, terraform has a lot of great documentation and tutorial available. While pulumi does also have documentation and available tutorials, they are not as "complete" as what you find with terraform. However, I would find myself hitting a problem in the infrastructure setup with terraform, switch to pulumi and even with less available information, I was able to reason my way out of the problem because I simply had to read Go code. The overhead of a new language and platform would be frustrating, but staying in something I already knew, removed one leg of complexity and the aws concepts was now the only thing I had to figure out. I suspect that would be one of the main feelings of other engineers that don't do full-time devOps as well.

I managed getting the poc running using both tools but a big help came from staying in an environment I was already familiar with. I'm a heavy (neo)vim user and while there are lots of good plugins for terraform, the Go environment is something I'm in everyday so nothing new were needed.

If I dedicated the time, I don't suspect it would take super long to get familiar with hcl and terraform to the point where the environment would feel close to as familiar as Go. But, given that I wouldn't have to work everyday with terraform after getting the initial setup done, that familiarity would soon start to fade. Meaning, every time a new infrastructure part had to be added, I would need to re-familiarize myself. And, for the rest of the team, this would be same but with the added overhead of getting into a part of the code they didn't help write. This could be mitigated by involving everyone from the get-go, but this is also a business, there are other priorities that needs building too.

Documentation might be the answer some of you are screaming at the screen right now, and I hear you, but documentation quickly goes stale. This is especially true in smaller teams where things move fast, if the majority can be documented by the code itself, you save yourself much trouble.

Why I choose pulumi over terraform

I ended up going with pulumi. I know from experience that the "correctness" of these choices only show its true self after you're well passed the point of making the switch to the other candidate feasible. But, from my current point of view, this is what will best benefit the team I'm currently working with.

Two questions kept coming up, as I were trying to decide on which tool to use:

What are our infrastructure requirements now and in the future? (i.e. are we ever going to need a full-time devOps)
What does the current team know? (i.e. programming languages, cloud providers etc.)

The first might seem arbitrary, but the likely hood of finding a full-time devOps engineer that knows terraform is properly much higher, than them knowing Go and pulumi. Given that the needed infrastructure now and in the foreseeable future most likely will not require a full-time devOps role, terraform begins to feel a bit overkill. It's a new language, everybody on the team would need to learn as well as any best practices that comes with it. All the while, getting into the nitty gritty of aws which will also be a new learning experience for the team (myself included).

The second, to me, was very important since I've, from multiple experiences, seen what happens when you have a big terraform setup that maybe one engineer was in charge of creating. Most of the time, it ends up being that one person that has to maintain and fix it, effectively making them a devOps person (which might be against their will -> they switch job -> team is fucked). The rest of the team, despite the person writing documentation, making presentations etc etc, are still very reluctant to touch it. It's something they are unfamiliar with, the infrastructure system can be quite difficult to "have in your head", you can potentially take down everything without knowing how to get back to a running state. It creates a lot of friction.

You might argue: "well, they will just have to learn it" and I agree, that would be a sensible path forward. But we're not trying to solve a technical problem right now, but rather, a human one. And without allocating the time for people to learn it, it probably will not happen. So, in lieu of a devOps team, you need to optimize for what gets people to actually want to learn infrastructure. Over the two weeks I spend researching and building some POCs, I would often find very good tutorials and materials on setting up infrastructure with terraform. But as soon as I hit my head against wall, not knowing a specific syntax/setup/approach used in the tutorial or what the specific aws services was about, it become very hard to move on. Multiple times, I switched back to pulumi where their docs had examples in Go code, which was much easier for me to wrap my head around since I already knew Go. I "only" had to figure out the aws part. After, I would switch back to terraform and fix the thing with my new understanding from pulumi/go. But it's important to note where that knowledge originated from, by reading code, which most of us are doing pretty much every day.

Taking away the overhead of both having to learn a new language and a cloud provider, to only the cloud provider, helps a tremendous amount. You also get the added benefit of already knowing best practices, when and why to break those practices. You get an api from pulumi so you can start provisioning infrastructure directly in your application(s), which makes spinning up demo environments for sales processes trivial.

When I would choose terraform

Up to a certain point, something becomes a standard because of it quality and appeal to a large part of the tools target group. After that point, better alternatives might actually exists but since the standard requirement is to know said tool, everyone else just continues learning that (I'm looking at you, React. Switch those frontends back to html and use htmx for interactivity!). If that's whats going on here with terraform and pulumi is hard to say. I actually had a much better experience exploring terraform, compared to previous experiences, where I only had to tweak it a little bit.

If your team includes members that already know terraform or you know that you have some large scale infrastructure needs in the near future, that will include a dedicated devOps team, terraform still seem like the better choice. I did pick terraform and hcl up rather quickly, but I'm still quite confident that this would fade again after not using it daily, so if you don't have that limitation, terraform would win for me.

The main argument from my side here is to remove friction from your team when they don't deal with infrastructure on a daily basis, learning a new language while your PM is blasting new tickets your way like his life depended on it, is tough. And having to wait for the one team member who truly knows the infrastructure code coming back from holiday to fix things, is very sub-optimal.

Was this a good decision

It's too early to tell, I will revisit this in a few months once we've had some more production experience with it and the team has started working more with it as well.

How to build a (simple) blog using Rust

MBV — Wed, 07 Feb 2024 21:15:39 +0000

So, you have been reading about rust for quite some time, learned that it’s been the most loved programming language multiple years in a row in Stack Overflow’s annual survey and you want in. But, you’re not a system-level programmer and Reddit keeps telling you to ‘use the right tool for the job’ and ‘rust is more aimed at low-level stuff like writing databases and programming drone’ (the first is actually good advice, I just need it to create a dramatic setting). So you get disheartened and choose to use what you’ve always been using. I think that’s a mistake as rust is an amazing language both in terms of speed of the execution but also developer experience. Not many languages give you a handy pair-programmer, i.e. the compiler, that only gives good advice, and shuts up in the meantime (rust’s compiler is amazing). I’ve been toying with Rust for some time now and overall it’s been a great experience (both learning and developer). So to get my hands a bit more dirty I decided to re-write my blog using Rust which was previously built using Next.js (This has also been a thing coming since I’m doing my best to move away from SPAs and Javascript, but that’s for another post).

The goal of the rewrite is to get something out which could serve as a learning example and be built upon to further my understanding of the language. I hope this post does the same for you.

Side note: I write Go for a living and have been coding professionally for the last ~5 years, so while I’m confident in my overall skills this tutorial might not always show idiomatic Rust and best practices. There’re a bunch of much more rust-abled developers than me who have written some great material, much of which lay the foundation for what I’m about to show you. At the end of the article, you’ll find a resource list that you can check out after reading (and hopefully coding along) this article. If you’re just interested in seeing the code you can find the repo here.

Kicking things off

So, to build your new blog using Rust, you're going to have Rust installed on your system and then run cargo new awesome-blog.
Then run cd awesome-blog into the directory, open Cargo.toml and add the following under [dependencies]:

[dependencies]
tera = "1"
actix-web = "4"
env_logger = "0.9.0"
lazy_static = "1.4.0"
actix-files = "0.6.1"
pulldown-cmark = { version = "0.9.1", default-features = false }
ignore = "0.4"
toml = "0.5"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0

Feel free to use another version than the ones listed above; it's what I used at the time of writing this piece.

Let me briefly touch upon some of these libraries. The main one here is actix-web which is one of the big players in Rust web frameworks.
There are multiple other choices (see for example warp, rocket, axum) but I thoroughly enjoy actix's API and have built multiple projects using it. Furthermore,
it performs really well and can handle a lot of requests.

Next is serde which is a library you most likely are familiar with if you've done any Rust programming. If not, it's basically a set of
methods for serializing and deserializing data structures efficiently and generically. If you can't find an implementation for your desired data structure,
you simply implement the methods for your data structure and you're off to the races.
For this project, serde will be used for serializing the metadata of a blog post (also named front matter) from a .toml file into a Rust struct.

Lastly, I want to touch upon tera which is the template engine I use for this blog. It's inspired by Jinja2 which you might be familiar with if
you've ever done any web development with Django. Back in the day, I started out building web apps using Django and really enjoyed the way it does
templating so wanted to replicate that experience in my personal projects. In a world where everything is components and the slightest duplication is a
death sin, the experience of using "old school" templating has been a relief.

A simple server

Right out of the box, after running cargo new awesome-blog, you should be able to open up src/main.rs and be presented with something like this:

  fn main() {
    println!("Hello, world!");
  }

Executing the command: cargo run will (are pulling all the dependencies) output the classic Hello, world! in your terminal.
We're going to treat main.rs as a thin entry point into our application, which basically just means it should do some high-level configs
and call start-up functions. With that requirement, we end up with something like this:

use std::net::TcpListener;

use awesome_blog::start;

#[actix_web::main]
async fn main() -> std::io::Result<()> {
    std::env::set_var("RUST_LOG", "actix_web=info");
    env_logger::init();

    let listener = TcpListener::bind("0.0.0.0:8080")?;
    start_blog(listener)?.await?;
    Ok(())
}

Not much going on here, except a bunch of errors, so let's break it down real quick:

We set an env variable RUST_LOG that determines what kind of log statements we're getting out
We initialize a simple logger that is configured using env variables
We create a listener that listens on port 8080
We pass the listener to the start_blog and we're off to the races

We will deal with start_blog later, which means that the only thing left that hasn’t been addressed is #[axtic_web::main].
This is macro, or more precisely, a proc_macro which generates some code for us that makes our awesome_blog run asynchronously.
The rust team made a conscious decision not to provide an async runtime in the standard library but instead opted to provide essentials and let the community do the rest.
The reason(s) why are manyfold, but one reason could be that there is no consensus on what such runtime should look like.
That topic is also out of scope for this article; you just have to know that it generates code for us, which makes an async runtime possible.

Peeling off the layers

We need to tackle the start_blog function, so create a new file called lib.rs under src and open it up.
We will be building it in steps, starting with adding the code needed to run an HttpServer:

use std::net::TcpListener;
use actix_web::{dev::Server, web, App, HttpResponse, HttpServer, middleware};

pub fn start_blog(listener: TcpListener) -> Result<Server, std::io::Error> {
    let srv = HttpServer::new(move || {
        App::new()
           .wrap(middleware::Logger::default()) // enable logger
           .route("/health", web::get().to(HttpResponse::Ok))
    })
    .listen(listener)?
    .run();

    Ok(srv)
}

So, we did a little more than just create an HttpServer. We also added in the logger defined in main.rs and created a route that
pings back a 200 Ok response.

Most of the above code should make sense to you (assuming familiarity with rust) but let's quickly touch upon the move keyword.
According to the docs: a move converts any variables captured by reference or mutable reference to variables captured by value.
If you're a bit confused about the above statement, join the club. But, it can be broken down into a more (to me, at least)
understandable sentence. What it boils down to is rust's ownership model and how it deals with memory (de)allocation and references.
What we see in this line HttpServer::new( move || { App::new() }) is a closure and closures might escape.
To (over)simply, actix_web will spin up multiple instances of your app, assuming a multi-thread environment, so variables/values passed to App::new() might outlive App, i.e. the closure might escape.
That would leave dangling references laying around that are not accounted far and that's no bueno in rust.
So, to get around that, we tell App to take ownership of the values passed to it so that each instance owns what is passed to it.

Even though we're going a bit "olds chool" here and using pure simple html we would still like to have some reusability.
And this is exactly where tera comes in. However, since the templates will remain the same once we chuck them into prod, it will probably be a good idea only to load them one time and then pass a reference.
To do this, we're going to be using lazy_static! so go ahead and add the following to your lib.rs:

use std::net::TcpListener;
use actix_web::{dev::Server, web, App, HttpResponse, HttpServer};
use tera::Tera;

#[macro_use]
extern crate lazy_static;

lazy_static! {
    pub static ref TEMPLATES: Tera = {
        let mut tera = match Tera::new("templates/**/*.html") {
            Ok(t) => t,
            Err(e) => {
                println!("Parsing error(s): {}", e);
                ::std::process::exit(1);
            }
        };
        tera.autoescape_on(vec![".html", ".sql"]);
        tera
    };
}

pub fn start_blog(listener: TcpListener) -> Result<Server, std::io::Error> {
    let srv = HttpServer::new(move || {
        App::new()
           .app_data(web::Data::new(TEMPLATES.clone()))
           .wrap(middleware::Logger::default()) // enable logger
           .route("/health", web::get().to(HttpResponse::Ok))
    })
    .listen(listener)?
    .run();

    Ok(srv)
}

All good, but one thing missing. We need to add a templates directory in the root directory so go
ahead and mkdir templates. We will get back to main.rs, lib.rs and all that soon but for now,
let's quickly touch upon the templates so we have something to show.

Dude, where are the components?

I started out with react when I got my first job as a front-end developer. Components were all the rage (well, they still
are but this makes me sound a bit older and wiser) back then and it was how I started thinking about UI elements. I've since
changed my mind quite a bit on this but that will be for another article. As you know, we're not going to be doing components here
but partials and blocks so in your templates directory, create a file called base.html

<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
    {% block head %}
    <title>{% block title %}{% endblock title %}</title>
    {% endblock head %}
    <link rel="stylesheet" type="text/css" href="/static/css/index.css">
</head>

<body class="flex flex-col justify-between min-h-screen font-sans leading-normal tracking-normal">
    <div class="h-16">{% block header %}{% endblock header %}</div>

    <main class="container flex-1 w-full md:max-w-3xl mx-auto overflow-x-hidden">
        {% block content %}{% endblock content %}
    </main>
    {% block footer %}
    <div class="flex w-full h-20 justify-center items-center">
        &copy; 2022 by <a class="pl-2" href="https://awesomeblog.com/">Awesome Blog</a>.
    </div>
    {% endblock footer %}
</body>

<script src="/static/js/highlight.min.js"></script>
<script>hljs.highlightAll();</script>
</script>

</html>

The above is going to be the base for everything else we do.

The part that's going to let us extend base.html (or, inherit..) is this {% block content %}{% endblock content %}.
Let's see how that is done, so go ahead and create a new file under templates called home.html:

{% extends "base.html" %}
{% block title %}Awesome blog | by DeveloperMan{% endblock title %}

{% block content %}
<div class="w-full h-full px-4 md:px-6 text-xl text-gray-800 leading-normal">
    <h1 class="text-center text-3xl">All about tech</h1>
    <h3 class="text-center text-lg mt-2 text-gray-600">More words describing the blog here</h3>
    {% for fm in posts %}
    <a class="mb-3" href="/posts/{{fm.file_name}}">
        <div class="flex flex-col mb-5 px-4 py-6 cursor-pointer">
            <h2 class="text-2xl font-semibold hover:underline">{{fm.title}}</h2>
            <div class="flex items-center mt-2">
                <p class="hidden md:block text-base ml-2 mr-2">{{fm.posted}}</p>
                <p class="hidden md:block text-base mx-2">{{fm.author}}</p>
                <p class="text-base ml-0 md:ml-2">Reading time:
                {{fm.estimated_reading_time}} min</p>
            </div>
        </div>
    </a>
    {% endfor %}
</div>
{% endblock content %}

First off, notice the {% extends "base.html" %} at the top. This lets us interact with all of the blocks we created in the base.html template.
Furthermore, we have a for loop in the markup: {% for frontmatter in posts %}. We can pass data as variables when serving the html as part of
the response to, say, http://awesomeblog.com/ which in the above case would return a variable named posts that is an array of posts.

If you try and spin up the application now, not much would be different, so let's implement some handlers/controllers/etc to actually serve some html!

The controller/handler layer

Now we're getting into the actual meat of this thing and starting to write some actual rust code!

We need a few things to get started, so go ahead and create a new folder under src called handlers.
In that holder, create a file called mod.rs and home_handler.rs. Let's start simple with some test data to show something when we load the home page.
Open home_handler.rs and add the following:

use actix_web::{get, web, HttpResponse, Responder};
use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize, Debug)]
pub struct Frontmatter {
    title: String,
    file_name: String,
    description: String,
    posted: String,
    tags: Vec<String>,
    author: String,
    estimated_reading_time: u32,
    order: u32,
}

#[get("/")]
pub async fn index(templates: web::Data<tera::Tera>) -> impl Responder {
   let mut context = tera::Context::new();

   let frontmatters = vec![Frontmatter{
    tags: vec!["Rusty".to_string(), "Test".to_string()],
    title: "Test posts".to_string(),
    file_name: "test_posts.md".to_string(),
    description: "Just testing out the system".to_string(),
    posted: "2022-08-09".to_string(),
    author: "MBvisti".to_string(),
    estimated_reading_time: 12,
    order: 1,
   }];

   context.insert("posts", &frontmatters);

   match templates.render("home.html", &context) {
    Ok(s) => HttpResponse::Ok().content_type("text/html").body(s),
    Err(e) => {
        println!("{:?}", e);
        HttpResponse::InternalServerError()
            .content_type("text/html")
            .body("<p>Something went wrong!</p>")
    }
   }
}

If you've been through the rust book most of this should make sense to you.

If not, let's do a quick rundown. We create a struct and derive (the #[derive()] on top of Frontmatter) some methods from serde
and the standard library. Next, we create a context through tera. Remember how we talked about providing data to the templates through our handlers?
that's how we do it. We then simply call .insert() on the context, provide the variable name and the data and we're good to go.
In the return part of the function, we put impl Responder so we just have to return something that implements Responder. It just so happens
that HttpResponse does implement Responder, so all there is left to do is to pull out the correct template from tera.
And since template.render() can fail we provide a backup that, admittedly, is a bit lazy for now but we can always come back and improve on this.
Lastly, you might be wondering how we actually access the templates we reference in the handler as templates: web::Data<tera::TerA>.

Actix has a neat way of sharing data though App::app_data and the struct Data<T: ?Sized>(_) type.
By passing something wrapped in web::Data::new to App.app_data (side note: we could improve upon this by passing web::Data::clone to app_data
instead of web::Data::new and just wrap our data once in web::Data::new and simply pass a reference to that in web::Data::copy. This is because Data
uses Arc internally which makes it very cheap to clone) we can now extract it in your handlers!

Lastly, before we can access our new shiny handler, we need to make it accessible so open up mod.rs under src/handlers:

mod home_handler;

pub use home_handler::index;

We just need to add a route so we can serve the content to the user. Open up main.rs and add the following:

use std::net::TcpListener;
use actix_web::{dev::Server, web, App, HttpResponse, HttpServer};
use tera::Tera;

pub mod handlers; // new line

#[macro_use]
extern crate lazy_static;

lazy_static! {
    pub static ref TEMPLATES: Tera = {
        let mut tera = match Tera::new("templates/**/*.html") {
            Ok(t) => t,
            Err(e) => {
                println!("Parsing error(s): {}", e);
                ::std::process::exit(1);
            }
        };
        tera.autoescape_on(vec![".html", ".sql"]);
        tera
    };
}

pub fn start_blog(listener: TcpListener) -> Result<Server, std::io::Error> {
    let srv = HttpServer::new(move || {
        App::new()
           .app_data(web::Data::new(TEMPLATES.clone()))
           .wrap(middleware::Logger::default()) // enable logger
           .route("/health", web::get().to(HttpResponse::Ok))
            .service(handlers::index) // new line
    })
    .listen(listener)?
    .run();

    Ok(srv)
}

Give it a spin and you should see (a rather ugly) web page with your blog post!

Static assets and the path of least resistance

While we were setting up our html templates you might have noticed what looks like tailwindcss classes and then promptly wondered what the hell kind of end result it had on the home page.
First of all, good on you for choosing great frontend tooling. Secondly, we haven't yet included the css needed for our styles to actually do something.

If you check out base.html, you will see this line: <link rel="stylesheet" type="text/css" href="/static/css/index.css"> which hints at we need a css folder within a static.
Now, we can go about this in a number of ways depending on what kind of traffic we expect to get and how (over)engineered we are planning to make this.

One option could be to serve the content of index.css through an S3 bucket and then point an URL directly at it which then becomes referenced here.
It would certainly keep the resulting binary smaller but it does add some overhead that is not really warranted right now (more complex CI/CD flow, handling various paths in dev and prod etc).
Another option (the one we're going with), would be to just add it as part of the overall binary.

It keeps things simple for now and should our blog become a raving success we can most likely worry about this at that point.
So, create two new folders at the root, namely static and under that, css. Next up we need to do some tailwind setup that you can find on their own site, so for your convenience, here are the cmds you can run in your terminal:

mkdir tailwind && cd tailwind

npm init -y

npm install -D tailwindcss

npx tailwindcss init

touch base.css

Almost there, just a few more steps so open up tailwind.config.js and copypaste the following:

module.exports = {
  content: ["../templates/**/*.{html,js}"],
  theme: {
    extend: {},
  },
  plugins: [],
}

Two more things, open base.css and copypaste the following:

@tailwind base;
@tailwind components;
@tailwind utilities;

Lastly, open up package.json and add the following under scripts:

{
    "scripts": {
        "watch-css": "npx tailwindcss -i ./base.css -o ../static/css/index.css --watch",
        "build-css-prod": "npx tailwindcss -i ./base.css -o ../static/css/index.css --minify"
    },
}

Those two scripts give you a cmd to watch for any changes in development and then build for production when we merge.
That also implies that you would have to remember to run build-css-prod before merging to master.
This can definitely be improved upon but since you're the lead (and only) developer on this, it will be fine for our needs.

Last step, we need to make actix_web aware of our static files. So open up lib.rs and update it as follows:

use actix_files::Files; // new line
use actix_web::{dev::Server, middleware, web, App, HttpResponse, HttpServer};
use std::net::TcpListener;
use tera::Tera;

pub mod handlers;

#[macro_use]
extern crate lazy_static;

lazy_static! {
    pub static ref TEMPLATES: Tera = {
        let mut tera = match Tera::new("templates/**/*.html") {
            Ok(t) => t,
            Err(e) => {
                println!("Parsing error(s): {}", e);
                ::std::process::exit(1);
            }
        };
        tera.autoescape_on(vec![".html", ".sql"]);
        tera
    };
}

pub fn start_blog(listener: TcpListener) -> Result<Server, std::io::Error> {
    let srv = HttpServer::new(move || {
        App::new()
            .app_data(web::Data::new(TEMPLATES.clone()))
            .wrap(middleware::Logger::default())
            .service(Files::new("/static", "static/").use_last_modified(true)) // new line
            .route("/health", web::get().to(HttpResponse::Ok))
            .service(handlers::index) 
    })
    .listen(listener)?
    .run();

    Ok(srv)
}

We just need to build the css so from the root directory, run: cd tailwind && npm run build-css-prod && cd ...

Now, give cargo run a spin, and you should see a much better-looking site.

A quick interlude

Two things are currently missing: 1. a place to store the actual blog posts and 2. a page to show a post in all its glory.

Let's start with the 2nd item on the list since we pretty much already did that. To not bore you with the same thing twice,
go ahead and open this link (link to the repo with the complete code),
copy the content and create a new file under templates called post.html.

Next, go to here copy the content and paste it into your base.css.
Most of this should look familiar to you if you've done any css and just applies some rather simple styles to the post page.

Steal like an artist (or, copy smarter people than yourself)

Everything is mostly done, we just need a way to store our awesome blog posts and a way to retrieve them so we can remove the hard coding
we did earlier. And the way to do this is heavily inspired by another blog post written by a much smarter guy than myself:
fasterthanlime whom, if you haven't already, should check out. This guy does some seriously deep dives and really
knows his craft.

Go ahead and make a new folder called posts at the same level as the static folder. Inside this one, create another folder, give it a
name like my-first-article and cd into it. I write all my stuff using markdown so that is what we're going to do here as well.

Create a file called post.md and copy-paste some lorem ipsum text (or, an article you've prepared). Next, create a file called
post_frontmatter.toml, open it and add the following:

title = 'This is my first article'
file_name = 'my-first-article'
description = 'The first article authored by me'
tags = []
posted = '22/08/2022'
estimated_reading_time = 13
author = 'Morten Vistisen' # feel free to swap with your own name
order = 1

If this looks suspiciously like our struct Frontmatter {....}, then its because it's.

Next order business, add some logic to extract all the frontmatters we might have and display it on our home page.
Let's start with the logic for getting all the frontmatters of our awesome blogs, so open home_handler.rs and add the following:

use std::{fs, io::Error}; // add these imports
use ignore::WalkBuilder; // add these imports

fn find_all_frontmatters() -> Result<Vec<Frontmatter>, std::io::Error> {
    let mut t = ignore::types::TypesBuilder::new();
    t.add_defaults();
    let toml = match t.select("toml").build() {
      Ok(t)=> t,
      Err(e)=> {
        println!("{:}", e);
        return Err(Error::new(std::io::ErrorKind::Other,
        "could not build toml file type matcher"))
      }
    };

    let file_walker = WalkBuilder::new("./posts").types(toml).build();

    let mut frontmatters = Vec::new();
    for frontmatter in file_walker {
        match frontmatter {
          Ok(fm) => {
            if fm.path().is_file() {
                let fm_content = fs::read_to_string(fm.path())?;
              let frontmatter: Frontmatter = toml::from_str(&fm_content)?;

              frontmatters.push(frontmatter);
            }
        }
        Err(e) => {
            println!("{:}", e); // we're just going to print the error for now
            return Err(Error::new(std::io::ErrorKind::NotFound, "could not locate frontmatter"))
        }
      }
    }

    Ok(frontmatters)
}

Since we're storing our posts as part of the binary we need a way to locate all the frontmatter of our posts and for this, we're going to use ignore::WalkBuilder.
This gives us a recursive directory iterator with a large number of configs that we can set, depending on the type of action we want to do.
For this case, we want it to look in the posts directory and look for all files with the .toml extension, and it does it, BLAZINGLY fast.

One thing to note here is that find_all_frontmatters returns a Result since there are actions that can fail, which means we also have to handle some errors.
As you might be able to tell, I haven't spent the most time on errors in this function and basically, just log whatever error comes from ignore and then return
the most similar one from std::io::ErrorKind. This can be done better using libraries like anyerror or thiserror. I encourage you to play around
with this yourself or open a pull request if you've improvements here.

To actually show some dynamic data on the home page, we need to use the above function in our index handler so open up home_handler.rs and make the following adjustments:

#[get("/")]
pub async fn index(templates: web::Data<tera::Tera>) -> impl Responder {
    let mut context = tera::Context::new();

    let mut frontmatters = match find_all_frontmatters() {
        Ok(fm) => fm,
        Err(e) => {
            println!("{:?}", e);
            return HttpResponse::InternalServerError()
                .content_type("text/html")
                .body("<p>Something went wrong!</p>");
        }
    };
    frontmatters.sort_by(|a, b| b.order.cmp(&a.order));

    context.insert("posts", &frontmatters);

    match templates.render("home.html", &context) {
        Ok(s) => HttpResponse::Ok().content_type("text/html").body(s),
        Err(e) => {
            println!("{:?}", e);
            HttpResponse::InternalServerError()
                .content_type("text/html")
                .body("<p>Something went wrong!</p>")
        }
    }
}

Give cargo run a spin and see the results!

The last thing that's needed is to add the handler for the post.html page and extract the markdown of the article,
convert it to html and serve it. Since we already have post.html ready, go ahead and create post_handler.rs under
src/handlers and open it up. We need to add two functions: 1. to extract the frontmatter from a specific post and 2.
to extract the markdown of the post.

use std::{io::Error, fs};

use super::home_handler::Frontmatter;

fn extract_markdown(post_name: &str) -> Result<String, Error> {
    let markdown = match fs::read_to_string(format!("./posts/{}/post.md", post_name)) {
        Ok(markdown) => markdown,
        Err(e) => {
            println!("{:?}", e);
            return Err(e)
        }
    };

    Ok(markdown)
}

fn extract_frontmatter(post_name: &str) -> Result<Frontmatter, Error> {
    let frontmatter_input = match fs::read_to_string(format!("./posts/{}/post_frontmatter.toml", post_name)) {
        Ok(s) => s,
        Err(e) => {
            println!("{:?}", e);
            return Err(e)
        }
    };

    let frontmatter = match toml::from_str(&frontmatter_input) {
        Ok(fm) => fm,
        Err(e) => {
            println!("{:?}", e);
            return Err(Error::new(std::io::ErrorKind::Other, "could not find post frontmatter"))
        }
    };

    Ok(frontmatter)
}

Not much new here, only the use of toml::from_str to deserialize a string into a specific type, in this case: Frontmatter.

Last thing we need is to create the handler and add it as a service to our App in lib.rs, so add the following to post_handler.rs:

use actix_web::{web, get, Responder, HttpResponse};
use pulldown_cmark::{Options, Parser, html};

#[get("/posts/{post_name}")]
pub async fn post(
    tmpl: web::Data<tera::Tera>,
    post_name: web::Path<String>,
) -> impl Responder {
    let mut context = tera::Context::new();
    let options = Options::empty(); // used as part of pulldown_cmark for setting flags to enable extra features - we're not going to use any of those, hence the `empty();`

    let markdown_input = match extract_markdown(&post_name) {
        Ok(s) => s,
        Err(e) => {
            println!("{:?}", e);
            return HttpResponse::NotFound()
                .content_type("text/html")
                .body("<p>Could not find post - sorry!</p>")
        }
    };

    let frontmatter = match extract_frontmatter(&post_name) {
        Ok(s) => s,
        Err(e) => {
            println!("{:?}", e);
            return HttpResponse::NotFound()
                .content_type("text/html")
                .body("<p>Could not find post - sorry!</p>")
        }
    };

    let parser = Parser::new_ext(&markdown_input, options);

    let mut html_output = String::new();
    html::push_html(&mut html_output, parser);

    context.insert("post", &html_output);
    context.insert("meta_data", &frontmatter);

    match tmpl.render("post.html", &context) {
        Ok(s) => HttpResponse::Ok().content_type("text/html").body(s),  
        Err(e) => {
            println!("{:?}", e);
            return HttpResponse::NotFound()
                .content_type("text/html")
                .body("<p>Could not find post - sorry!</p>")
        }
    }
}

Much of this is familiar, the difference here is that we make two variables available to our template: post and meta_data.

Last thing, we need to include the new handler in lib.rs and our handlers/mod.rs. I will leave that up to you!

After adding that, give cargo run a spin again and you should be able to click on the article from the home page, be re-directed to the post page and see your article!

Closing thoughts

I hope you enjoyed the walk-through and are interested in doing more web development with rust, it's in a really good state right now, with an exciting future ahead.
Yes, the beginning might be a little bit frustrating and getting your head wrapped around ownership, borrowing and referencing can take some time.
But I promise you, it's well worth it!

The compiler errors you get can be annoying, but will also guide you towards the correct path, and compared to something like Typescript it's a godsend.
The more you write the easier it becomes.

It has quickly become my go-to language for writing small services as it plays very nice with aws lambdas and due to how fast it is, cold starts are not even an issue anymore.
So, please take what you have here and extend it as much as you want.

Resources

Here is a collection of all the materials I've used to get started with rust. Some of it is free and some of it is paid, but all of them are highly recommendable
so I hope you find some further learning!

How to write integration tests in Go apps

MBV — Wed, 07 Feb 2024 21:09:35 +0000

Picture this: you've just left Node for the promised land of Go. You've learned about composition, interfaces, simplicity, domain-driven design (and understood nothing) and unit tests. You feel all grown-up and ready to take on concurrency and ride off into the promised land. You've built a sweet CRUD application that will revolutionise how we do X.Since this is a sure-fire success; you've of course already contacted a real estate agent to view that penthouse apartment you've always wanted and tweeted at multiple VC funds and investors. You're basically ready to go live but before you release your code into the wild, you whip out your favourite terminal and type in go run main.go. It runs, glory days!

You set up multiple demos and investor meetings. You add "CEO and Founder" to your Linkedin profile (and tinder, of course) and tell your mom that you're going to vacate the basement any day now. You show up to your first meeting, do a couple of power poses and start demoing that really-crucial-endpoint-that-is-the-backbone-of-the-app. With your right thumb on the ENTER key (and your left thumb mentally ready to pop the champagne), you go ahead. Aaaaaand, it completely crashes. What the fuck. Not only do you get laughed at your super important meeting, but you also have to tell the real estate agent that you probably can't afford that penthouse apartment, you have also to stay in your mom's basement (and continue trying to convince your tinder dates that it's definitely a temporary solution).

Hopefully, you haven't experienced the above scenario and are just here because you typed some words into google. I'm going to show you how I set up and run integration tests in Go, using docker and some nifty Makefile cmds.

Aim of the article

A lot has been written on this subject (check out the resources list at the end of the article). This is not an attempt to improve on what is currently available. It's an attempt to show how you can set up and configure integrations tests, both locally and externally (CI env), that is extendable and portable. I'm sure there are ways to improveupon what I'm going to show you here, however, the main goal is to provide you with the tools to build out a great test suite. Once you've understood the concepts and configurations it's much easier to customize them to your need. Furthermore, you get a better idea of what to google (unknown unknowns are a tricky subject), so you can solve yourown issues that might come up later on.

Requirements

Alright, first thing first. You'll need some tools, so please install and setup the following:

Go
Docker

There are more than one way to skin a cat

As with everything in software, there are a lot of ways of doing the same thing and even more opinions about how to do that thing. With integration testing, we might discuss how to set up/configure the test suite, what library (if any) to use and even when something classifies as an integration vs. unit test, E2E test, and so forth. This can make concepts more complex than they need to be especially when you're just starting out. I think one of the best ways to approach such situations is to agree on some sort of guiding principles/first principles, that becomes the basis for what we do. I've found one of the best principles for this is:

test behaviour, not implementation details

To expand upon this, we want to have our tests cover cases/scenarios of how a user might actually use our software. I use the terms user and software very broadly here as it's very context-specific. A user might be a co-worker who calls some method in the codebase that we have created or it might also be someone consuming our API. Our tests shouldn't really worry about how we actually implemented the code, only that given X input we see Y response. We are in a sense testing that the "contract" we have made as developers by exposing our code/functions/methods to the world, is actually kept.

If we want a more concrete definition of an integration test we can dust off the big ol' google book and see how they define it:

...medium-scoped tests (commonly called integration tests) are designed to verify interactions between a small number of components; for example, between a server and a database

Source: Software Engineering at Google, chap. 11

If we borrow some terminology from clean architecture we can think of it as when we include code from the infrastructure layer in our tests, we're in the integration testing territory.

What are we doing

When I started out writing integration tests in Go; what I struggled the most with was how to configure and set it up for real-world usage. This, again, will also differ based upon the developer/use-case/reality as this approach might not work for well a big multinational company. My criteria for my integration tests are that they should run across different operating systems, be easy to run and play nicely with CI/CD workflows (basically, play nice within a dockerized environment).

I'm a strong believer in YAGNI (you're not going to need it), so I'm going to
show you two ways of setting up integration tests:

the vanilla only-using-the-standard-library approach
using a library

This should hopefully illustrate how you can start out relatively simple (we could skip the Docker part, but honestly, that would make it a little trickier to set up our CI/CD flow) and then add on as needed.

What we are testing

I'm going to re-use some of the code from my article on how to structure Go apps (which can be found here. If you haven't read it, it basically builds a small app that lets your track your weight gain during the lockdown. The article is due for an update (I would suggest checking this repo for a great example on how to structure Go apps using DDD and clean architecture), so we're just going to focus on adding integration tests based on the code (a good example of testing behaviour vs. implementation details). We want to make sure that calls to our services behave as expected.

You can find the repo here.

"Infrastructure" Setup

Much of modern web development uses Docker and this tutorial will be no exception. This is not a tutorial on Docker so I won't be going into much detail about the setup, but provide some foundations on how to get started. There are ways to improve upon this setup by extending the docker-compose file, using tags inside of our Dockerfile etc. But I often find that to give me more headaches than gains. We're violating DRY, to some extent, but it does give us the ability to have completely separate dev and test environment. And after reading this, you could yourself try to shorten the docker setup.

Docker test environment setup

FROM golang:1-alpine

RUN apk add --no-cache git gcc musl-dev

RUN go get -u github.com/rakyll/gotest

RUN go install -tags 'postgres' github.com/golang-migrate/migrate/v4/cmd/migrate@latest

WORKDIR /app

And the


 file:


```yaml
version: "3.8"
services:
    database:
        image: postgres:13
        container_name: test_weight_tracker_db_psql
        environment:
            - POSTGRES_PASSWORD=password
            - POSTGRES_USER=admin
            - POSTGRES_DB=test_weight_tracker_database
        ports:
            - "5436:5432" # we are mapping the port 5436 on the local machine
              # to the image running inside the container
        volumes:
            - test-pgdata:/var/lib/postgresql/data

    app:
        container_name: test_weight_tracker_app
        build:
            context: .
            dockerfile: Dockerfile.test
        ports:
            - "8080:8080"
        working_dir: /app
        volumes:
            - ./:/app
            - test-go-modules:/go
        environment:
            - DB_NAME=test_weight_tracker_database
            - DB_PASSWORD=${DB_PASSWORD}
            - DB_HOST=test_weight_tracker_db_psql
            - DB_PORT=${DB_PORT}
            - DB_USERNAME=${DB_USERNAME}
            - ENVIRONMENT=test
        depends_on:
            - database

volumes:
    test-pgdata:
    test-go-modules:

With that in place we can now easily run our integration tests, which can be done with:

make run-integration-tests

Approach 1: Vanilla setup

Side note: if you want to see the code separated from the rest, check out the branch vanilla-approach/running-integration-tests-using-std-library

There is a tendency in the Go community to lean more towards the standard library than to pull in external libraries. And for a good reason, you can do a lot with just the standard library. I'm by no means a purist in this regard and we are also using external libraries for the routing, migration etc. here, but I think it gives a great understanding of what is happening by starting with the standard library and then using other libraries as you go.

In the earlier segment, we got our infrastructure up and running so we have an active database, a running application, or in this case, the possibility to trigger a test run against our app.

To maximize our confidence in our code we want our integration tests to mimic our production environment as much as possible. This means that we need to have some setup and teardown functions that can run migrations, populate the database with seed data and tear everything down after the test so we have a clean environment each time. The last part is important as we want to have a reliable test environment so we don't want previously run tests to affect the current one running.

It's worth noting here that this also means that our integrations tests have to run sequentially, and not in parallel, as we most likely would have our unit test do which means it would take longer running the test suite. I wouldn't worry too much about this at the beginning and simply focus on having some integration tests with decent coverage. Once you start to reach unbearable long integration test runs then it's time to start looking into other setups/improving the current one. For example, we could create a new database for each test and use that, but it would add a more complex setup. Or we could use an SQLite database for our integration tests, but again, cross that bridge once you get to it.

It would be a great exercise to try and change this code to try out different strategies to speed up the test runs.

Migrations

I'm a big fan of the golang-migrate library so that is what we are going to use to write our migrations. In short, it generates up/down migration pairs and treats each migration as a new version, so you can roll back to the last working version if you need to.

I'm not going to touch upon migration strategies here, so our tests will be written with the assumption that we have a database with all the latest migrations. To achieve this in our tests we will run the up version before each test is run. For that we need the following function:

func RunUpMigrations(cfg config.Config) error {
    _, b, _, _ := runtime.Caller(0)
    basePath := filepath.Join(filepath.Dir(b), "../migrations")
    migrationDir := filepath.Join("file://" + basePath)
    db, err := sql.Open("postgres", cfg.GetDatabaseConnString())
    if err != nil {
        return errors.WithStack(err)
    }
    defer db.Close()
    driver, err := postgres.WithInstance(db, &postgres.Config{})
    if err != nil {
        return errors.WithStack(err)
    }
    defer driver.Close()

    m, err := migrate.NewWithDatabaseInstance(migrationDir, "postgres", driver)
    if err != nil {
        return errors.WithStack(err)
    }

    if err := m.Up(); err != nil {
        if errors.Is(err, ErrNoNewMigrations) {
            return errors.WithStack(err)
        }
    }
    m.Close()
    return nil
}

After the test is down, we would like our environment to be clean so we also
need this function:

func RunDownMigrations(cfg config.Config) error {
    _, b, _, _ := runtime.Caller(0)
    basePath := filepath.Join(filepath.Dir(b), "../migrations")
    migrationDir := filepath.Join("file://" + basePath)
    db, err := sql.Open("postgres", cfg.GetDatabaseConnString())
    if err != nil {
        return errors.WithStack(err)
    }
    defer db.Close()
    driver, err := postgres.WithInstance(db, &postgres.Config{})
    if err != nil {
        return errors.WithStack(err)
    }
    defer driver.Close()

    m, err := migrate.NewWithDatabaseInstance(migrationDir, "postgres", driver)
    if err != nil {
        return errors.WithStack(err)
    }

    if err := m.Down(); err != nil {
        return errors.WithStack(err)
    }

    return nil
}

Basically, we get a new connection to the database, create a new migrate instance where we pass a path to the migrations folder, the database we use and the driver. We then run the migrations and close the connection again, pretty straightforward.

Next up, we need some data in our database to run our tests against. I'm a pretty big fan of just keeping it in a SQL file and then having a helper function run the SQL script against the database. To do that we just need a function similar to the two above:

func LoadFixtures(cfg config.Config) error {
    pathToFile := "/app/fixtures.sql"
    q, err := os.ReadFile(pathToFile)
    if err != nil {
        return errors.WithStack(err)
    }

    db, err := sql.Open("postgres", cfg.GetDatabaseConnString())
    if err != nil {
        return errors.WithStack(err)
    }

    _, err = db.Exec(string(q))
    if err != nil {
        return errors.WithStack(err)
    }
    err = db.Close()
    if err != nil {
        return errors.WithStack(err)
    }

    return nil
}

With that setup we are ready to write our first tests.

Our first test

Technically, we could test the code interacting with the database by providing a mocked version of the methods in the database/SQL package. But that doesn't really give us much as it would be tricky to mock a situation where you, for example, miss a variable in your .Scan method or have some syntax issue. Therefore, I tend to write integration tests for all my database functionality. Let's add a test for the CreateUser function. We need the following:

// testing the happy path only - to improve upon these tests, we could consider
// using a table test
func TestIntegration_CreateUser(t *testing.T) {
    // create a NewStorage instance and run migrations
    cfg := config.NewConfig()
    storage := psql.NewStorage()

    err := psql.RunUpMigrations(*cfg)
    if err != nil {
        t.Errorf("test setup failed for: CreateUser, with err: %v", err)
        return
    }

    // run the test
    t.Run("should create a new user", func(t *testing.T) {
        newUser, err := entity.NewUser(
            "Jon Snow", "male", "90", "thewhitewolf@stark.com", 16, 182, 1)
        if err != nil {
            t.Errorf("failed to run CreateUser with error: %v", err)
            return
        }

        // to ensure consistency we could consider adding in a static date
        // i.e. time.Date(insert-fixed-date-here)
        // creationTime := time.Now()
        err = storage.CreateUser(*newUser)
        // assert there is no err
        if err != nil {
            t.Errorf("failed to create new user with err: %v", err)
            return
        }

        // now lets verify that the user is actually created using a
        // separate connection to the DB and pure sql
        db, err := sql.Open("postgres", cfg.GetDatabaseConnString())
        if err != nil {
            t.Errorf("failed to connect to database with err: %v", err)
            return
        }
        queryResult := entity.User{}
        err = db.QueryRow("SELECT id, name, email FROM users WHERE email=$1",
            "thewhitewolf@stark.com").Scan(
            &queryResult.ID, &queryResult.Name, &queryResult.Email,
        )
        if err != nil {
            t.Errorf("this was query err: %v", err)
            return
        }

        if queryResult.Name != newUser.Name {
            t.Error(`failed 'should create a new user' wanted name did not match 
                returned value`)
            return
        }
        if queryResult.Email != newUser.Email {
            t.Error(`failed 'should create a new user' wanted email did not match 
                returned value`)
            return
        }
        if int64(queryResult.ID) != int64(1) {
            t.Error(`failed 'should create a new user' wanted id did not match 
                returned value`)
            return
        }

    })

    // // run some clean up, i.e. clean the database so we have a clean env
    // // when we run the next test
    t.Cleanup(func() {
        err := psql.RunDownMigrations(*cfg)
        if err != nil {
            if errors.Is(err, migrate.ErrNoChange) {
                return
            }
            t.Errorf("test cleanup failed for: CreateUser, with err: %v", err)
        }
    })
}

We start by creating a new instance of config and storage (just like we would in main.go when running the entire application) and then run the up migrations function. If nothing goes wrong, we should have something similar to what we would have in production.

We then use the storage instance that we just set up to create a new user, open a new connection to query for the user we just created and verify that said user is created with the values we expect. After, use the Cleanup function provided by the testing package to call the down migrations. This basically clears the database.

One more thing you might notice is that we have a psql_test.go file. Open it, and you will find the following function:

// TestMain gets run before running any other _test.go files in each package
// here, we use it to make sure we start from a clean slate
func TestMain(m *testing.M) {
    cfg := config.NewConfig()
    // make sure we start from a clean slate
    err := psql.DropEverythingInDatabase(*cfg)
    if err != nil {
        panic(err)
    }

    os.Exit(m.Run())
}

TestMain is a special function that gets called before all other tests in the package it's located in. Here, we're being (justifiable, I would say) paranoid and call a function that drops everything in the database so we are sure we are starting from a clean slate. You can find the function in repository/psql.go if you want to take a closer look.

And that's basically it for running integration tests against our database functions. We could use table-driven tests here and probably should, but this will do for illustration purposes. See here for an explanation of table-driven tests if you don't know them. Next up, let's do a "proper" integration tests and ensure that our endpoints are working as expected!

Testing our endpoints

Now we get into the meat of the stuff. We've arrived at (or closer to, at least) the definition found in the ol' google book. We're testing that multiple parts of our codes work together, mostly in the infrastructure layer, works together in the way we expect. Here, we want to ensure that whenever our API receives a request that is fulfilling the contract we as developers put out, it does what we want. That is, we want to test the happy path. Ideally, we would also want to test the sad path (not sure if that's the word for it, but this is my article, so now it is) but integration tests are more "expensive" so it's a delicate balance. You could choose to mock out database responses and test the sad path in a more unit-test kind of way, or you could add integration tests until the time it takes to run the test suite becomes unbearable. I would probably err on adding 1 integration test to many, and deal with the "cost" when it becomes too big.

Alright, enough rambling. Let's get started. A side note here; I'm using gofiber which is inspired by express, the Node web framework. The way I'm setting up the POST request sort of depends on how gofiber does things. I say sort of because the underlying thing when sending a post request from Go is using Marshaling. I will point it out when we get to it, but just be aware that if you like gorilla or gin you might have to google a bit.

A quick rundown of router setup

We're not going to be spending much time on this, as you can find the code in the repo. Basically, we have this:

type serverConfig interface {
    GetServerReadTimeOut() time.Duration
    GetServerWriteTimeOut() time.Duration
    GetServerPort() int64
}

type Http struct {
    router        *fiber.App
    serverPort    int64
    userHandler   userHandler
    weightHandler weightHandler
}

func NewHttp(
    cfg serverConfig, userHandler userHandler, weightHandler weightHandler) *Http {
    r := fiber.New(fiber.Config{
        ReadTimeout:  cfg.GetServerReadTimeOut(),
        WriteTimeout: cfg.GetServerWriteTimeOut(),
        AppName:      "Weight Tracking App",
    })
    return &Http{
        router:        r,
        serverPort:    cfg.GetServerPort(),
        userHandler:   userHandler,
        weightHandler: weightHandler,
    }
}

We set up an HTTP struct that has some dependencies to get our server up and running with a router. On that struct, we define some server-specific methods. It's pretty straightforward.

Testing our endpoint to create a new user

Our endpoint is pretty simple. There are no middleware and authentication, everybody can just spam our server with requests and create a ton of users. That's not ideal, but also not really what we care about right now. We just want to make sure our API does what it's supposed to do.

func TestIntegration_UserHandler_New(t *testing.T) {
    cfg := config.NewConfig()
    storage := psql.NewStorage()

    err := psql.RunUpMigrations(*cfg)
    if err != nil {
        t.Errorf("test setup failed for: CreateUser, with err: %v", err)
        return
    }

    userService := service.NewUser(storage)
    weightService := service.NewWeight(storage)

    userHandler := http.NewUserHandler(userService)
    weightHandler := http.NewWeightHandler(weightService)

    srv := http.NewHttp(cfg, *userHandler, *weightHandler)

    srv.SetupRoutes()
    r := srv.GetRouter()

    req := http.NewUserRequest{
        Name:          "Test user",
        Sex:           "male",
        WeightGoal:    "80",
        Email:         "test@gmail.com",
        Age:           99,
        Height:        185,
        ActivityLevel: 1,
    }
    var buf bytes.Buffer
    err = json.NewEncoder(&buf).Encode(req)
    if err != nil {
        log.Fatal(err)
    }
    rq, err := h.NewRequest(h.MethodPost, "/api/user", &buf)
    if err != nil {
        t.Error(err)
    }
    rq.Header.Add("Content-Type", "application/json")

    res, err := r.Test(rq, -1)
    if err != nil {
        t.Error(err)
    }

    if res.StatusCode != 200 {
        t.Error(errors.New("create user endpoint did not return 200"))
    }

    // query the database to verify that a user was created based on the request
    // we sent
    newUser, err := storage.GetUserFromEmail(req.Email)
    if err != nil {
        t.Error(err)
    }

    if newUser.Height != req.Height {
        t.Error(errors.New("create user endpoint did not create user with correct details"))
    }

    t.Cleanup(func() {
        err := psql.RunDownMigrations(*cfg)
        if err != nil {
            if errors.Is(err, migrate.ErrNoChange) {
                return
            }
            t.Errorf("test cleanup failed for: CreateUser endpoint, with err: %v", err)
        }
    })
}

Most of this looks similar to what we had in the repository tests. We set up the database, the services and lastly, the server. We create a request, encode it, send it to our endpoint and check the response. An important thing to note here is that we don't really know what is happening under the hood of this beast. We just know that we sent a request with some data and that returns OK and create a user in the database with the expected data. This is also known as black-box testing. We don't care how this is done, we care that the expected behavior occurs.

One thing about the above code is that there is quite some repetitiveness in how we set up the tests and tear them down after each run. It would be nice if we didn't have to copy-paste all of this and take a long hot bath after each test run because we violated DRY. We could do this ourselves of course, or we could use Approach 2 - using test suites with Testify.

Approach 2 - using Testify to run our integration tests

For this, we are going to use the testify package which I have used for quite some time now. The main thing this does for us is save some lines on configuration and ensuring consistency in our tests suites. It's easy enough to
have the entire codebase in your head when it's only this size, but as it grows, having the setup and configuration done in one place makes things so much easier. Let's see how the setup is done for our handler integration tests:

type HttpTestSuite struct {
    suite.Suite
    TestStorage *psql.Storage
    TestDb      *sql.DB
    TestRouter  *fiber.App
    Cfg         *config.Config
}

func (s *HttpTestSuite) SetupSuite() {
    log.SetFlags(log.LstdFlags | log.Lshortfile)
    cfg := config.NewConfig()

    db, err := sql.Open("postgres", cfg.GetDatabaseConnString())
    if err != nil {
        panic(errors.WithStack(err))
    }

    err = db.Ping()
    if err != nil {
        panic(errors.WithStack(err))
    }
    storage := psql.NewStorage()

    userService := service.NewUser(storage)
    weightService := service.NewWeight(storage)

    userHandler := http.NewUserHandler(userService)
    weightHandler := http.NewWeightHandler(weightService)

    srv := http.NewHttp(cfg, *userHandler, *weightHandler)

    srv.SetupRoutes()
    r := srv.GetRouter()

    s.Cfg = cfg
    s.TestDb = db
    s.TestStorage = storage
    s.TestRouter = r
}

We basically take the entire setup step and automate it for each test suite. If we check the documentation for the SetupSuite method we see that it's basically a method that runs before the test in a suite is run. So the whole setup we did with the standard library like here:

func TestIntegration_UserHandler_CreateUser(t *testing.T) {
    cfg := config.NewConfig()
    storage := psql.NewStorage()

    ..... irrelevant code removed

    userService := service.NewUser(storage)
    weightService := service.NewWeight(storage)

    userHandler := http.NewUserHandler(userService)
    weightHandler := http.NewWeightHandler(weightService)

    srv := http.NewHttp(cfg, *userHandler, *weightHandler)

    srv.SetupRoutes()
    r := srv.GetRouter()

    ..... irrelevant code removed

}

is automated for us, nice! Now, we also did have some other requirements, namely that we had a "fresh" environment for each test run. This means that we need to run up/down migrations to ensure our database is clean. This was done in the setup and teardown portion before in each test, but with testify, we can just define beforeTest and afterTest where we can run the same methods as we did before, without having to copy-paste them for each test.

One thing you will notice if you check out the repo is that we have almost the entirely same code in the repository as we do here. Only except for the TestRouter in the struct. I don't really mind the duplication here as my needs
for the endpoints test could change in the future, and keeping my dependencies as few as possible is desirable. So you could, if you wanted, make one large integration test suite. I just prefer to split things up, each to their
own.

In conclusion

Will the above steps prevent you from the disaster we went through at the beginning of the article? Maybe, it depends (every senior developer's favorite reply). But, it will definitely increase the amount of confidence you can have in your code. How many integration tests to have is always a balance since they do take some time longer to run, but there are ways at fixing that, so testing the happy path until things becomes unbearably slow is a good rule of thumb. Cross that bridge when you get to it.

As mentioned in the introduction, this is not an attempt to add something new and revolutionize the way we do integration tests in the Go community. But to give you another perspective and some boilerplate code to get going with your integration testing adventures. Should you want to continue and learn from people way smarter than me (you definitely should), check out the resource section.

Videos

I've started creating tutorials on YouTube. If you're interested, you can find them here:

mbv labs - YouTube

Making videos about Golang, machine learning, indiehacking/bootstrapping. Mostly tutorials, mini-courses and projects.

youtube.com

Resources

Learn go with tests: basically read this entire thing. Chris does an amazing job showing you how to get started with Go and test-driven development. Definitely worth the read.
HTML forms, databases, integration tests: though it's not in Go, but in Rust, Luca does a great job explaining integration testing. Always try to look for what concepts transcend programming languages and what doesn't. It's always beneficial to have a nuanced view.

A practical approach to structuring Golang apps

MBV — Wed, 07 Feb 2024 21:07:04 +0000

2022 update. At the time of writing this update, it's been over a year since I published this article. At the end of this article I promise a part two, which I must admit, will probably never come for a couple of reasons. I no longer use this structure (I still think this article gives people some help to get going, but would like to update it at one point. I suggest you checkout this repo. Secondly, in regards to integration testing, I've just relased a full-length article about it which you can find here.

Go is an amazing language. It’s simple, easy to reason about and gives you much tooling right out of the box. However, when I started with Go I struggled to figure out how to structure my applications in a way that didn’t use “enterprisey” approaches. This is my take on structuring Golang applications that start out simple, but with the flexibility to grow, and what I would have wanted when I started with Go.

Disclaimer

This article is based on the following people (link to their Twitter):

All of the above-mentioned people have articles and tutorials that are amazing on their own, which you should definitely checkout.

If you just want to see the code for this article’s project, check out the Github repository.

Project & tools

It is always frustrating to learn a new language and having to create the same to-do app over and over again. So we are going to make an application that allows people to create an account, where they can track their weight and calculate their nutritional requirements (inspired by weight increase during lockdown). Feel free to add to this application with features you see fit.

Now, I’m assuming some familiarity with Golang and programming in general, as this is a practical example of how to structure a Go application. We will be using the following:

Gin Web Framework
Go version 1.15+
Postgresql
Golang-migrate
Your favorite IDE (could be Goland or VS code — I highly suggest Goland, it’s amazing)

Elements of a robust and scalable structure

Spend some time working in software development and you quickly learn how programs sometimes can be brittle and other times rigid. Therefore, I look for these three things in my application structure:

Testability
Readability
Adaptability

Writing programs in a way that also makes it easy to write tests dramatically affect developer productivity. Along with readability, it not only makes it easier for you to maintain and improve old code. You also make it easier for new people to add to the project as they can feel somewhat safe when changing or adding code as you already have test in place. Lastly, having adaptability means your project can adapt to changing needs and complexity.

Starting out as simple as possible have the advantage of you being able to quickly iterate on new ideas getting to the actual problem solving right from the get-go. Most developers don’t need to think about domain-driven development when starting new projects, and it would probably be a waste of time in most cases (unless, of course, you are working in Big Co. but then why would you be reading an article for beginner Go developers).

The tutorial is split into three parts:

The application structure
User Service implementation
TDD implementation of Weight Service

This should hopefully give you an overview of the overall structure, a step-by-step guide on how to add a service and finally how easy it is to write test and develop with TDD.

The application structure

Alright, enough talk, let’s go going. Create a new folder wherever you like to store your projects called weight-tracker and run the following command inside the folder:

go mod init weight-tracker

The application is going to follow a structure that looks something like this:

weight-tracker
- cmd
  - server
    main.go
- pkg
  - api
    user.go
    weight.go
  - app
    server.go
    handlers.go
    routes.go
  - repository
    storage.go

Go ahead and create the above structure in your designated folder.

Most go projects seem to be following a convention of having a cmd and a pkg directory. The cmd will be the entry point into the program and gives you the flexibility to interact with the program in several ways. he pkg directory will contain everything else:

routes
database interactions
services etc.

We will be working with four packages:

main
api
app
repository

The main package should be self-explanatory if you’ve ever done any golang programming before. All of our services, i.e. user and weight service, are going into the api package along with a definitions file that will contain all of our structs (we will create this later). In the app package, we will have our server, handlers and routes. Lastly, we have repository that will contain all of our code related to database operations.

Open up main.go and add the following:

package main

import (
    "database/sql"
    "fmt"
    "github.com/gin-contrib/cors"
    "github.com/gin-gonic/gin"
    "os"
    "weight-tracker/pkg/api"
    "weight-tracker/pkg/app"
    "weight-tracker/pkg/repository"
)

func main() {
    if err := run(); err != nil {
        fmt.Fprintf(os.Stderr, "this is the startup error: %s\\n", err)
        os.Exit(1)
    }
}

// func run will be responsible for setting up db connections, routers etc
func run() error {
    // I'm used to working with postgres, but feel free to use any db you like. You just have to change the driver
    // I'm not going to cover how to create a database here but create a database
    // and call it something along the lines of "weight tracker"
    connectionString := "postgres://postgres:postgres@localhost/**NAME-OF-YOUR-DATABASE-HERE**?sslmode=disable"

    // setup database connection
    db, err := setupDatabase(connectionString)

    if err != nil {
        return err
    }

    // create storage dependency
    storage := repository.NewStorage(db)

    // create router dependecy
    router := gin.Default()
    router.Use(cors.Default())

    // create user service
    userService := api.NewUserService(storage)

    // create weight service
    weightService := api.NewWeightService(storage)

    server := app.NewServer(router, userService, weightService)

    // start the server
    err = server.Run()

    if err != nil {
        return err
    }

    return nil
}

func setupDatabase(connString string) (*sql.DB, error) {
    // change "postgres" for whatever supported database you want to use
    db, err := sql.Open("postgres", connString)

    if err != nil {
        return nil, err
    }

    // ping the DB to ensure that it is connected
    err = db.Ping()

    if err != nil {
        return nil, err
    }

    return db, nil
}

You should have quite a few errors showing in your IDE now, we are fixing that in a moment. But first off, let me explain what is going on here. In our func main() we call a method named run, that returns an error (or nil, if there is no error). If run should return an error our programs exits and gives us an error message. Setting up our main function in this way allows us to test it and thereby following the element of a robust service structure, testability. This way of setting up the main function comes from Mat Ryer, who talks more about it in his blog post.

The next two topics we need to discuss are Clean Architecture and Dependency Injection. You might have heard about Clean Architecture under a different name, as a few people wrote about it around the same time. The majority of my inspiration comes from Robert C. Martin so this is who I will be referencing.

We are not going to be religiously following Clean Architecture but mainly adopt the idea of The Dependency Rule. Take a look at the below picture:

Source: Robert C. Martin’s blog — Clean Coder Blog

A quote from Robert Martin’s article helps define this rule:

This rule says that source code dependencies can only point inwards. Nothing in an inner circle can know anything at all about something in an outer circle.

In rough terms, the inner layers should not know about the outer layers. This allows us to practically change the database we use. Say we are changing from PostgreSQL to MySQL or exposing our API through gRPC instead of HTTP. You would most likely never do this but it does speak for the adaptability of this concept.

To achieve this, we are going to use dependency injection (DI for short). Basically, DI is the idea that services should receive their dependencies when they are created. It allows us to decouple the creation of a service’s dependencies from the creation of the service itself. This will be helpful when we get to testing our code. If you want to read more about DI, I suggest this article.

I learn best by looking at actual code, so let’s start by adding some of the missing code that will make the code in main.go make more sense. Open up storage.go and add the following:

package repository

import (
    "database/sql"
    _ "github.com/lib/pq"
)

type Storage interface{}

type storage struct {
    db *sql.DB
}

func NewStorage(db *sql.DB) Storage {
    return &storage{
        db: db,
    }
}

This allows us to create a new storage component where and whenever we want, as long as it receives a valid db argument of type *sql.DB.

As you may have noticed, we have a lowercase and an uppercase version of storage, one is a struct and one is an interface. We will define any methods (like CRUD operations) on the lowercase version of storage and define the methods in the uppercase version. By doing so, we can now easily mock out storage for unit testing. Also, we now get some nice suggestions from our IDE when we add methods to the storage interface when we haven’t implemented the methods.

Now, lets set up the base structure of one of our services, user.go. This should give you a feel for how the API package is going to be structuring services. You will have to repeat this for the weight.go service as well. Just copy-paste the content of user.goand change the naming. Open up user.go and add the following:

package api

// UserService contains the methods of the user service
type UserService interface{}

// UserRepository is what lets our service do db operations without knowing anything about the implementation
type UserRepository interface{}

type userService struct {
    storage UserRepository
}

func NewUserService(userRepo UserRepository) UserService {
    return &userService{
        storage: userRepo,
    }
}

Notice that our user service have a repository dependency. We will define repository methods (i.e. database operations) here later on, but the important part is that we only define the methods we need. Since we are not interested in the implementation of these methods, only the behaviour, we can write mock functions that suit a given test case.

This might seem a bit fluffy right now but bear with me for now, it will make sense later on. Let’s get a running application that we can add actual logic to the services.

Open up server.go and add the following:

package app

import (
    "github.com/gin-gonic/gin"
    "log"
    "weight-tracker/pkg/api"
)

type Server struct {
    router        *gin.Engine
    userService   api.UserService
    weightService api.WeightService
}

func NewServer(router *gin.Engine, userService api.UserService, weightService api.WeightService) *Server {
    return &Server{
        router:        router,
        userService:   userService,
        weightService: weightService,
    }
}

func (s *Server) Run() error {
    // run function that initializes the routes
    r := s.Routes()

    // run the server through the router
    err := r.Run()

    if err != nil {
        log.Printf("Server - there was an error calling Run on router: %v", err)
        return err
    }

    return nil
}

Next, open routes.go and add the following:

package app

import "github.com/gin-gonic/gin"

func (s *Server) Routes() *gin.Engine {
    router := s.router

    // group all routes under /v1/api
    v1 := router.Group("/v1/api")
    {
        v1.GET("/status", s.ApiStatus())
    }

    return router
}

We are going to take advantage of gin’s group functionality so we can easily group our endpoints after the resources they intend to serve. Next up, let’s add a handler so we can actually make calls to the status endpoint and verify that our application is running. Open up handlers.go:

package app

import (
    "github.com/gin-gonic/gin"
    "net/http"
)

func (s *Server) ApiStatus() gin.HandlerFunc {
    return func(c *gin.Context) {
        c.Header("Content-Type", "application/json")

        response := map[string]string{
            "status": "success",
            "data":   "weight tracker API running smoothly",
        }

        c.JSON(http.StatusOK, response)
    }
}

At this point, we only need to sync a few dependencies: Gin Web Framework and a driver for PostgreSQL. Go ahead and type the following into your terminal:

  go get github.com/gin-contrib/cors
  go get github.com/gin-gonic/gin
  go get github.com/lib/pg

Everything should be ready now, so go into your terminal and write:

go go run cmd/server/main.go

, visit

sh http://localhost:8080/v1/api/status

, and you should receive a message along the lines of:

{
    "data": "weight tracker API running smoothly",
    "status": "success"
}

If you don’t get the above message, please go back to the previous steps and see if you missed something or checkout the accompanying GitHub repository.

User Service implementation

At this point, we are ready to start building the meat of the application. To do so, we probably need to set up a database with tables and relations. To do this, we are going to use the library [golang-migrate](https://github.com/golang-migrate/migrate/). I personally like this library as it has a CLI tool to add migrations which enables me to do things like adding Makefile commands to create migrations. I encourage you to give the library’s documentation a look, it is an amazing project.

I won’t cover how to set this up as the article would be too long. For now, go into your terminal, make sure you are in the repository folder and run:

git clone https://gist.github.com/ed090d782dc6ebb35e344ff82aafdddf.git

This clones the migrations needed for the project. You should also now have a folder named ed090d782dc6ebb35e344ff82aafdddf, ****lets change that to migrations by running:

mv ed090d782dc6ebb35e344ff82aafdddf migrations

The last thing we need now is to add the RunMigrations method to storage.go:

// update your imports to look like this:
import (
    "database/sql"
    "errors"
    "github.com/golang-migrate/migrate/v4"
    _ "github.com/golang-migrate/migrate/v4/database/postgres"
    _ "github.com/golang-migrate/migrate/v4/source/file"
    _ "github.com/lib/pq"
    "path/filepath"
    "runtime"
)

// add the RunMigrations method to our interface
type Storage interface {
    RunMigrations(connectionString string) error
}

// add this below NewStorage
func (s *storage) RunMigrations(connectionString string) error {
    if connectionString == "" {
        return errors.New("repository: the connString was empty")
    }
    // get base path
    _, b, _, _ := runtime.Caller(0)
    basePath := filepath.Join(filepath.Dir(b), "../..")

    migrationsPath := filepath.Join("file://", basePath, "/pkg/repository/migrations/")

    m, err := migrate.New(migrationsPath, connectionString)

    if err != nil {
        return err
    }

    err = m.Up()

    switch err {
    case errors.New("no change"):
        return nil
    }

    return nil
}

To run the migrations, open up main.go and add the following:

// everything stays the same, so add this below
// storage := repository.NewStorage(db)
// run migrations

// note that we are passing the connectionString again here. This is so
// we can easily run migrations against another database, say a test version,
// for our integration- and end-to-end tests
err = storage.RunMigrations(connectionString)

if err != nil {
    return err
}

You should now have two tables in your database: user and weight. Let’s get started on writing some actual business logic.

We want to let people create an account through our API. So let’s started by defining what a user request is, create a file called under the API folder definitions.go and add the following:

package api

type NewUserRequest struct {
    Name          string `json:"name"`
    Age           int    `json:"age"`
    Height        int    `json:"height"`
    Sex           string `json:"sex"`
    ActivityLevel int    `json:"activity_level"`
    WeightGoal    string `json:"weight_goal"`
    Email         string `json:"email"`
}

We are defining what a new user request should look like. Note that this might be different from what a user struct would look like, we define our struct to only include the data we need. Next up, open user.go and the following to UserService and UserRepository:

// user.go
type UserService interface {
    New(user NewUserRequest) error
}

// User repository is what lets our service do db operations without knowing anything about the implementation
type UserRepository interface {
    CreateUser(NewUserRequest) error
}

Here we define a method on our UserService, called New and a method on the UserRepository called CreateUser. Remember we talked about The Dependency Rule earlier? This is what’s going on with the CreateUser method on UserRepository, our service does not know about the actual implementation of the method, what type of database it is etc. Just that there is a method called CreateUser and takes in a NewUserRequest and returns an error. The benefit of this is twofold: we get some indication from our IDE that a method is missing (open up main.go and check api.NewUserService) and what it needs, and it allows us to easily write unit tests. You should also see an error from NewUserServicein user.go, telling us that we are missing a method. Let’s fixed that, add the following:

// add these imports after the package declaration
import (
    "errors"
    "strings"
)

// add this after NewUserService
func(u * userService) New(user NewUserRequest) error {
    // do some basic validations
    if user.Email == "" {
        return errors.New("user service - email required")
    }

    if user.Name == "" {
        return errors.New("user service - name required")
    }

    if user.WeightGoal == "" {
        return errors.New("user service - weight goal required")
    }

    // do some basic normalisation
    user.Name = strings.ToLower(user.Name)
    user.Email = strings.TrimSpace(user.Email)

    err := u.storage.CreateUser(user)

    if err != nil {
        return err
    }

    return nil
}

We do some basic validations and normalisation, but this method could definitely be improved upon. We still need to add the CreateUser method, so open up storage.go and add the following to the CreateUser method to the Storage interface:

// storage.go
type Storage interface {
    RunMigrations(connectionString string) error
    CreateUser(request api.NewUserRequest) error
}

Notice that this makes the error in main.go go away, but results in a new error on the NewStorage function. We need to implement the method, just like we did with the UserService. Add this below RunMigrations:

// add "log" to imports to your imports look like this:
import (
    "database/sql"
    "errors"
    "github.com/golang-migrate/migrate/v4"
    _ "github.com/golang-migrate/migrate/v4/database/postgres"
    _ "github.com/golang-migrate/migrate/v4/source/file"
    _ "github.com/lib/pq"
    "log"
    "path/filepath"
    "runtime"
    "weight-tracker/pkg/api"
)

// add this below RunMigrations
func (s *storage) CreateUser(request api.NewUserRequest) error {
    newUserStatement := `
        INSERT INTO "user" (name, age, height, sex, activity_level, email, weight_goal)
        VALUES ($1, $2, $3, $4, $5, $6, $7);
        `

    err := s.db.QueryRow(newUserStatement, request.Name, request.Age, request.Height, request.Sex, request.ActivityLevel, request.Email, request.WeightGoal).Err()

    if err != nil {
        log.Printf("this was the error: %v", err.Error())
        return err
    }

    return nil
}

Again, this code could probably do with some improvements, but I’m going to keep it short.

Now, all that is left to do is to expose this through HTTP so that people can actually start using our API and create their accounts. Open handlers.go and add the following:

// update your imports to look like this
import (
    "github.com/gin-gonic/gin"
    "log"
    "net/http"
    "weight-tracker/pkg/api"
)

// add this below APIStatus method
func (s *Server) CreateUser() gin.HandlerFunc {
    return func(c *gin.Context) {
        c.Header("Content-Type", "application/json")

        var newUser api.NewUserRequest

        err := c.ShouldBindJSON(&newUser)

        if err != nil {
            log.Printf("handler error: %v", err)
            c.JSON(http.StatusBadRequest, nil)
            return
        }

        err = s.userService.New(newUser)

        if err != nil {
            log.Printf("service error: %v", err)
            c.JSON(http.StatusInternalServerError, nil)
            return
        }

        response := map[string]string{
            "status": "success",
            "data":   "new user created",
        }

        c.JSON(http.StatusOK, response)
    }
}

We are going to accept a request with a JSON payload and using gin’s ShouldBindJSON method to extract the data. Go ahead and try it out!

It took some time to get here, so let me show you one of the benefits that I keep talking about: testability.

Create a file called user_test.go under the API folder and add the following:

package api_test

import (
    "errors"
    "reflect"
    "testing"
    "weight-tracker/pkg/api"
)

type mockUserRepo struct{}

func (m mockUserRepo) CreateUser(request api.NewUserRequest) error {
    if request.Name == "test user already created" {
        return errors.New("repository - user already exists in database")
    }

    return nil
}

func TestCreateNewUser(t *testing.T) {
    mockRepo := mockUserRepo{}
    mockUserService := api.NewUserService(&mockRepo)

    tests := []struct {
        name    string
        request api.NewUserRequest
        want    error
    }{
        {
            name: "should create a new user successfully",
            request: api.NewUserRequest{
                Name:          "test user",
                WeightGoal:    "maintain",
                Age:           20,
                Height:        180,
                Sex:           "female",
                ActivityLevel: 5,
                Email:         "test_user@gmail.com",
            },
            want: nil,
        }, {
            name: "should return an error because of missing email",
            request: api.NewUserRequest{
                Name:          "test user",
                Age:           20,
                WeightGoal:    "maintain",
                Height:        180,
                Sex:           "female",
                ActivityLevel: 5,
                Email:         "",
            },
            want: errors.New("user service - email required"),
        }, {
            name: "should return an error because of missing name",
            request: api.NewUserRequest{
                Name:          "",
                Age:           20,
                WeightGoal:    "maintain",
                Height:        180,
                Sex:           "female",
                ActivityLevel: 5,
                Email:         "test_user@gmail.com",
            },
            want: errors.New("user service - name required"),
        }, {
            name: "should return error from database because user already exists",
            request: api.NewUserRequest{
                Name:          "test user already created",
                Age:           20,
                Height:        180,
                WeightGoal:    "maintain",
                Sex:           "female",
                ActivityLevel: 5,
                Email:         "test_user@gmail.com",
            },
            want: errors.New("repository - user already exists in database"),
        },
    }

    for _, test := range tests {
        t.Run(test.name, func(t *testing.T) {
            err := mockUserService.New(test.request)

            if !reflect.DeepEqual(err, test.want) {
                t.Errorf("test: %v failed. got: %v, wanted: %v", test.name, err, test.want)
            }
        })
    }
}

A lot of things are happening here so let us go through them step-by-step.

We start by creating a struct called mockUserRepo. Our UserServiceknows that it needs a UserRepository with the following method: CreateUser. However, it does care about the actual implementation of said method, which allows us to mock the behaviour any way we want. In this case, we say that when the Name of the request is equal to “test user already created” (a bad name I know, but hopefully you get the point), return an error, and if not, just return nil. This allows us to mock the behaviour of our database to make it fit different situations, and test if our logic handles it in a way we expect it too.

Next, we create a new variable called mockRepo of type mockUserRepo We then create a mockUserService and pass the mockRepo to it, and we are ready to go! The actual test is what is known as a table-driven test. I won’t go into details about it as it is beyond the scope of this article, but if you want to know more, check out Dave Chaney’s article about it here.

Now, whenever we add a method to the UserRepository we would also have to add it here, in our mockUserRepo. We would of course also like to have some integration test, but what I really wanted to show with this article was how to do unit tests as these are cheap and easy to write.

Run go test ./... from the root directory and all tests should pass.

TDD implementation of Weight Service

Our application is not worth much right now. We can only create a user but not track our weight or calculate the amount of calories we need. Let’s change that!

Our application is not worth much right now. We can only create a user but not track our weight or calculate the number of calories we need. Let’s change that!I’m a big fan of test-driven development (TDD) and the way this application is structured makes it really easy to use. We are going to need three methods on our weight service: New, CalculateBMR and DailyIntake and two on our repository: CreateWeightEntry and GetUser. Open weight.go and add the following to WeightService and WeightRepository:

// weight.go

// WeightService contains the methods of the user service
type WeightService interface {
    New(request NewWeightRequest) error
    CalculateBMR(height, age, weight int, sex string) (int, error)
    DailyIntake(BMR, activityLevel int, weightGoal string) (int, error)
}

type WeightRepository interface {
    CreateWeightEntry(w Weight) error
    GetUser(userID int) (User, error)
}

Next, we need to add three structs to our definitions file: ´NewWeightRequest, Weight and User. Open up weight.go and add the following:

// add this after the package declaration
import "time"

// add this below NewUserRequest
type User struct {
    ID            int       `json:"id"`
    CreatedAt     time.Time `json:"created_at"`
    UpdatedAt     time.Time `json:"updated_at"`
    Name          string    `json:"name"`
    Age           int       `json:"age"`
    Height        int       `json:"height"`
    Sex           string    `json:"sex"`
    ActivityLevel int       `json:"activity_level"`
    WeightGoal    string    `json:"weight_goal"`
    Email         string    `json:"email"`
}

type Weight struct {
    Weight             int `json:"weight"`
    UserID             int `json:"user_id"`
    BMR                int `json:"bmr"`
    DailyCaloricIntake int `json:"daily_caloric_intake"`
}

type NewWeightRequest struct {
    Weight int `json:"weight"`
    UserID int `json:"user_id"`
}

Now, you will see an error on NewWeightService about missing methods. We don’t want to write the actual implementation yet, as we are doing TDD, so for now, just add this below NewWeightService:

// add these below NewWeightService
func (w *weightService) CalculateBMR(height, age, weight int, sex string) (int, error) {
    panic("implement me")
}

func (w *weightService) DailyIntake(BMR, activityLevel int, weightGoal string) (int, error) {
    panic("implement me")
}

func (w *weightService) New(request NewWeightRequest) error {
    panic("implement me")
}

Open up main.go and you will see that we also have some missing methods on storage passed to api.NewWeightService. Open up storage.go and add these:

// add this to the Storage interface
type Storage interface {
    CreateWeightEntry(request api.Weight) error
    GetUser(userID int) (api.User, error)
}

// add this below the CreateUser method
func (s *storage) CreateWeightEntry(request api.Weight) error {
    panic("implement me!")
}

func (s *storage) GetUser(userID int) (api.User, error) {
    panic("implement me!")
}

Lets now add the tests that we will need, create a file calledweight_test.go in the API folder and add the following:

package api_test

import (
    "errors"
    "reflect"
    "testing"
    "weight-tracker/pkg/api"
)

type mockWeightRepo struct{}

func (m mockWeightRepo) CreateWeightEntry(w api.Weight) error {
    return nil
}

func (m mockWeightRepo) GetUser(userID int) (api.User, error) {
    if userID != 1 {
        return api.User{}, errors.New("storage - user doesn't exists")
    }

    return api.User{
        ID:            userID,
        Name:          "Test user",
        Age:           20,
        Height:        185,
        WeightGoal:    "maintain",
        Sex:           "female",
        ActivityLevel: 5,
        Email:         "test@mail.com",
    }, nil
}

func TestCreateWeightEntry(t *testing.T) {
    mockRepo := mockWeightRepo{}
    mockUserService := api.NewWeightService(&mockRepo)

    tests := []struct {
        name    string
        request api.NewWeightRequest
        want    error
    }{
        {
            name: "should create a new user successfully",
            request: api.NewWeightRequest{
                Weight: 70,
                UserID: 1,
            },
            want: nil,
        }, {
            name: "should return a error because user already exists",
            request: api.NewWeightRequest{
                Weight: 70,
                UserID: 2,
            },
            want: errors.New("storage - user doesn't exists"),
        },
    }

    for _, test := range tests {
        t.Run(test.name, func(t *testing.T) {
            err := mockUserService.New(test.request)
            if !reflect.DeepEqual(err, test.want) {
                t.Errorf("test: %v failed. got: %v, wanted: %v", test.name, err, test.want)
            }
        })
    }
}

func TestCalculateBMR(t *testing.T) {
    mockRepo := mockWeightRepo{}
    mockUserService := api.NewWeightService(&mockRepo)

    tests := []struct {
        name   string
        Height int
        Age    int
        Weight int
        Sex    string
        want   int
        err    error
    }{
        {
            name:   "should calculate BMR for a female",
            Height: 170,
            Age:    22,
            Weight: 65,
            Sex:    "female",
            want:   1441,
            err:    nil,
        }, {
            name:   "should calculate BMR for a male",
            Height: 170,
            Age:    22,
            Weight: 65,
            Sex:    "male",
            want:   1607,
            err:    nil,
        }, {
            name:   "should return error because sex wasn't properly specified",
            Height: 170,
            Age:    22,
            Weight: 65,
            Sex:    "",
            want:   0,
            err:    errors.New("invalid variable sex provided to CalculateBMR. needs to be either male or female"),
        },
    }

    for _, test := range tests {
        t.Run(test.name, func(t *testing.T) {
            BMR, err := mockUserService.CalculateBMR(test.Height, test.Age, test.Weight, test.Sex)
            if !reflect.DeepEqual(err, test.err) {
                t.Errorf("test: %v failed. got: %v, wanted: %v", test.name, err, test.want)
            }

            if !reflect.DeepEqual(BMR, test.want) {
                t.Errorf("test: %v failed. got: %v, wanted: %v", test.name, BMR, test.want)
            }
        })
    }
}

func TestDailyIntake(t *testing.T) {
    mockRepo := mockWeightRepo{}
    mockUserService := api.NewWeightService(&mockRepo)

    tests := []struct {
        name          string
        BMR           int
        ActivityLevel int
        weightGoal    string
        want          int
        err           error
    }{
        {
            name:          "should calculate daily intake for activity level 1 with weight loss as goal",
            weightGoal:    "loose",
            BMR:           1441,
            ActivityLevel: 1,
            want:          1229,
            err:           nil,
        }, {
            name:          "should calculate daily intake for activity level 2 with weight loss as goal",
            weightGoal:    "loose",
            BMR:           1441,
            ActivityLevel: 2,
            want:          1481,
            err:           nil,
        }, {
            name:          "should calculate daily intake for activity level 3 with weight loss as goal",
            weightGoal:    "loose",
            BMR:           1441,
            ActivityLevel: 3,
            want:          1733,
            err:           nil,
        }, {
            name:          "should calculate daily intake for activity level 4 with weight increase as goal",
            weightGoal:    "gain",
            BMR:           1441,
            ActivityLevel: 4,
            want:          2985,
            err:           nil,
        }, {
            name:          "should calculate daily intake for activity level 5 with weight maintenance as goal",
            weightGoal:    "maintain",
            BMR:           1441,
            ActivityLevel: 5,
            want:          2737,
            err:           nil,
        },
    }

    for _, test := range tests {
        t.Run(test.name, func(t *testing.T) {
            BMR, err := mockUserService.DailyIntake(test.BMR, test.ActivityLevel, test.weightGoal)
            if !reflect.DeepEqual(err, test.err) {
                t.Errorf("test: %v failed. got: %v, wanted: %v", test.name, err, test.want)
            }

            if !reflect.DeepEqual(BMR, test.want) {
                t.Errorf("test: %v failed. got: %v, wanted: %v", test.name, BMR, test.want)
            }
        })
    }
}

The actual content of the tests is not so important as the fact that we can quickly write tests and mock external dependencies such as the methods to interact with the database.

Run go test ./... from the root directory and you should receive a lot of failed tests. We are going to fix this by implementing the logic of these methods, starting with the three in our WeightService. Add this below NewWeightService:

// add this after the package declaration
import "errors"

// we define some activity multipliers here - these are just part of the
// formula I choose to calculate your daily caloric intake.
const (
    // Very Low Intensity activity multiplier - 1
    veryLowActivity = 1.2
    // Light exercise activity multiplier - 3-4x per week - 2
    lightActivity = 1.375
    // Moderate exercise activity multiplier - 3-5x per week 30-60 mins/session - 3
    moderateActivity = 1.55
    // High exercise activity multiplier - (6-7x per week for 45-60 mins) - 4
    highActivity = 1.725
    // Very high exercise activity multiplier - for people who is an Athlete - 5
    veryHighActivity = 1.9
)

func (w *weightService) New(request NewWeightRequest) error {
    if request.UserID == 0 {
        return errors.New("weight service - user ID cannot be 0")
    }

    user, err := w.storage.GetUser(request.UserID)

    if err != nil {
        return err
    }

    bmr, err := w.CalculateBMR(user.Height, user.Age, request.Weight, user.Sex)

    if err != nil {
        return err
    }

    dailyIntake, err := w.DailyIntake(bmr, user.ActivityLevel, user.WeightGoal)

    if err != nil {
        return err
    }

    newWeight := Weight{
        Weight:             request.Weight,
        UserID:             user.ID,
        BMR:                bmr,
        DailyCaloricIntake: dailyIntake,
    }

    err = w.storage.CreateWeightEntry(newWeight)

    if err != nil {
        return err
    }

    return nil
}

func (w *weightService) CalculateBMR(height, age, weight int, sex string) (int, error) {
    var sexModifier int

    switch sex {
    case "male":
        sexModifier = -5
    case "female":
        sexModifier = 161
    default:
        return 0, errors.New("invalid variable sex provided to CalculateBMR. needs to be either male or female")
    }

    return (10 * weight) + int(float64(height)*6.25) - (5 * age) - sexModifier, nil
}

func (w *weightService) DailyIntake(BMR, activityLevel int, weightGoal string) (int, error) {
    var maintenanceCalories int

    switch activityLevel {
    case 1:
        maintenanceCalories = int(float64(BMR) * veryLowActivity)
    case 2:
        maintenanceCalories = int(float64(BMR) * lightActivity)
    case 3:
        maintenanceCalories = int(float64(BMR) * moderateActivity)
    case 4:
        maintenanceCalories = int(float64(BMR) * highActivity)
    case 5:
        maintenanceCalories = int(float64(BMR) * veryHighActivity)
    default:
        return 0, errors.New("invalid variable activityLevel - needs to be 1, 2, 3, 4 or 5")
    }

    var dailyCaloricIntake int

    switch weightGoal {
    case "gain":
        dailyCaloricIntake = maintenanceCalories + 500
    case "loose":
        dailyCaloricIntake = maintenanceCalories - 500
    case "maintain":
        dailyCaloricIntake = maintenanceCalories
    default:
        return 0, errors.New("invalid weight goal provided - must be gain, loose or maintain")
    }

    return dailyCaloricIntake, nil
}

With these methods added all of the tests should pass.

The last thing we need is to add the methods needed to interact with the database. Open storage.go and add the following:

// replace the methods: CreateWeightEntry and GetUser with the code below
func (s *storage) CreateWeightEntry(request api.Weight) error {
    newWeightStatement := `
        INSERT INTO weight (weight, user_id, bmr, daily_caloric_intake)
        VALUES ($1, $2, $3, $4)
        RETURNING id;
        `

    var ID int
    err := s.db.QueryRow(newWeightStatement, request.Weight, request.UserID, request.BMR, request.DailyCaloricIntake).Scan(&ID)

    if err != nil {
        log.Printf("this was the error: %v", err.Error())
        return err
    }

    return nil
}

func (s *storage) GetUser(userID int) (api.User, error) {
    getUserStatement := `
        SELECT id, name, age, height, sex, activity_level, email, weight_goal FROM "user"
        where id=$1;
        `

    var user api.User
    err := s.db.QueryRow(getUserStatement, userID).Scan(&user.ID, &user.Name, &user.Age, &user.Height, &user.Sex, &user.ActivityLevel, &user.Email, &user.WeightGoal)

    if err != nil {
        log.Printf("this was the error: %v", err.Error())
        return api.User{}, err
    }

    return user, nil
}

We didn’t do this in true TDD style, however, it should again paint a picture of how we can structure Go applications and develop them using TDD. Feel free to re-implement this section and do it in true TDD style: create one test, make it pass, create the next one, and so on.

We have almost everything we need now. The last thing we need is to add routes and a handler to create a weight entry for a given user. I’m going to leave this up to the reader to implement, as the building blocks should be in place. If you are feeling lazy you can just check out the accompanying Github repository.

Conclusion

Hopefully, this tutorial gave you some insights on how to structure your Golang applications. It was a long one I know, but hopefully, you didn’t waste your time. In a part two, I will show how to add Makefile commands, integration tests and easily deploy the application. If you have any questions or criticisms, please do not hesitate to reach out.

My twitter and github.