Forem: Mihail Marian

My case against running containers in tests

Mihail Marian — Thu, 20 Mar 2025 19:27:36 +0000

I've been working with Testcontainers for over 3 years in my day job.
Initially I bought into the idea - being able to run service tests locally, with realistic mocks for your external dependencies. But I've changed my opinion.

The main problem is that they require a fair amount of setup and maintenance. Plenty of code will be written, so you need to figure out the right pattern for scaling it. You'll encounter issues with image downloads (there are some beefy images out there), or container instantiation. Especially if you have machines with multiple OSes, or if you run these tests in your CI/CD. And troubleshooting is a pain - often your only recourse is going through the logs of those containers, which is not straightforward.

Because of the effort involved, we ended up with service tests that only covered the same happy path that our end-to-end test did. WTF testing pyramid, I know, but the reality is that we needed to deliver features - it's hard to justify spending a whole day on the service tests (which happened anyway).

Speaking of reality, I didn't feel like these tests gave me that much reassurance on what would happen in a deployed environment. I doubt the containers for our dependencies could replicate how cloud-native services
actually perform - a DynamoDB image wouldn't have the eventual consistency of the actual service, for example. Then there's the configuration that we added to those cloud resources via Terraform; you can't reflect that in
the service tests either.

Okay, rant over. But what is the alternative? I propose a combination of two things.

Numero uno - in your unit test suite, make sure you have tests that run through your entire application, as your service test would've done, with only the external dependency touchpoints being mocked.

Numero dos - create one/multiple playground environment where developers can quickly deploy their branches.

I probably missed out some edge cases where this wouldn't work. But I hope my ramblings managed to convince you, my dear reader, to think carefully before deciding to add container-based service tests.

The filesystem - a quick & dirty way to store temp data in your API

Mihail Marian — Fri, 14 Mar 2025 19:20:18 +0000

I have an API service for a Google Sheets add-on that I built, and I wanted
to temporarily store some request metadata that I started collecting in the client.

How the add-on works. The API generates the forecast data.

The service was built with Django Ninja, and it's not connected to a database. I thought about setting one up for this feature. But I couldn't be bothered dealing with the configuration and the migrations. I wasn't sure how long I'd be collecting this data for.

I then considered dumping the metadata in the logs. The problem with that approach is that it's a hassle to aggregate all the relevant messages. I was planning to move the information to another place to do some analysis.

I briefly reflected on a global variable as well, but I was worried about the memory management.

And then, at last, I got to the most robust solution in the galaxy - the filesystem. I would store each piece of request metadata in a separate file, so as to avoid issues with concurrent writes. And, I would expose
an endpoint in the API that combines all those files to show me all of the data.

Voilà the code:

def request():
  # at the end, before I return the response
  entry = {
          "timestamp": datetime.now().isoformat(),
          "request_id": request_id,
          "metadata": metadata,
  }
  with open(f"{REQUESTS_DIR}/{request_id}.json", "w") as f:
      json.dump(entry, f)

  return response

# decorator to authenticate with API key header
def get_logs():
  requests = []
    for filename in glob.glob(f"{REQUESTS_DIR}/*.json"):
        with open(filename, "r") as f:
            try:
                data = json.load(f)
                # Add filename as id for reference
                data["id"] = os.path.basename(filename).replace(".json", "")
                requests.append(data)
            except json.JSONDecodeError:
                pass

    # Sort by timestamp
    requests.sort(key=lambda x: x["timestamp"])
    return requests

With that deployed, I can now fetch all of the data in my API's OpenAPI docs. Django Ninja generates this page automatically for me. Lovely jubbly!

The OpenAPI endpoints

The OpenAPI response for the endpoint I just created

I started this blog because I wanted to have conversations on topics that I find interesting. So, feel welcome to reach out to me, even if to challenge my perspective. I'm on LinkedIn; please add a message when you connect.

How I switched to a software engineering career, for free

Mihail Marian — Fri, 21 Feb 2025 13:28:48 +0000

5 years ago, I made the decision to become a software engineer. It wasn't an obvious choice. I had been flailing around until that point,
exploring different careers - sales, financial analyst, financial advisor, bookkeeper, and quantity surveyor.

Odd bunch, eh? Well, I couldn't stick to any of them.

What I did consistently do, throughout all that time, was write code. I created little scripts in Python to automate personal tasks. I built websites in Django to test out startup ideas. I read the Python Crash Course and Automating the Boring Stuff with Python. It felt fun!

The media was hyping up the software engineering career path at the time, too. It was the start of the unprecedented spike in hiring. So, I figured I should give it a go.

Job postings in the US, 2020-2021. Will only go up, right? :)

I started off with freeCodeCamp, which enabled me to grow more confident in my programming skills.
But I soon realised that there were too many training wheels. I didn't want to solve JavaScript challenges inside their IDE; I wanted to launch a website that I can open from anywhere, ASAP!

I reached out for help to a friend of mine, who was also re-training as a software developer. He recommended me The Odin Project.

I liked that website. It was less of a course; more of a rough guide on where to look for what in order to publish a React site. I got turned off when I reached the backend (phrasing!) though. They covered Express, which made perfect sense in the curriculum, but not for me. As you probably guessed, I wanted to use my good ol' pal, Django!

So, after plenty of Googling, I managed to publish my first proper web app - a React/Django beauty that calculated the positivity of any news site it was given.

Sorry for the small image size, this is the only demo of the app I still have

In the meantime, I had also started networking for job opportunities. I reached out to a few people I knew who were hiring developers;
none bit the bullet on me. Then, I posted in a local Facebook group asking if any developers are open to grabbing a drink. A couple replied. By luck, one of them was working at a tiny startup which was hiring. He introduced me to the CTO, who ran a very basic FizzBuzz test by me. I passed it, and got offered the job!

The pay was very poor, even by UK standards. But experience is king, so I took it without any hesitation.

Now that I wrote this post, I have to admit I feel proud of what I was able to accomplish. Within 4 months, I went from a hobbyist programmer to a software engineering professional, without paying a single dime. This means something to me, as someone who tends to put himself down too often.