Forem: Douglas Parsons

Outside of dev.to, what's your favourite online community?

Douglas Parsons — Thu, 20 May 2021 15:42:56 +0000

I've recently discovered IndieHackers, but it seems there are a million and one fantastic communities out there. What are some of your favourites?

Tolerances: How Formula 1 pistons can teach us to be better developers

Douglas Parsons — Mon, 18 Jan 2021 10:37:00 +0000

Recently, while procrastinating from work, I found myself watching a youtube video on Formula 1 pistons.

This video seeks to answer the question: why does a Formula 1 car piston cost around 1,000 times the amount of a typical road car (£50,000 vs £50), despite appearing very similar and serving the same purpose?

Left: piston from an American muscle car. Right: piston from a formula 1 car.

So, what does this teach us about programming, and how to be a better developer?

The cost of immaculate tolerances 💰

Manufacturing a car piston is a game of playing with tolerances.

The piston itself has to fit into a cylinder block snugly, with a small gap around the sides to allow it to move freely. If the gap between a piston and the cylinder is too big, then burning fuel can escape, and the piston is far less efficient. If the gap is too small, then the piston can seize up, blocking the engine completely.

The process a car piston goes through in a single cycle

This adds some serious complexity for building these parts. Not only do they have to be designed to an incredible specification, but the margin for error when manufacturing these parts is so small that they require specialised machinery and special processes to make sure they are fit for purpose.

Rather than immaculate tolerances, what if we wanted to be able to guarantee the quality of code we write as programmers? Imagine if a single bug in your code would be a catastrophic failure or even a risk to life. Would that change the way you write code? Would it change the way you review and test code? And how do you think that would affect the speed at which you develop?

No doubt development would be much slower. Just like the creation of pistons for Formula 1, the process would have to become far more thorough. You would need specific tools and processes to guarantee the right results. You would have to change the way you approach writing software to ensure it behaved exactly as you wanted in every possible situation. And, no doubt, the overall cost of developing software would become much greater as a result of these changes.

When we write software, we have to bear this situation in mind. It’s easy to claim that everything should be of the highest possible quality, but there is a cost for achieving that. Instead, the bar for quality needs to appropriate for the task at hand. Set your tolerances too small, and development becomes slow and expensive. Set them too low, and you end up with inefficient results.

Regular oiling and refactoring 🛢️

In a typical road car, pistons require a bit of help to slide up and down inside their cylinder block: metal doesn’t slide particularly well over other metal, so a layer of oil makes the whole process much easier. In new cars, this oil is a very thin, watery liquid thanks to a solid manufacturing process of car engines. In classic cars, oil can be thick, black gunge - the difference being the gap between the piston and the cylinder block is much larger, so the oil needs to be much thicker to work across the gap. As Formula 1 cars have such minuscule tolerances, their pistons require no oil at all.

Now, imagine oiling a car as being like refactoring code.

What does that mean? Take an old banger of a codebase, and you’ll find yourself having to change the oil regularly. Changing the oil is a messy process that really involves rolling your sleeves up and diving in. It can take time to do, and by the time you’ve managed to do a big refactor, you’re sure everything will be running much smoother as a result, but you find you’re worn out and covered in bits of greasy old oil.

What about a modern codebase? The oil here is much thinner: development practices have got a lot better and technology has really come a long way. As a result, we don’t have to spend nearly as much time under the bonnet. This is still an important part of our work, though. Much as a car needs a regular oil change, codebases need a regular touch up to keep them running. Don’t keep on top of this maintenance, and you never know when it’ll catch up to you with a bang.

Finally, imagine our immaculate codebase from earlier, where quality and lack of bugs trumps all. This is the Formula 1 car of the software world. There’s no need for oil, so there’s no need for oil changes. If all the code here is perfect the first time around, why would we ever need to refactor?

Closing thoughts 💭

If we think of the two practises of managing tolerances or quality, and refactoring code as linked, then we draw some nice conclusions.

Focussing on tolerances and quality the first time around can lead to a huge increase in upfront cost, both in terms of time and effort. This can net you some neat benefits though. Not only can you guarantee the codebase as bug-free and brilliant, but you end up with code that needs no further work. There’s little to no maintenance cost, and you’ll never find yourself refactoring.

Conversely, a focus on the low-cost, fast-paced development necessitates a lower tolerance for quality. This can be useful for getting a project into the world, but comes with a cost later down the line: you need regular oil changes and refactoring to keep everything neat, or you risk blowing up with a bang.

In reality, most codebases lie between these two extremes. Rather than focussing solely on one extreme or the other, we need to be mindful of a balance. Insisting on quality over all else is a tradeoff, and at times isn’t the right one to make. Equally, we need to be careful of the costs down the line in cutting upfront quality. Next time you review a pull-request or submit a new feature, think of this tradeoff: do we mind oiling this a bit later, or do we want an immaculate Formula 1 piston right now?

3 popular technologies that I will strive to never use again

Douglas Parsons — Fri, 08 Jan 2021 14:54:00 +0000

There are a lot of different technologies out there. More than anyone could possibly explore. The de-facto position is to look for technologies that are popular, following the logic that “if enough people like it, it must be good”.

In my experience, this isn’t always the case though.

Throughout my career, I’ve worked across a wide range of technologies. While I’ve grown to love some of them, at times I felt like I was fighting a losing battle and spending more time scrapping with technology rather than solving real-world problems. Whenever this is the case, I make a mental note to avoid that technology in future.

Here are the three I found the most painful to use.

1) Django ⏰

Django is a hugely popular framework for developing websites using a model-template-view architectural pattern. It’s amazing for getting something done very quickly. It also comes with a fantastically powerful Object Relational Mapper (ORM). So what’s the problem?

Its opinions are too strong!

If you’re working just with a relational database, and only want model-template-view, and you’re completely fine with containerising and managing your own scaling, then it may be completely fine. When I used it we were updating our architecture though. Our entire workload wasn’t suited to a relational database and we wanted to take more benefit from auto-scaling, lambdas and managed services in AWS. We were also looking to use React to build more complete experiences for our UI. This became a real break-away from the Django core use-cases. As time went on, we began fighting Django’s opinions more and more. Every change became a battle, digging through the documentation and hacking in workarounds. Over time the codebase became scrappier and scrappier.

Django didn’t feel like the right tool for us. It felt messy and difficult to avoid. If used in its particular setting, it’s probably fine, but it’s not for me.

2) Scala 🐌

Scala is designed to avoid the pitfalls of Java. It runs on the JVM, allows interop with existing Java code, yet provides a much stronger type system and enables functional paradigms.

Despite being in many ways an obvious improvement over Java, my experience with the language was not great. There were two main pitfalls I encountered: quirky behaviour and tooling.

As Scala derives from Java but mixes in functional paradigms, the language contains a huge amount. While being a strength in some ways, parts of Scala have some bizarre hangovers. We’ve all mocked JavaScript for some of its bizarre and unexpected behaviour in code that looks correct. Scala often takes this to another level. It’s easy to slip up when writing code, and unless you know the language incredibly well, need to constantly be on the lookout for common pitfalls (or set up a linter for these).

The second issue I faced with the language was really poor tooling:

Getting support in vim (my editor of choice) was a nightmare,
Automatically formatting code was far more difficult than I expected,
Finding a lightweight logger that didn’t require a thousand line XML file was impossible,
Setting up linting rules for all the common language quirks was an exercise in finding unmaintained repositories and configuring XML,
And, finally, once everything finally worked together, I was stuck battling slow compile times and integration tests that were slower than intercontinental drift.

For a popular language, I was really surprised. There were bits I loved, but overall the experience was akin to pulling teeth. Not one I’d go in for again!

3) Segment 💰

Last year, we stumbled across Segment. A fantastic looking tool that drastically reduces the amount of tracking code you need to write. It allows seamless integration into a plethora of different services and seemed to match up precisely with what we wanted. Pricing wise there’s a generous free tier (1,000 Monthly Active Users), and competitive pricing outside of this, or so we thought... Reading around on the internet, it was generally well-received, so we thought we’d give it a try.

Plugging it into our website was a straightforward experience, and it appeared to deliver on all its promises. We soon had all our events being fired into Segment and tracked across a range of different tools. Brilliant. We put it live.

The next day, we had a sudden spike in traffic caused by a third-party accidentally scraping us repeatedly. Oops. While the third-party resolved the issue on their site, I logged back in to find out how our Monthly Active User (MAU) allowance had been affected.

It had been completely demolished

Not only had we overshot our MAU limit in a matter of hours, but we’d racked up a substantial bill. It turns out every single anonymous visit to our website was being counted as an additional MAU. Pricing per MAU meant overshooting our limit was expensive. We had to pull the ripcord and tear it out immediately.

One expensive bill later, that’s a mistake I don’t want to make again.

Closing thoughts 💭

These three technologies have scarred me as a developer. Having had to fight through difficult situations with each one of them, I know I’ll strive to avoid them in the future. In my experience, Django, Scala and Segment just aren't worth the pain.

Do you have any technologies you feel the same about? Feel like I’ve missed some out? If so, what are they?

Think I’m wrong about any of these? Get in touch and let me know.

Creating challenges: three ways to improve the technical assessment

Douglas Parsons — Wed, 30 Dec 2020 18:45:00 +0000

Through December, I spent a large amount of time doing two things: working my way through the 2020 Advent Of Code; and assessing applications for a job opening we had at Shamaazi. The two initially sound unrelated, but part of our hiring process involves sending candidates an engineering assignment. Similarly, some people use Advent Of Code as interview preparation.

It’s safe to say I spent most of December looking at or thinking about programming challenges 👀

I’ve been reflecting on ‘what makes a good engineering assessment’, and I’ve come to the conclusion that a good assessment – as part of recruiting – should pay careful attention to three main things.

1) Keep it simple, stupid (KISS) 😗

If you’re applying for a new programming job, changing career or trying to get your foot onto the ladder, then the chances are you’re applying to a few different companies. Even if you’re just applying to one though, it takes time. Time to write a CV, time to write a cover letter, research the company, send an application, prep for an interview. Engineering assignments are just one more thing that can eat away at your time.

Time is a limited resource.

Expecting candidates to spend hours decrypting sparse instructions, or spend hours going ‘above and beyond’ in implementing a solution just further reduces any free time they would have. Having lengthy tech-assessments, or expecting candidates to devote their lives to you just creates a false barrier to entry. Some of the best developers I know have lives, kids, families or hobbies outside of their jobs. When recruiting, you should look for well-rounded individuals, not just those who live and breathe software development.

So instead, let’s keep Tech Assessments short. Not simple, but short – short enough that they can be done in a small amount of time. Short so the instructions are easy to wrap your head around. Short so you don’t feel like you have to work all weekend just for a chance at an interview. Because let’s face it, nobody wants that.

2) Make sure it’s explicitly clear what you’re looking for 👀

Recently, I had a friend apply to a new job. He’s a fantastic engineer, works incredibly hard, comes up with brilliant ideas and really contributes towards creating a fun-loving, happy culture. In my eyes, he’s exactly the sort of developer you would want to work with.

As part of applying to a new job, he was asked to complete a short technical assessment. Nothing out of the ordinary here – he completed it pretty quickly and sent it over to me to check through. Together we checked through it and came away happy. It was a really neatly organised, clean and efficient solution to the problem he was given. So he submitted it.

A week went by. No response.

Another week went by. No response.

By this point, he began to get agitated. Maybe they had missed his email, maybe they had forgotten to look at his solution? Maybe the person intended to review it was on leave?

So he reached out to them, expressing his concerns, wondering what had happened. Why had he got no reply? They emailed him back a simple, short answer:

”your submission, while solving the given solution, did not go above and beyond and as a result, was not what we were looking for”

He told me this, and I was immediately outraged. How could they be so stupid to pass up on a talented engineer? Why would they want their engineers to show terrible self-control, to be constantly going off-piste? After some reflection, I realised the problem wasn’t just that they were looking for the wrong characteristics in their developers, but that they had never made it clear in their assignment what they wanted you to do. If they didn’t really care about the solution to the problem, but what you did above and beyond that, then why didn’t they say so?

So instead, let’s keep it explicitly clear what qualities we wish people to display, what areas they should focus on, and make sure it’s crystal clear what they should do. Communication, after all, is a key skill.

3) Make sure the challenge is representative 🏁

It’s very easy to forget when you’re screening candidates and reviewing tech assessments, but every single part of the recruitment process communicates what it’s like to work at a company. Job descriptions give a clear indication of the role, invitation emails should give a bit of a sense of the people who work at a company. Equally, a technical assessment should give an indication of the type of work you will be doing.

If an assignment is all about React hooks, then that’s a strong indication I’ll both need to learn React, and that I’d be writing a tonne of React at that job.

Sounds simple, right?

But people seem to forget this. Instead, they focus on obscure algorithms, copy-pasted katas, or completing contrived challenges –like Advent of Code, fun as it is.

So instead, let’s make technical assessments relevant. Rather than reaching for an obscure problem, what’s the most recent, complicated piece of code you had to write, or edit, or maintain? Can that be explained in a short brief? If so, let’s use that instead.

Closing thoughts 💭

Technical challenges are an incredibly useful tool to use as part of the hiring process. They can often give a much better indication of a candidate than any amount of phone screening, whiteboard-quizzing, or CV scrutinising can. But we have to be careful with them.

Through creating representative challenges, keeping technical assessments lightweight and mindful of time, and by making it explicitly clear what qualities a good candidate would show, we can create technical assessments that don’t feel irrelevant, don’t feel like a time-drain, and are much more useful to both the assessor and the assessee.

Want to know more of my thoughts on recruitment? Read about the qualities I believe developers should show

Do you want to know more about the problems people have in technical assessments? Would you like to know how to avoid these pitfalls when completing a technical assessment? Please get in touch and let me know.

Is Next JS really that good?

Douglas Parsons — Thu, 24 Dec 2020 11:21:20 +0000

Throughout 2020 I've written a tonne of front-ends in both React (using Create-React-App) and Vue. I've also dabbled into Preact and Svelte.

Currently, next.js seems to be all the rage, but I'm struggling to see what makes it so compelling over plain, old React. Is server-side rendering really that good? To me, it seems like it causes more trouble that it solves.

Is next.js really that good? Is it worth making a switch?

Testing is important: three ways to easily improve test quality

Douglas Parsons — Fri, 18 Dec 2020 14:39:00 +0000

It’s no secret that testing is important. We rely on tests to describe intended behaviour, catch any subtle bugs and prevent regressions in our code. But why are tests always such a pain to write well? In mature codebases tests quickly become convoluted and, in my experience, testing is one of the most challenging aspects of software engineering.

This is because we don’t hold our tests – unit tests, integration tests, end-to-end tests or smoke-tests – to the same standards production code. Poor testing can make a codebase even more difficult to maintain than having no tests at all. Despite this, good testing practice flies under the radar and is easily neglected.

Let's challenge this and look at three qualities we expect of good production code, and apply this same thinking to test code – where such quality control is often absent.

1) Don’t Repeat Yourself (DRY) 🔁

People are obsessed with DRY when it comes to production code, often taking it too far. This same anti-repeating is rarely applied to tests. Instead, testing becomes a haven for duplication, with information copied all-over the place. This is most prevalent in two forms.

Asserting – Often there are a tonne of very similar tests, copy pasted with minor tweaks. In reality, they often cover the same test case, with the rationale that it’s “making extra sure”.
Setup – Some tests require laborious setup. Creating mock users, seeding test-data and making sure any dependencies are stubbed out. This setup often gets duplicated between tests or test-suites, with only minor tweaks.

Duplicating assertions and setup both have the same knock-on impact. Bug-fixes, feature tweaks or refactoring quickly becomes a headache. Instead of being able to make a simple modification, a change becomes a game of whack-a-mole, wading through duplicated logic with seemingly unrelated tests starting to fail. You then notice some mocks are wrong, some tests don’t even work. We end up feeling like we need a sledgehammer rather than a scalpel.

Dave Cheney published a brilliant micro-blog on this very topic - you should definitely check it out. It summarises the mentality behind most duplication far better than I can.

2) Scrutinize tests the same as any other code 🔍

I recently wrote a post on one of the larger projects I’ve worked on during my career. This project, despite having some talented engineers working on it, was a complete mess. In particular, let's talk about code reviews and tests.

We all worked in the same physical office, so pull-requests were usually reviewed face-to-face.

This was great and worked really well – it was much easier to have open discussions, loop in people who should be involved, or to get answers to questions. I once overheard a discussion over a pull-request between two experienced developers. Their conversation bounced around discussing sensible topics – the high-level approach to solving a problem, justifying the design and making sure it was efficient. They then delved into the low-level, technical details – making suggestions for improving variables names, neatening up some abstractions, adhering to best practices and agreed standards.

Then it came to ~~reviewing~~ the tests.

"Yeah, it has tests" said one engineer to the other. "Do they pass?", the second questioned. "Yes", replied the first. "That's good", confirmed the second, as both engineers sat nodding to each other as they absent-mindedly scrolled through several hundred lines of tests.

Let's look at the real problem here: the measure of quality had nothing to do with the tests, beyond them simply existing and passing. There was no discussion around edge cases. Were they testing the right things? Was the generation of the test data suitable? Did they take the right approach to mocking? Did the tests even accurately describe what they're doing.

It came to no surprise to anyone, certainly myself, that the majority of tests on the project were useless. Needlessly so, too. By asking simple questions and caring enough to review the tests properly, they could have saved hours of work later down the line, for the sake of five minutes now.

3) Avoid mocking integrations in integration tests 🔗

It sounds obvious when it’s written out like that, right? But you’d be amazed how often this happens. Consider that we are writing a function responsible for adding new users to a mailing list for a product. A test for this might look like the following:

describe("mailing list list", () => { 
  beforeEach(() => { 
    jest.spyOn(emailStorage, "save") 
    jest.spyOn(emailStorage, "rollback") 
  }) 

  it("should add an email to a mailing list", async () => {
    const email = mockEmail() 

    const response = await mailingList.addEmail(email) 

    expect(response).toEqual({ 
      email: email, 
      subscribed: true 
    }) 
    expect(emailStorage.save).toHaveBeenCalledTimes(1)
    expect(emailStorage.rollback).toNotHaveBeenCalled() 
  })
})

This looks fairly typical, right? Although you could say that’s a lot of mocking for one test. It prompts the question:

“What are we actually testing here?”

Are we unit testing the logic the function contains? Or are we testing that it integrates properly with the email storage?

If it's a unit test, you’d argue to mock as much as you can so you are just testing the logic. We seem to be asserting on the mocks a lot though, which wouldn’t be the case if we weren’t also testing the integration.

In this case, how useful really is this test? It’s attempting to test an integration by integrating with a mock. This test looks a lot like it’s not really testing any behaviour at all - it’s just checking that the code does what the code does, at the same level of abstraction.

Say for example, that the email storage didn’t behave the way we expected it to. Would this test fail? Should this test fail? If we rewrote the test to use the real email storage, and then tested it worked in reality, would this be more valuable?

Closing Remarks 💬

Tests are just more code. More code that you should treat with the same level of respect as any other code. Write them well, and they can be a powerful asset that help you safely refactor and add new features. Write them poorly, and they will quickly become a burden. Every change you make becomes wading through mud, sledgehammer in hand. We must think carefully how to write our tests, and be as meticulous with testing as we are with the rest of our code. Unless we do this, tests are a nightmare.

Three unusual qualities we look for to hire amazing developers

Douglas Parsons — Fri, 11 Dec 2020 12:05:00 +0000

Here at Shamaazi, we’ve recently gone through a round of hiring… and wow, hiring is hard!

It’s challenging finding engineers who have the right qualities, attitude and personality to join a team. We’re a tiny company, with just 2 engineers, so any new teammates have a massive impact on our products and culture.

We’re super careful about who we hire because of this.

We don’t expect or search for rockstar developers who can recite algorithms and live and breath software. We don’t expect the world’s most qualified experts in the technology we use. We don’t expect people to spend every moment of their spare time programming.

The qualities we do look for are probably considered unusual compared to most software engineering outfits. Let's take a look at the three we use to ensure a good fit for working at Shamaazi.

Eagle-eyed attention to detail 🔍 #

We give our potential candidates a short engineering assignment as part of our hiring process. We explicitly request that they don't spend more than an hour on it, but it still gives us a vast amount of information that just simply isn't evident from their CV.

Reviewing these assignments has been eye-opening. Most solutions lack attention to detail in at least one area. We begin by asking relatively simple questions of the submissions:

Correctness – does the solution work for all inputs?
Validation – are the inputs validated, what happens if we try to break the system?
Testing – How do we know their solution is correct? Is there any proof that it reliably works?
Quality – How easy is their solution to understand, maintain or extend?

Most technical tests we’ve seen fail on at least two or three of these, if not all four. Quite simply, they lack attention to detail in both their quality and behaviour.

This lack of attention to detail also applies to CVs. We’ve seen poor grammar, typos and even sentences that don’t make sense! Quite frankly, if they can’t complete a short assignment with rigour, or can’t write a CV without errors, can we really trust them with a product that could shape the future of our company?

The ability to express clearly 🗣️ #

Writing good code, a good CV, or writing prose all have a lot in common. They are all forms of communication.

Writing good code is about communicating a set of requirements both to the computer, and to a future maintainer, CVs communicate your skills and knowledge to a potential employer, and prose is a form of communicating ideas by writing them down.

As a result, the ability to express yourself clearly is super important. Engineering teams don’t work in isolation, and working remotely causes a whole new set of challenges for communication. In order to stay well-aligned, agile, and have the ability to discuss complicated topics, we communicate constantly as a team. This is a fundamental aspect of a team being high-performing.

So, when reviewing engineers, we have to consider a few key questions: Does their code communicate its purpose clearly? Is it well structured? Well written? Does it clearly separate concerns? Does it have sensible naming? Does their CV communicate their strengths and experience eloquently and concisely? When prompted in an interview, can a candidate clearly express their thought-process? Can they articulate complicated ideas?

If the answer to all of these is ‘yes’, then we know we have a candidate who is going to be a delight to work with.

A positive outlook 😄 #

This one sounds strange, but I was first introduced to the idea by Patrick Collison, the Founder and CEO of Stripe. In his excellent talk ‘How to Scale’, Patrick discusses in great depth the difficulties in building the culture he wanted.

At an ideal company, we would go to work and everyone would be happy. People would love their jobs and be enthusiastic and supportive. The question then becomes: ‘how do we create a company where this is the culture? Where the atmosphere is happy’. This is an incredible challenge, one that has no easy solution.

As a side note, if anyone knows how to build a company that can turn unhappy people into happy ones, please let me know (I’d like to send my parents there)!

Thankfully, Patrick Collison also shared his ‘cheat’ to achieve this. Simply hire people who are happy, positive people already. This is the culture we want to build, so we’re taking the same shortcut he did at Stripe. By hiring happy people, we can ensure we maintain a happy culture.

Closing Thoughts #

As a tiny company, we have to be incredibly careful about the people we hire. They can have a massive impact on our products, for better or for worse, and contribute heavily towards our culture. Because of this, we deliberately look for these three qualities as signs of excellence in people.

If we can find people to hire who show three key qualities: a happy outlook, the ability to express themselves clearly, and a keen eye for details, then we know they will be a brilliant hire.

Would you like to hear more about our hiring process? Do you want to know more about the problems people have in our technical assessment, or the common errors on CVs? Get in touch and let me know.

Enjoyed this post? Want to share your thoughts on the matter? Found this article helpful? Disagree with me? Let me know by messaging me on Twitter.

Don't Write Utils: how to become an amazing programmer by naming carefully

Douglas Parsons — Tue, 01 Dec 2020 15:08:00 +0000

I hate the word ‘utility’. It says absolutely nothing. And yet, I come across it in so often when programming. It can take different guises too: ‘common’, ‘shared’, ‘lib’, ‘pkg’, ‘tools’, etc. These names can appear both as filenames or as directory names and drive me insane when they do.

Why?

They can contain literally anything.

And yet any code that ends up in these places has a clear purpose. It must be useful, otherwise, why would we write this code? I mean, some people do enjoy writing useless code... but most of the time this isn’t the case. So why do utils crop up everywhere? How do we avoid this problem in our own code? Why does the name utils trigger me? Let me tell you my story.

The intent behind a bad name #

When I first started my career in software engineering, I worked on a large project for NHS Digital. The project involved a variety of products that all shared libraries and packages. The source for these products and packages was a single monorepo, supported by around 30 developers. One ticket on the backlog that I picked up was to refactor a database client shared across these products.

I began by researching the database clients the products currently used. I painstakingly mapped out an upgrade path for each product. I worked through all common scenarios for the database client. I deliberately designed the minimal interface this database client would need. In my eyes, it was going to be a work of art. Perfection.

I set about building the database client and after endless weeks of meticulous upgrades, I finally had the finished product. My masterpiece. The code ran like clockwork.

The downfall #

A year later I returned to work on the same codebase to find everything had become a mess. It felt like a rats nest to work in. There was no separation of concerns and every pull-request felt like you were wading through mud. I tried to figure out what the problem was. What had changed? Why had it all gone wrong? After a while, I realised it was entirely my fault.

When I introduced the database client to the codebase, I’d put it into a /shared/database/ directory. Sounds sensible, right? It was going to be shared across all the products, so that would be the logical place to put it. My flaw though was in creating that /shared/ directory.

This shared directory triggered a rapid descent into chaos. Any code that could be conceived as potentially reusable was immediately placed into the shared directory without any consideration. More and more of the logic of each product ended up here, and less and less in the product’s directory. Changes risked breaking everything. It wasn’t long before some of the shared code became product-specific, or became a spiral of self-consuming code with no clear interface.

Making a change to the shared code either meant changing every caller, overloading the behaviour, or making slightly different variations of the same code. Searching the codebase became near-impossible, duplication began to run rife and separation of concerns was non-existent. It was a nightmare.

So what did I learn from this? #

In creating a shared directory, I’d opened the floodgates. The problem with the name ‘shared’ is that it does not describe the code I had written. It only describes my intent for that code. Code being reused, or shared, is incredibly common. Utility code, shared code, library code, package code. These names are all to do with intent, rather than describing what the code is. I should have stuck to putting my code in a database/ directory. Forget the shared part.

So, what have we learned? We shouldn't call shared code shared, or utils, or common. It should just be code: named for what it is.

Looking for other ways to keep your code organised? Check out my article on writing IMMUTABLE code.

Enjoyed this post? Want to share your thoughts on the matter? Found this article helpful? Disagree with me? Let me know by messaging me on Twitter.

3 amazing REACT HOOKS to keep your code organized neatly

Douglas Parsons — Wed, 25 Nov 2020 08:52:00 +0000

Hi, my name is Doug. I’ve been a developer for several years and now work as the Lead Engineer at Shamaazi. Over this period of time, I have written a lot of different UIs and learned a lot of ways to structure React code.

This week I wanted to share my experience of the custom React Hooks I have found the most useful for producing websites in the cleanest, simplest way possible.

React Hooks

Hooks were first introduced to React in version 16.8, after being teased in 2018. There’s a fantastic guide introducing them on the React website. Simply stated, they are a way to write side-effects for functional UI components. This allows you to write parts of your UI as JavaScript functions, but still have the ability to manage state, call APIs, use storage, authenticate users, and so on.

React provides some hooks out of the box (useState, useEffect and useContext being the main three). On top of this, it allows you to compose your own higher-level hooks to separate out reusable logic. These custom hooks are what I’ll explore here. Here are the three I’ve found the most useful across the range of products we produce at Shamaazi.

Performing Asynchronous Actions

Most websites have to perform some form of asynchronous actions, whether it is loading data to display on the page or submitting data based on a user’s input and actions. It’s helpful to keep a track of the status of these asynchronous actions; is it currently loading? has it returned a result? Was there an error?

We found a lot of our components started sharing a lot of similar code, either for fetching data on an initial load or for submitting data. This looked like the following:

const MyComponent = () => {
  const [loading, setLoading] = useState(false)
  const [error, setError] = useState(null)
  const [result, setResult] = useState(null)

  useEffect(() => {
    const loadData = async () => {
      setResult(null)
      setError(null)
      setLoading(true)
      try {
        const result = await doSomeAction();
        setResult(result)
      } catch (e) {
        setError(e)
      } finally {
        setLoading(false)
      }

    loadData()
  }, [])

  if (loading) {
    return <>loading...</>
  }

  if (error) {
    return <>something broke</>
  }

  return <>{result}</>
}

All this loading and error logic can be pulled into a hook, making our interface much neater.

const MyTidyComponent = () => {
  const {loading, result, error} = useAsync(doSomeAction)

  if (loading) {
    return <>loading...</>
  }

  if (error) {
    return <>something broke</>
  }

  return <>{result}</>
}

This useAsync hook is responsible for managing the loading, error and result states, removing the need for all this logic within the actual component. It also lets us reuse this throughout our application. This massively simplifies loading data onto a page.

As a bonus, we found we also wanted the ability to execute an action later, rather than just when the component is created. This is useful for performing asynchronous actions based on a user's input; actions like submitting a form can use the same hook but pass a false value as a second parameter. This indicates that they don’t want the action to be executed straight away.

const { execute, loading, result, error } = useAsync(submitSomeForm, false)

<form onSubmit={execute}>
  ...
</form>

We also found that the hook sometimes caused a memory leak if a form submission navigated away from the component (e.g. a form might take you to the next page when it is submitted, but setting loading to false after you’ve been taken away from the form is a memory leak). We’ve handled this by tracking whether the hook is mounted on the page (tracked through useRef). We’ll only update any state if the component is still present. This avoids any memory leaks.

The full version of our useAsync hook is here:

import { useEffect, useState, useCallback, useRef } from 'react'

export default (asyncFunction, immediate = true) => {
  const [loading, setLoading] = useState(false)
  const [result, setResult] = useState(null)
  const [error, setError] = useState(null)

  // Track a reference to whether the useAsync is actually on a mounted component.
  // useEffect below returns a cleanup that sets this to false. Before setting
  // any state, we check if the cleanup has run. If it has, don't update the state.
  const mounted = useRef(true)

  useEffect(() => {
    return () => {
      mounted.current = false
    }
  }, [])

  const execute = useCallback(async (...args) => {
    setLoading(true)
    setResult(null)
    setError(null)
    try {
      const r = await asyncFunction(...args)
      if (mounted.current) {
        setResult(r)
      }
      return r
    } catch (e) {
      if (mounted.current) {
        setError(e)
      }
    } finally {
      if (mounted.current) {
        setLoading(false)
      }
    }
  }, [asyncFunction])

  useEffect(() => {
    if (immediate) {
      execute()
    }
  }, [execute, immediate])

  return { execute, loading, result, error }
}

Updating LocalStorage or SessionStorage

As part of some of our products, we populate a 'shopping basket'. This keeps a track of what a user has been doing. Sometimes, we want this to persist even if they navigate away from our site, refresh the page, or close the browser. To achieve this, we use a combination of localStorage and sessionStorage

React itself doesn't provide any hooks for storing data in localStorage or sessionStorage, but we wanted a consistent experience with useState. Realistically, it shouldn't be any harder to use localStorage than it would be to use state normally.

For example, we might want to use localStorage to keep track of a user's input.

const storageComponent = () => {
  const [value, setValue] = useLocalStorage('storage_key', 'default_value')

  return <input value={value} onChange={e => setValue(e.target.value}/>
}

Our hooks to achieve this look like the following:

const useStorage = (key, initialValue, storage) => {
  // Pass initial state function to useState so logic is only executed once
  const [storedValue, setStoredValue] = useState(() => {
    try {
      const item = storage.getItem(key)
      return item ? JSON.parse(item) : initialValue
    } catch (error) {
      console.error(error)
      return initialValue
    }
  })

  useEffect(() => {
    try {
      // Update storage every time the value is changed
      storage.setItem(key, JSON.stringify(storedValue))
    } catch (e) {
      console.error(e)
    }
  }, [storedValue, storage, key])

  return [storedValue, setStoredValue]
}

export const useLocalStorage = (key, initialValue) => {
  return useStorage(key, initialValue, window.localStorage)
}

export const useSessionStorage = (key, initialValue) => {
  return useStorage(key, initialValue, window.sessionStorage)
}

Authenticating users

A super common scenario we've come across is having a bunch of components that all care whether a user is logged in. They often care about acting on the user too, through methods like login, logout or resetPassword.

In order to keep all these components in sync, we only want a single source of information about the current user. We could do this by having a component wrapping our entire application that manages a user state, and passes any props down to where they are used for the user, login, logout or resetPassword methods.

This quickly becomes messy though, with many components that don't really care being passed user login and logout props even if they don't use them themselves - only a child of theirs does.

Luckily React provides the idea of a context. Allowing us to solve this problem.

We can create an Auth context, and use a hook to get any information from it we want. We can also embed our auth API calls into this context.

This would look like the following to use:

// In our top level App.js
import { ProvideAuth } from 'hooks/useAuth'

export default () => {
  return <ProvideAuth>
    <RestOfApplication/>
    ...
  </ProvideAuth>
}

// in a component that wants to use Auth
import useAuth from 'hooks/useAuth'

export default () => {
  const { user, login, logout, resetPassword } = useAuth();

  return <>
    {user}
  </>
}

This hook itself looks like the following:

import React, { useCallback, useState, useEffect, useContext, createContext } from 'react'

const authContext = createContext()

// Hook for child components to get the auth object and re-render when it changes.
export default () => {
  return useContext(authContext)
}

// Provider component that wraps components and makes useAuth() available
export function ProvideAuth({ children }) {
  const auth = useAuthProvider()
  return <authContext.Provider value={auth}>{children}</authContext.Provider>
}

// Provide Auth hook that creates auth object and handles state
function useAuthProvider() {
  const [user, setUser] = useState(null)

  // Get the logged in user when created
  useEffect(() => {
    const user = getLoggedInUser()
    setUser(user)
  }, [])

  const login = async (...) => {
    const user = ...
    setUser(user)
  }

  const logout = async () => {
    ...
    setUser(null)
  }

  const resetPassword = async () => {
    ...
  }

  return {
    resetPassword
    login,
    logout,
    user
  }
}

This has the additional benefit of keeping all of the authentication logic together. To change to a different auth provider, we would only have to change this one file.

Conclusion

React provides some really powerful abstractions for creating code that is neatly organised and easy to read. Here, we’ve looked at the three React Hooks I’ve found the most useful: useAsync for executing asynchronous actions either when a component is created or when a user performs an action, useStorage for using localStorage and sessionStorage in the same way as useState, and finally, useAuth for managing users and authentication.

These three hooks provide powerful abstractions that let you build React components in a simple manner.

Do you have any other custom React Hooks you find useful? Think I’ve missed any key ones? Please let me know.

Looking for other ways to keep your code organised? Check out my article on writing IMMUTABLE code.

Enjoyed this post? Want to share your thoughts on the matter? Found this article helpful? Disagree with me? Let me know by messaging me on Twitter.

How to write IMMUTABLE code and never get stuck debugging again

Douglas Parsons — Tue, 17 Nov 2020 14:41:00 +0000

I've written production code in a variety of different languages throughout my career, including Haskell, Scala, Go, Python, Java or JavaScript. While each language has its own clear benefits, working as a polyglot across a range of different paradigms has changed the way I write code. Certain skills and concepts are transferable regardless of the language being written. I believe immutability is one of these key concepts. By writing immutable code it is possible to make programs easier to reason about, easier to write and easier to debug.

Here, we’ll look at three things:

how walruses eating cheese can explain how immutability works,
why you should care, and
why the counterarguments against immutable code aren’t worth considering.

What is immutability? #

“unchanging over time or unable to be changed.” - Oxford Languages definition.

Immutability is the idea that once an object or variable has been created, its value should never change or be updated by anything. For objects or classes, this also includes any fields; literally, nothing should change! The object is effectively read-only.

Writing code in this style requires a mindset shift at times though. The first time I came across the idea, it made absolutely no sense to me and seemed insane. I was confused and wanted to immediately unpick it all, writing it in a way I was familiar with. Gary Bernhardt, in his talk on boundaries, gives a fantastic example of why it feels so wrong.

He talks about feeding walruses cheese.

In a mutable version, we might instruct each walrus to eat some cheese. This cheese then gets added to the contents of their stomach. Makes a lot of sense, right?

In an immutable version, we have to perform a mind-bending operation. To feed the walruses we would have to:

create a brand new stomach that’s the same as the old stomach, but with some cheese in it.
Then, create a new walrus that’s the same as the old walrus, except, with the stomach replaced.
Then, throw away all the old walrus.

At first glance, this sounds bonkers but stay with me - let’s look at what makes writing code like this worthwhile.

How does it prevent pain when debugging? #

Have you ever encountered:

undefined is not a function in JavaScript?
NullPointerExceptions in Java?
SegFault in C/C++?
panic in Go?
NoneType has no attribute foo in Python?

If you’ve worked in any of these languages, then chances are you probably have. The thing is, all of these errors are caused by the same thing: missing, or null, data.

Missing data and null values are definitely among the most difficult types of bugs to track down and fix. I’ve spent countless hours in the past sifting through JavaScript code trying to figure out why the value I thought should be there, wasn’t. Why my application suddenly crashed when everything seemed to be going fine. Sir Tony Hoare even describes null as “The Billion Dollar Mistake” because of the countless bugs, security vulnerabilities and crashes that have resulted from it.

Let’s just agree: nulls can be evil.

The reason these bugs are so hard to hunt down and to fix is that the effect (the exception) is far away from the cause (the introduction of null). Actually throwing a null pointer error happens some arbitrary amount of time after we introduce a null, and we get undefined errors accessing a property miles away from where we thought the property was set. Debugging becomes a case of reading carefully back through code until we find the cause.

The more state changes that happen in code, the more places these bugs can be introduced. Instead, we can attempt to reduce the surface area of any code. The fewer mutations in a codebase, the less surface area there is for bugs. This leads to fewer bugs.

If you only ever set a value once, there’s only one place that value can be faulty. If you make changes to an object as it gets passed around, any one of those places could introduce potential issues. If one of our walruses is faulty, we know it can only have happened when we made the latest walrus, complete with the new stomach. It can’t be an issue with an earlier walrus - they are long gone.

So really, immutability, or, never changing a value, really saves us from getting stuck debugging.

Why performance isn’t a concern #

Some eagle-eyed people might be thinking “those walruses earlier… isn’t throwing them all in the bin and making new ones pretty expensive? Won’t it make my code slow?”.

The answer isn’t simple.

You’re right in saying that throwing away walruses all the time isn’t totally necessary, and it can make things the tiniest amount slower sometimes. The keyword being sometimes here though. Sometimes compilers are clever enough to optimise this behaviour with something more efficient. Some languages even prefer immutability by default. Immutability also has great benefits when it comes to multi-threading or parallelisation, as it allows lock-free sharing, knowing that values won’t be changed.

Despite all this, even if creating new walruses is slower in the language you use, the cost of allocating a new object is almost certainly minuscule compared to anything else within an application. Unless you are benchmarking and actively measuring performance, then you almost certainly shouldn’t care.

Conclusion #

Immutability is a powerful tool when programming. It allows us to write code that is easier to debug and reason about. It requires a bit of a mindset shift, but in my experience, it’s definitely worth making the mental leap.

Give it a go, and let me know what you think :).

Looking for other ways to improve the clarity of your code? Why not check out my post on never using else statements.

Enjoyed this post? Want to share your thoughts on the matter? Found this article helpful? Disagree with me? Let me know by messaging me on Twitter.

Write better code and be a better programmer by NEVER USING ELSE statements

Douglas Parsons — Tue, 10 Nov 2020 13:26:00 +0000

I've been a professional programmer for the last several years. During this time I've risen rapidly through the ranks. I started as an intern, but I'm now the Lead Engineer responsible for a suite of products that serve over 160,000 people in over 140 different countries.

Recently, I took a look back across all the code I've written during these years (that I still have access to). I've written production code in a huge variety of languages, through Haskell, Scala, Go, Python, Java or Javascript. Across all these, I noticed one significant trend: I pretty much never use the ELSE statement.

I realised there's a clear rationale behind my aversion to else statements though. I believe that they shouldn't be used, and should be treated as a code smell instead. There are two reasons I think this: else statements break the line-of-sight rule, and they always lack context. I'll explain these two points in detail before showing how you can avoid using else statements.

Line of Sight rule #

I'm a firm believer that code should be optimised to be read by people in the future, rather than being optimised for being executed by machines. In this, I echo Donald Knuth's sentiment:

“Programs are meant to be read by humans and only incidentally for computers to execute.” - Donald Knuth, The Art of Computer Programming.

The problem is the ability to read code is subjective: it's hard to define exactly what makes code readable. One rule that seeks to clarify this though, is the line-of-sight rule. This is a popular rule in the Go community. Mat Ryer defines it concisely in his talk and article. Simply stated, this is the idea that the ‘happy path' in code should be indented as little as possible.

Contrastingly, any error handling or special case code should be indented further.

Any code that follows this has a unique property: scanning the least indented code is sufficient to understand what any piece of code is doing. Scanning the more indented code shows all the special cases and errors that can occur. This makes it super easy to understand at just a glance.

So how do else statements relate to this?

Else statements are problematic as they force code down a level of indentation. It suddenly becomes unclear what code relates to a ‘happy path', and what a special case really is.

This lack of clarity makes the code harder to scan through, and hurts the readability.

Lack of Context #

The ability to quickly and efficiently scan code is super important. Digesting small sections of code in isolation is a key part of this. We don't want to always have to read every line of code to understand a small part of a codebase.

Else statements make this harder as they space out the if condition and the code that is affected by it. This is best explained through two examples. First, can you tell what happens when these three lines of code are run?



if myVariable == nil { 
    return “”
}

Hopefully, this is fairly obvious. Let's take a contrasting example though:



} else { 
    return “”
}

We can see that without the if statement, we can't determine what this is meant to be doing. Why would it return an empty string? Is this an error, or the ‘normal' behaviour? This code instead relies on us remembering, and having read, the earlier context. This doesn't matter much when the statements are small, but if there's complicated logic within the if { … } block or we are scanning quickly, then the separation of context from code can hurt readability massively. It hurts even more when if/else statements are nested, or there are multiple of them in one function (which if statement is this else for?).

How to remove else statements? #

Now we've agreed that else statements are rubbish. But that's not much help by itself. The real trick is how to avoid them. Thankfully, there are two simple ways to do this:

Inverting the if condition and returning early, and,
Creating helper functions.

Inverting the condition #

This is the most common instance I come across. It can take two forms too - one where the else is implicit, one where it is explicit. The explicit version looks like the following:



func doSomething() error {
  if something.OK() {
    err := something.Do()
    if err != nil {
      return err
    }
  } else {
    return nil, errors.New("something isn't ok")
  }
}

The implicit is similar, but without containing an else statement per se. Instead, the else is implied by simply dropping off the end of the function (this one is more common in Python or JavaScript, where None or undefined are returned if nothing is explicitly stated).



function doSomething() {
  if (something.OK()) {
    return something.Do()
  }
}

Again, this isn't super clear what the full extent of the behaviour is. Without reading the whole function, the return values aren't clear.

By simply inverting the if condition, we can solve all these problems though.



function doSomething() {
  if (!something.OK()) {
    // return or throw error
  }
  return something.Do()
}

We can now scan this function and clearly see the indented error condition and normal flow, satisfying the line-of-sight rule. The behaviour is fully explicit, and we have no separation of context. This is much better.

Helper Functions #

We also get else statements that don't directly result in a return. This is usually through some special-case logic that isn't isolated properly. For example



  let charities
  if (country != "") {
    if (tier != "") {
      charities = getCharitiesByCampaignCountryAndTier(campaign, country, tier)
    } else {
      charities = getCharitiesByCampaignAndCountry(campaign, country)
    }
  } else {
    charities = getCharitiesByCampaign(campaign)
  }

  // do something with charities

The readability of this can be improved by pulling the charity-getting logic into its own function. This then lets the special cases be handled appropriately, and return early. By inverting some of the if statements, this can be improved further.

For example:



function getCharities(campaign, country, tier) {
  if (country == "") {
    return getCharitiesByCampaign(campaign)
  }

  if (tier == "") {
    return getCharitiesByCampaignAndCountry(campaign, country)
  }

  return getCharitiesByCampaignCountryAndTier(campaign, country, tier)
}

This helper function neatly encapsulates all the logic we'd need, removes the need for any else statements, and does a much better job of keeping the happy-path code to the left. This is much easier to scan through, and much more readable as a result.

Conclusion #

Else statements are a weird code smell. They harm the readability of any code by forcing equal levels of indents for error handling and for happy paths. They also have the unique ability to separate code from the logic that affects it. They are easy to avoid through the two techniques of returning early and splitting logic into helper functions. As a result, they are unnecessary. You can write better code and be a better programmer by never using them.

Some caveats (to stop the pedants).

In SQL CASE WHEN … ELSE … isn't really avoidable.
In Scala, implicit returns (avoiding return statements for referential transparency) means you have to use them - you don't really have the ability to 'return early'.
Ternary operators are fine.
In python, the ternary operator uses else. This is also fine.

Why EFFECTIVE COMMUNICATION is the key skill for being an awesome programmer

Douglas Parsons — Tue, 03 Nov 2020 12:54:00 +0000

Programming is all about communication. We communicate with computers to tell them what we want them to do. We instruct them to carry out tasks to the finest degree of accuracy. We communicate with our peers, sharing ideas and technology. We communicate with business owners, gathering requirements and context on problems. And yet, communication is a difficult thing to get right.

Have you ever tried to write a blog, teach someone about a topic, or write an email on a difficult topic?

If you have, then you will know how challenging precise communication can be.

🗣️ Why is communication so hard

When programming, all our communication immediately becomes at least twice as difficult. We write programs that have to be understood by the computer so they can run. But, they also have to be understood by future readers: people editing or maintaining them. They also have to capture exact requirements with little or no room for error. These requirements typically come from communication with non-programmers.

At the very least this means all programs are acting on around four levels of communication:

🗣️ A non-technical individual communicating to a developer.
💻 A developer communicating his intent to an application.
🤖 The application running and communicating to the computer.
🐛 Another developer looking to extend or debug the application.

Bugs, misunderstandings and issues in translation can occur at any, or all, of these layers. Writing software becomes a game of Chinese whispers.

🙊 Chinese whispers

This game becomes even bigger and more error-prone as you scale up teams, or have open-source contributors. The more programmers working on a codebase, the more miscommunication and misunderstanding there can be.

This communication is even more challenging because of the huge variation in what programs can do. A web application is concerned with attractively arranging pixels on a screen. A finance application might need complex calculations to be exact to the penny/cent. A device driver might care about manipulating bespoke hardware to certain voltages. The range is huge, and this complexity is part of the reason programming can be so overwhelming.

🤷 Ambiguity

Communication is also inherently hard because of how language works. In everyday speech, there are multiple ways to interpret any sentence. These ways are often dependent on context or our environments. And then, there are multiple ways to represent everything as code. There's an old joke that illustrates this well.

A software engineer gets sent to the shops by his wife.

She tells him "Go and get a pint of milk, and if they have eggs get six".

The engineer disappears and returns an hour later with six pints of milk.

“Why on earth did you get six pints of milk!?” His wife asks, furious.

He confidently replies "they had eggs!".

💥 So what can we do?

We've identified that communication can act as Chinese whispers. Language can be ambiguous, and we have many different points of communication in our day-to-day working. As a result, the quality of any communication is of critical importance. An individual who is capable of communicating effectively with non-technical people, other programmers and with a computer is invaluable.

It's important to remember how we communicate. When thinking of communication my mind always jumps straight to two people talking, or a board room full of people wearing suits.

However, this is rarely the case. We communicate far more frequently in our work, and talking isn't the only way we communicate. Talking often isn't even the best way to communicate.

There are four topics we communicate about and four ways we usually communicate these:

Code - we communicate through reading and writing code
Architecture - typically represented both in code and in diagrams.
Requirements - these are usually gathered through conversations or user stories.
Deadlines - usually through conversations or calendar events.

For code. Having clear code, that is easy to understand and digest makes life much easier. It frees up our brains to solve problems, rather than focusing on the minutiae of how an application works. Just like Donald Knuth, I believe that "programs are meant to be read by humans and only incidentally for computers to execute". When writing code, optimising for readability should always be our main focus. A well-structured, well-written codebase is far less prone to misinterpretation. Far easier to digest, and much more enjoyable to work with.

For architecture, we must clearly define and delineate any systems. Everything should have a clear purpose and a single responsibility. It should also be clear how these systems communicate. This should all be easy to digest (via diagrams, sensible naming, clear purpose and sensible code structure).

For requirements, we must extract every degree of detail possible. Clarify on everything, even points that seem obvious. Make sure they are all written down and that a common understanding is reached across the team.

Finally, deadlines. You might think these are unambiguous: "have X done by Y date". It seems straightforward, but remember that many "Agile" teams have to define what 'done' really means. Does it mean code complete? Tested? Deployed? Handed over to a different team? All the above and defects fixed?

✨ In Conclusion

Communication is challenging. Everyday language is ambiguous: we can often interpret very different meanings from the same sentence. As teams grow, our points of communication grow with them, and, even in small teams, several layers of communication take place. Mis-interpretation or a poor understanding can sneak in at any of these layers.

As a result, we have to focus continually on communicating effectively. Having good quality code that is clear in purpose and easy to read and understand is essential. We need to keep our systems organised as neatly as possible, with clear diagrams and purpose. Requirements and deadlines need constant questioning, and every assumption challenged. Through doing all this, we can communicate more effectively, and be awesome programmers as a result.

Thanks for reading. If you enjoyed this article, please share or follow me on twitter.