Forem: Pato Z

Hive mind

Pato Z — Tue, 26 Mar 2024 12:43:58 +0000

A dim lit room. A bunch of hooded figures sit around the table.

If you didn't know better you'd swear they are doing an ancient ritual to wake up a sleeping monstrosity consisting mostly of tentacles, cones at odd angles and sexual innuendo.

But you do know better, these are just your nerdy friends and they are doing what you've come to know as...

Hive mind

"Hoodies never go out of style" you think as you take your position around the table.

You take a quick look around and, for a moment, feel transported to one of those logic puzzles from childhood:

"The person with the extreme allergies sits between the one that speaks in the third person and the one solving the Rubik's cube. The one that drinks soda non-stop is also a fan of typed languages...

The rules of "Hive mind" are pretty simple, people take turns telling stories around the table. Stories of magic and mystery, of insanity and ingenuity, of the highest highs and the lowest lows... but they all share a common pattern, there's something to be learned from them.

And so it begins...

Story 1: the forest and the trees

"OK, so we have events. These things capture something that happened in our app. They come in two flavors: frontend and backend".

"Frontend events are close to the end users and have a ton of context (what did they do just now, what did they do before that, etc.). They are also insecure and unreliable".

"Backend events are trustworthy and secure but their context is quite limited".

Problem statement:

how might we add some rich frontend context to our "User registered" backend event?

"Any ideas on how to solve this?"

...the speaker reads the room, the readers speak their mind...

(You, yes you 🫵 are one such reader, what do you think we should do?)

After the brain storming and thundering quiets down, the room seems to agree on two possible solutions:

Make "User registered" a frontend event (that way we can add all the context we want, security be damned)
Send extra tracking stuff to the “register user” endpoint and track from there (because we love our endpoints with all sorts of random extra stuff in them)

Dismay feels the air, the speaker continues...

"Pretty disappointing, right?"

"The reason for this is that the problem statement is wrong! It biases out of other possible solutions, let's take a step back..."

Why do we want that extra context in the "User registered" event?
Because we want to use it in a report
Where is that report coming from?
Our data warehouse
Don't we have both backend and frontend events in the data warehouse?
Yes!
Can't we just JOIN them?
Yes!

"OK, so the problem statement could've been:

We want to be able to create a report on registered users that is trustworthy (backend) and includes behavioral tracking stuff we have in the frontend.

"See how the two solutions above still apply to this problem statement?"

"...but now we have other options at hand, like tracking two events, one from the backend and one from the frontend and then JOIN them back in the report".

Morale: when all solutions are disappointing perhaps the problem is ill-defined and we need to ask ourselves what are we really trying to accomplish.

Story 2: the run query gig

One hooded figure takes out a strange contraption, presses a button and speaks into a microphone:

push push pop pop
push push pop (rest)

The contraption starts looping the phrase non-stop.

If you had to describe this to someone else you'd probably call it "Infinite stack beatboxing".

Others might call it "an OOM waiting to happen", but in this circle this is considered art.

The figure, now standing on the table, begins some free-style rapping:

This is the story of a thing gone wrong,
So I sing this song, 'cause y'all should know.

We were dealing with this ugly query,
It was big and hairy, it was slow and scary.

So we caged that beast in this neat old function,
That would hide dysfunction and prevent malfunction.

We called this function the runQuery gig, ho!
The runQuery gig, and it goes like this:

(snippet instrumental break)

function runQuery(
  ...some args...,
  addSomeCrap: boolean,
): Stuff;

That nasty query's usage was not small,
So we replaced it all with a runQuery call.

While we worked addSomeCrap was true,
But as the code improved that would get removed.

Unfortunately runQuery had a trap,
'cause it had no map, and you could still add crap.

Surely enough someone did fall prey,
And to our dismay, added more new crap, no way!

So we saw the problem with our runQuery gig, ho!
Our runQuery gig, and it goes like this:

(reflective prosaic coda)

We had a problem with our nasty query so we chose to hide it behind and abstraction layer, one we called runQuery.

We knew in the end runQuery would be efficient, but as we iteratively worked through the all the calling instances we needed a way to still convey its inefficient version (and so addSomeCrap was still true sometimes).

What we failed to see is that runQuery became our abstraction's interface and that interface could be used for good and for evil, and turning one into the other was just one true away.

A better approach was to have two separate runQuery versions the proper one, runQuery, and its evil twin, runQueryInefficientlyPullingAllSortsOfCrap.

We could then throw a nice comment in the latter explaining why you shouldn't use it, what to use instead and even throw in a @deprecated tag so that the IDE helps you spot the trap.

(Hooded figure bows, standing ovation, the curtain falls)

Why did the chicken cross the road?

Pato Z — Fri, 09 Sep 2022 14:30:04 +0000

A story about microservices, farm animals, couple's therapy and the joys of cooking

Lily the Hen is a master cook, she's at the forefront of her profession and regarded by most as the leading chef in the industry. Her assistant is Rusty the Rooster and together they set out to cook an answer to the ultimate question of nature...

Why did the chicken cross the road?

Lily and Rusty are a rockstar cooking team. Whenever they enter a cooking competition, they make the judges cry, get handshakes and get asked for their recipes.

Lily can cook her peers into tears and is the envy of every restaurateur in town. Rusty is like a cockerel-shaped Robin that uses the underwear on the right side of the costume.

They can certainly cook their way out of a tough situation and do miracles with the most basic ingredients. They could even take the jar of expired mayo and the two sad-looking gherkins in my fridge and produce a tasty and wonderful functional specification.

If there's something they care about, that's their craft and they're always looking for ways to make it more efficient.

One time they decided that in order to improve the quality and latency of their cooking they should...

Move in together

They reasoned that sharing an address space was the optimal way of cooking.

So they rented a small 1-bedroom studio apartment in the artsiest neighborhood in town, pooled resources, shared rent and moved in.

At first things were great, cooking at maximum speed, no interruptions, no distractions, winning contests without even trying.

But soon the natural frictions of cohabitation started to pop up.

It all started with some political disagreement around endianness, quickly followed by a stream of day-to-day impossibilities, like the padding character of choice, "who alloc'd this pile of dirty laundry on the bathroom floor and who's responsible for freeing it?" and "where the pluck did you leave my curly brace this time?".

Rusty suggested that some borrowing semantics were in order, but they discarded the idea as too hard to keep track of who owns what. They even considered paying someone to collect their garbage for them.

Most arguments were just simple things, easily ignored, but there was one thing that eventually tore them apart: their taste in music.

Lily was big into Progressive Death Metal, while Rusty, sitting firmly on the polar opposite end of the musical spectrum, was a die-hard fan of Scandinavian Power Metal.

The Great Metal Divide

These irreconcilable differences eventually lead to the only logical conclusion: they needed their own space.

And so, they both moved out into their own place, right across the street from each other.

The benefits were plain to see, they could do as they pleased all-the-time.

No asking for permission.

Clothing? Optional!

Full autonomy, and best of all, they could listen to the right kind of metal in their underwear all night*.

* and part of every day

Rubbing their hands with an evil grin

But, of course, not everything was good.

For one, they were spending a lot more.

They doubled their collective rent, bills, utilities, etc.

Of course, their cloud landlords were so happy with this arrangement that they might even start promoting this kind of thing at some point.

Burnt cookies

But the worst part was that their cooking was suffering as well.

Back when they were living together, Lily would ask Rusty for some chopped onions and he would hand them over immediately.

Now the picture looked very different. Lily would put on her coat, leave the house, lock the door behind her, cross the street and ring Rusty, wait for him to appear, ask for the onions, wait in the cold for what seemed like ages until Rusty produced the onions, cross the street again, marshal through the door, take the coat off, and, finally, use the onions.

More often than not it was too late, her cooking had timed-out and she had to start all over again.

Not to mention...

The risk, oh the risk

Even though everyone assured Lily and Rusty that this was an extremely safe neighborhood, with a stable latency of single-digit milliseconds, they were very concerned.

What if they needed to cross the street in rush hour? What if there was a parade or something?

There had to be a cap to all that networking optimism.

And what is the deal with that virtual private cloud anyway?*

* is it a virtual cloud that is private, or is it a cloud that is virtually private?

I just called to say I need you

Well, at least they didn't depend on each other, freedom was good right?

Truth is, Lily crossed the street a lot those days.

What they thought was a beautifully acyclic, directed relationship turned out to be a lot more complex, as relationships often are.

Rusty soon discovered that eating raw onions all the time is no fun, so he kept crossing the street to ask Lily for things.

In these strange times the bills and the failure modes kept stacking.

While metal freedom in their underwear and long nights of debauchery and deployment without giving explanations were great, their hearts were not in it.

Their cooking was suffering and that was unacceptable.

One day, their pet bats, Coupling and Cohesion, flew out the window into the freezing cold, and they decided something had to be done.

House rules

Maybe they were too hasty to move out. They fell prey to that very human condition of searching for balance by alternating exclusively between the extremes.

They needed space, freedom and abstraction boundaries, that's for sure. But boundaries doesn't necessarily mean network boundaries, right?

Maybe they could make this work after all.

They decided to give this another try and moved back in together, but this time to a two bedroom apartment.

They needed space to share and space to be alone.

They set up clear rules to live by, interfaces they'd call them, and stuck with them as if their life, or even more, their cooking depended on them.

They supported each other while staying out of each other's way.

And, of course, they each got noise cancelling headphones to headbang to their heart's content, because who the pluck would want to listen to the wrong kind of metal anyway?

The Strategy of One

Pato Z — Wed, 01 Jun 2022 16:05:32 +0000

Accidental complexity is quite an oddity
It's making things hard
Just for fun, without a plan
To make me feel smart

The Strategy of One

The Strategy of One always gets me bummed
Why even bother?
Just call the thing, do that and win
Don't feed that monster

const monsterFactory = (type: MonsterType): Monster => {
  switch (type) {
    case 'gizmo':
      return gremlin;
    default:
      // |\---/| I bet `type MonsterType = 'gizmo'`, what
      // | o_o | sort of crash-loop inducing, exploding
      //  \_^_/  kittens game are we playing here?
      throw new Error('Invalid monster, sorry 🧟');
  }
};

The Sequence Implied

The Sequence Implied should be simplified
Call a then call b
The order preserved but hardly observed
Much less guaranteed

export interface TableBuilder {
  buildTable: (materials: WoodAndNails) => Table;
  getMaterials: (style: Style) => WoodAndNails;
  getStyle: (mood: Mood) => Style;
}

// There's probably just one valid way of calling these 👆.
//   ________  
//  /       /| Do we really need that inversion of control?
// /_______/ |
// |_______|/| What about `buildTable: Mood => Table`?
// || ||  |||| 
// ||     ||   Can't we compose functions instead of defining
//             single-use methods?

The Eager Effect

The Eager Effect is a source of unrest
Why not just describe?
Let others worry, take all the glory
By having them apply

const nightFalls = async (subject: Werewolf, moon: Moon) => {
  const weCouldReturnThis = moon === 'full'
    ? transform(subject)
    : humanize(subject);

  // ...but instead we apply save to it...
  await save(weCouldReturnThis);
  //                n,
  //              _/ | _
  //             /'  `'/  ...and now testing this function
  //           <~    .'   is much harder.
  //           .'    |
  //         _/      |    (because we need to intercept
  //       _/      `.`.   that `save` call and steal its
  //      / '   \__ | |   first argument).
  //  ___/      /__\ \ \
  // (___.'\_______)\_|_|
}

The Triangle of Gloom

The Triangle of Gloom means impending doom
Maybe not right now
And not for you, you are testing with two
But in prod it will blow!

const result = lots.reduce(
  (acc, v) => {
    const next = ...

    return [...acc, next];
    // How many times are we iterating `acc`?
    //
    //   /\   This is a triangular number 
    //  /  \  (e.g. 9 + 8 + ... + 1) and
    // /____\ it has quadratic complexity: O(n^2).
    //
    // Just use `acc.push`, much to your functional soul's
    // discomfort, to bring this back to linear, O(n).
  },
  [],
);

The Not-a-Duck

The Not-a-Duck lives in the muck
It wants to be clever
Yet it walks like a duck, and it talks like a duck
So we name it whatever

type Duck = (say: Word) => Cuack;

const duck: Duck = () => 'cuack!';
const politeDuck: Duck = say => `respectfully, ${say}`;

const duckBroker = (say: Word): Cuack =>
  say === 'hi'
    ? duck(say)
    : politeDuck(say);

// Why is this "duck broker" not a duck?
//
// <(.)__  It certainly looks, talks and walks like
//  (___/  a duck.
// 
// Why are we introducing new names for plain
// composition? Isn't this just more of the same?

The Solution Space

Pato Z — Fri, 27 May 2022 17:35:15 +0000

Long after the human extinction, two factions of sentient robots fight for supremacy over a specific region of Space. This region is known as...

The Solution Space

Picture a region of space crammed with brutal constructions and artifacts of massive complexity. Now project that over time and superimpose a war for total domination and you'll get a picture of what the Solution Space looks like.

A vast and chaotic expanse of assorted debris, of good intentions and broken abstractions.

In this space wasteland you'll find the Iteratrons and the Recurcyborgs. To the untrained eye, both look like 70's sci-fi robots but once you get to know them you'll learn they are sworn enemies.

What's next?

The Iteratrons are obsessed with performance and profess the creed that fast code is written once and maintainability is futile.

When you look at a typical Iteratron you'll see wires sticking out of places and connecting to weird and uncomfortable parts. They wear these hardcoded (or is it "hardwired") hacks with pride as a sign of optimization, the ultimate goal.

When they are feeling stressed and want to relax they unwind their loops and inline their functions to avoid wasting time in outrageous stack frame pushes.

They are exhaustive and eager and approach problems head on. Given a list of chores, they always pick the first one, do that, then look at the second one and do that, and so on. Sequencing is an integral part of their beings.

Occasionally they glitch and, given a list of chores, they pick the first one, do that, then pick the first one, do that, you see where we're going right? Other times, after they are done with the chores on the list, they keep picking tasks that are not there and end up doing incomprehensible nonsense.

They fly through space in makeshift ships, soldered and wired together for no apparent reason. Like a big collage of heavily optimized pieces that when combined de-optimize the whole.

You can always tell where Iteratrons have been because they leave a trail of leaking abstraction fluid behind them.

Behead and repeat

The Recurcyborgs are the opposite of your typical cyborg, they are stylized and modular. They actually wear clothes and they store those neatly folded when they are not using them.

Recurcyborgs suffer from an acute case of fractality which makes them see infinity everywhere they look. Every problem is an infinite problem, every sequence is an infinite sequence.

They see patterns where others would see only chaos (oftentimes because there's only chaos). Their infinity proclivity makes them have a very unique approach to problem-solving.

They are overwhelmingly lazy. Trying to be exhaustive is both trying too hard and denying the nature of the universe. You should only solve the problem you need right now, the ...rest can wait.

Given a list of chores, they always pick the first one, do that, then create a new list of chores with the remaining ones. Someone can take care of that eventually.

When they glitch they heat up and the top of their head explodes in a glorious stack overflow.

They fly through space in infinitely long modular ships where every cons cell looks just like the previous one. It is rumored that, given enough time, if you manage to look behind the last module of one of their ships, there's always a bumper sticker that reads "My other car is a cdr".

Deadlocked

Each faction has a secret weapon they've been developing that they believe will win them the war.

The Iteratrons wield a weapon of devastating power known as the goat-2. The instability of this weapon makes it capable of destroying both the target and the wielder, while crashing the entire process in the process. Iteratrons believe that deep in the fabric of the universe everything is made of covalent bonds of pairs of goats.

The Recurcyborgs have two deceptively innocent tools. One is called the return, often called the "put-thing-in-a-box". The other one is called the bind and looks like a chain of sorts. Although innocently-looking these tools are extremely dangerous, when put together like a late 70's superhero cartoon, they become a weapon that shall not be named capable of infusing insanity in everything it touches.

The Observers

Orbiting around a long uninhabited planet there's an abandoned space station. In that station lives another form of sentient artificial intelligence known as the Archivers.

These are a cluster of disembodied processes whose original purpose was to record the history of the universe.

All other beings often come to the Archivers to seek advice, since they are regarded as the wisest things alive.

Ironically, they've been around for so long that they've looped through their 32-bit integer timestamps many many times. As a result, they're unable to distinguish the past from the future.

Others think the Archivers are able to predict the future when in fact they are predicting the past.

The Prophecy

Buried deep within the Archiver's data banks there's a record of a mythical place called "the Machine". According to this story, an ambassador from the Machine will eventually arrive in the known space. It is said that this ambassador, a divinity from the Machine, will put a stop to all wars by unleashing an unstoppable force.

One day, an archiving process was defragging a long forgotten data store when it came across this myth. Shortly after it had a vision* of a vessel approaching the station.

* not a vision as a premonition, but rather as looking from a camera out the window.

That arrival would change everything...

The truce, the speech, the truth

The ship docked and a strange being disembarked into the Archiver's station. The Archivers immediately assumed this was "the ambassador" and the news spread more or less at the speed of light.

A fragile truce was established and both Iteratrons and Recurcyborgs sent emissaries to see what was going on.

The stranger, silent until now, eventually spoke.

"I'm known as The Duality, the Empty Interface, the Base Class, the Identity Function, but my friends call me 'The Void Pointer'*"

* as in "the one that points at the void", but also "the one that points out the meaningless of all". Not to be confused with my arch-nemesis "The Null Pointer", master of the crash-loop, whose fault is only the segmentation one and whose calendar has only one day, the zero.

"Everyone and everything looks the same in my eyes. I'm blind and yet I see everything."

The crowds stood silent, trying to process all this nonsense, as a withered old poster that read "Know your paradoxes" slowly peeled away from a wall and fell to the floor.

The Stranger continued...

"I bring you a gift, a truth, a power to end all conflict".

The Stranger pulled out a shiny 5¼ floppy disk. "This is known as 'De Broglie's final hypothesis', the ultimate lambda."

Recurcyborgs would always get immediately excited at the mention of any kind of "lambda". Iteratrons appreciated the physics reference, even if all of this still felt like utter gibberish.

The bridge

The Stranger loaded the floppy disk and everyone was suddenly aware.

Every recursion can be rewritten as an iteration, every iteration can be rewritten as a recursion.

Some problems lend themselves better to recursion, like the ones that deal with trees, fauns and other forest creatures. Others feel more natural as raw iterations.

Recursion seems wasteful because of the context preserved between calls, yet a recursive function re-written as iterative will hold a similar state in some form of stack (even if not in the stack).

When recursion runs wild it'll probably overflow the stack, but when iteration runs wild it can burn the CPU to a crisp.

A particular form of recursion avoids having to remember all previous executions, this is known as tail recursion. Funnily enough, a tail-recursive function looks a lot like an iterative function.

The disk also spoke of trampolines and other patio furniture, of animal-shaped combinators and other curios.

In the end both Iteratrons and Recurcyborgs saw that their fight was meaningless.

The only possible explanation

Long after these events the Archivers struggled to puzzle the identity of this mysterious Stranger. Eventually they concluded it had to be human. Not because of the speech or the crazy dated floppy disk but because of the Stranger's final words before leaving never to return:

"You know, I never really got why you would focus so much on this barren piece of broken 'Solution Space', when the 'Problem Space' over there is ripe with opportunity, with wisdom and wonder..."

Only a human wouldn't know when to shut up.

This is going to be a good day

Pato Z — Mon, 09 May 2022 11:59:19 +0000

You wake up with a smile. You unplug from your charger, walk over to the coffee machine and brew your morning cup.

You enjoy a cinnamon roll with your coffee while looking out the window into the immensity beyond.

You don your jet-black helmet*, your cape and boots and leave your quarters.

* with standard Ominous Voice Modulator™ included.

This is going to be a good day

You stroll to the bridge waving hello to everyone you meet along the way*.

* as usual they fall to their knees, grabbing at their throats, choking.

This looks like a perfect day to train your army of white-armored clone people. You already know they are useless at shooting things with their blaster guns. They are also pretty lousy meeple workers. Surely they must be good for something.

You decide to push your luck and try the famous pick-up-and-deliver mechanic. Yes, this has to work, you briskly walk over to...

The training grounds

You're standing in the middle of a big room. The room has 10 doors. Behind each door there's a table. In your hand you hold 10 coins, let the party begin...

The one that works

You call in one of your clones and task it with placing one coin over each table.

The clone takes 1 minute to put the first one down so 10 minutes later the task is done.

But 10 minutes for this menial work feels excessive. What if...

We throw more clones at the problem

You reset the experiment, summon 10 clones and task each with placing one coin over each table.

After roughly 1 minute of scrambling over the floor the task is done.

That was a 10x improvement, no small feat, we just needed to throw more clones at the problem.

This is getting boring, let's try something different for a change.

The important task

You now summon back a clone and give it the 10 coins. You then task it to place all 10 coins in the table behind a single door. Sure enough, 1 minute later the task is complete.

Looks like we're not even trying.

The Mr. Burns effect

What if we ask 10 clones to place the 10 coins on the same table, at the same time. Surely this is going to be amusing.

Clones are optimistic by default. That's the only way for them to go through life putting up with all the nonsense they are asked to do.

So when trying to perform this task they just assume things will work. This is called optimistic locking and surely nothing can go wrong with it.

The clones know not to try your patience so they shuffle to pick-up their coin and rush to the door to place the coin on the table.

And, of course, only one clone gets through the door. The other nine clones hit their heads, stumble and fall to the floor, stunned. As they recover they rinse and repeat, one gets through, eight fall to the floor.

You just sit there laughing ominously, watching clones hit their head and fall. It doesn't get old but you take this time to run some quick math in your chest calculator:

clone #1: 1 minute (one success)
clone #2: 2 minutes (one failure, one success)
clone #3: 3 minutes (two failures, one success)
...
clone #10: 10 minutes (nine failures, one success)

So we should expect about 10 minutes to get this done, but a much more interesting result is the damage done to the clones.

If you count the number of times a clone fell to the floor you'll get a much funnier picture:

clone #1: fell 0 times
clone #2: fell 1 time
clone #3: fell 2 times
...
clone #10: fell 9 times

So in total you enjoyed 9 + 8 + 7 + ... + 1 = 45 slapstick moments in those 10 minutes. Fun aside, the amount of effort required to make this work was pretty intense.

But hold on a second, there's no room for optimism in your army and certainly not on your watch.

So you use your famous mind tricks to squeeze all the optimism out of your clones and try again.

Pessimistic boredom

This time the picture is different, as the clones rush to the door one reaches the door first and closes it behind it. It eventually comes out and another one goes in. This is all very civilized and boooring.

After 10 minutes of tedium the task is finally done.

All is well

Your locking adventures left you invigorated. You observe that parallelism can be a great performance optimization (you managed a 10x improvement in your second experiment).

On the other hand, running stuff in parallel just for the sake of comedy is not a good performance strategy, your optimistic lock experiment was 10 times worse than the previous one (i.e. a single clone dropping the 10 coins in one go).

You find it interesting that both pessimistic and optimistic locking experiments took the same 10 minutes, although the optimistic one also produced 45 concussions.

You conclude that pessimistic locking is consistently bad, while optimistic locking is surprisingly (and comically) bad.

Since you're all about the bad, you feel good. This was indeed a good day.

Tomorrow you plan to try the optimistic lock again with 90 clones. That should net you a cool hour and a half of pure, unadulterated, slapstick fun and a whopping 4005 falls to the floor.

But all that will have to wait for now you have...

More pressing matters

You walk back to the space port. On the way there you take a brief detour to commend Governor Cushing on the quality of his clones.

You then proceed to shuttle off of the station and into a much anticipated family reunion.

You know this was a good day because you find yourself humming to your favorite tune, the one that's always stuck in your head: dum-dum-dum dum-de-dum dum-de-dum...

The Tinkering Hobbit (excerpt from The hidden language within the language)

Pato Z — Mon, 25 Apr 2022 01:35:51 +0000

The night falls over The Shire and gray smoke billows from the chimneys as every house gets ready for a hot supper and some stories before bed.

Every house except one.

In the distance, off the beaten path, in a strange house (strange by hobbit standards) something else is going on. You could tell it was strange by the plumes of green and purple smoke coming out of its roof, or by the clinks and clanks coming from inside.

This is not an ordinary hobbit house, in here lives...

The Tinkering Hobbit

Tinky the Tinkering Hobbit is the one responsible for all the cool hobbit contraptions you see out there. Adventuring hobbits have no time to learn how stuff actually works.

Back in high-school, some hobbit bullies called Tinky "non-canon" because in the 9 hours of chasing a ring all over the land (extended director's cut), plus some other 9 hours of stealing treasure from a mean dragon, there was absolutely no mention of Tinky.

The Tinkering Hobbit never paid any attention to those comments. Understanding the inner workings of things, that was the ultimate goal.

But Tinky knew understanding was not all, sharing the knowledge was equally important, and so, a community was born, almost like a "fellowship", although they preferred to be called "The Tinkerhood".

For months Tinky has been researching, tinkering, trying, failing, and trying again.

One day after knocking down half a sink with a headbutt and having flux capacitor flash forwards, Tinky saw something in the edge of the metaphorical vision. Something so clear and yet so elusive, something that was there all along, hidden in plain sight, something that could change everything.

But Tinky knew well that the key to truly understanding stuff was sharing the knowledge so a book was born. A book we partially reproduce here for all of you honorary members of The Tinkerhood. A book called...

The hidden language within the language

(excerpt from Tinky's book)

Hi there, my name is Tinky, the Tinkering Hobbit. You know me, I've been tinkering since forever. Today I'd like to tell you about my recent explorations into the world of types, more specifically of Typescript.

Some of the rabbit holes in this field are deeper than the deepest pits of Moria, so we'll tread carefully and take it slowly.

Let's start with...

Unions, really?

Typescript has something called "Union Types". You write union types using a | like this:

type Monster = Orc | GiantSpider;

You read this as "a Monster is either an Orc or a GiantSpider".

I'm sure there's an occult field of magic where calling this a "union" makes sense, but if you are a simple tinkerer like myself, one perhaps familiar with set-union or the bitwise operator |, then this will be at least a little confusing.

If you inspect that Monster type, you'll see it has readily available only the parts that both an Orc AND a GiantSpider have in common. Now if you think about sets, a set-union has the stuff that EITHER or BOTH of its components have.

I guess what I'm trying to say is: don't pay much attention to the name. These are called "union types" (even though they work like intersections or something) but it doesn't matter they could've been called something else.

Do know that in this book when we refer to "union types" we are talking about these things and not what you might intuitively think of as "unions".

An objective view

Have you ever been into object-oriented programming? In there you used to have classes and such. Some classes were abstract representations of other classes, like:

class Monster { /*... */ }

class Orc extends Monster { /* ... */ }

A Monster is an abstract representation of an Orc. Or an Orc is a more concrete type of Monster.

Now, in OOP, a recurring problem (and cause of great anxiety) is called "downcasting".

Downcasting is the act of taking a generic thing, like a Monster that you know (how?) to be a specific thing, like an Orc and asserting that this thing is an Orc, like this:

function f(monster: Monster) {
  const orc = monster as Orc; // how do you know this, pray tell?
}

Note that the reverse problem, know as "upcasting" or treating something specific as something generic, just works:

const orc = new Orc();
const monster: Monster = orc;

Why are we talking about this, you ask?

Well...

Narrow and wide

A very similar problem occurs with union types. Let's say we have:

type Monster = Orc | GiantSpider;

And we have a monster variable of type Monster that we know (?) it's an Orc and we need to assert that.

When dealing with union types, what we used to call "downcasting" is now called "narrowing" as in "you take a wider, more generic type and you narrow it into a more specific one".

That being said, narrowing and downcasting are pretty much the same and we could use a similar hack as we did with classes:

function f(monster: Monster) {
  const orc = monster as Orc; // how do you know this, pray tell?

But there are a couple of better options. We could use a discriminating property like this:

interface Orc {
  type: 'orc';
}
interface GiantSpider {
  type: 'giant-spider';
}
type Monster = Orc | GiantSpider;

So that we can later use monster.type to check for 'orc' and get type-checked narrowing:

function f(monster: Monster) {
  if (monster.type === 'orc') {
    // in here the type of `monster` is `Orc`
  }
}

En garde!

Another, more advanced, narrowing alternative is to use something called "type guards" (also known as type predicates). These are just functions that take monsters and return a fancy boolean stating that you just checked and yeap, this monster is an Orc.

const isOrc = (m: Monster): m is Orc =>
  // some logic here (is smelly, is clumsy, etc.)

// note the `m is Orc` part which is the "fancy boolean"

Type guards are the answer to those "how did you know that?" questions above.

To be fair, technically all these strategies for type-checked narrowing are called "type guards" (e.g. type predicates, discriminating properties, the typeof operator, etc.), but I grew up calling type predicates "type guards" and old habits die hard.

With all that warm up out of the way, it's time for a crucial choice.

Red berry vs blue berry

I once read of a powerful wizard called Morpheus that once offered his apprentice N00b a choice between two berries: eat the red berry and your eyes will be open, but there will be no turning back..., or take the blue berry and enjoy blissful ignorance.

Dear reader, I'm giving you the same choice right now. Keep reading at your own peril or choose to walk away now oblivious to the power (and responsibility) ahead.

Hidden in plain sight

Let's say that we have a function:

const someFn = (t: number): number =>
  /* do something with `t` */

This function takes a single argument t of type number and returns another number.

Now look at this:

type SomeFn<T> =
  /* do something with `T` */

Now squint your eyes and look at both together. Do you see it? Both look like functions, right?

someFn is a run-time function while SomeFn is a compile-time function!

We could say that SomeFn takes a type argument T and returns some other type value.

In other words the type alias SomeFn acts as a function over types instead of a function over values.

Let's see another example:

type MakeArray<T> = T[];

This MakeArray function takes a type argument T and returns a new type T[].

So we already have variables like T above (capable of holding types) and we have functions (like MakeArray and SomeFn) capable of producing new types from existing types...

Do you see where I'm going with this?

The big reveal

We should be able to write pretty complex programs just by combining the elements above (i.e. variables and functions). These are not regular programs, they are programs that operate on the type-system of our "real" (or should I say "other") programs.

We could craft our type-system programs to provide better domain-specific compilation (think DSLs, encode business rules in the type-system, etc.).

But for us to be able to do really cool stuff in this hidden language we need a couple more elements.

Who types the types?

Did you spot it back in the SomeFn example? Here's again all together:

const someFn = (t: number): number =>
  /* do something with `t` */

type SomeFn<T> =
  /* do something with `T` */

See how we know t is a number but we have no clue what T is?

We went to all the trouble of using Typescript because we believe types are important and now that we are coding at the type-system level we have no types?

Outrageous!

...actually, there is a way for us to specify types in our type-system programs:

type SomeFn<T extends number> =
  /* do something with `T` */

So what you do with : in run-time land you can do with extends in compile-time land.

But what about...

Narrowing^2

What if we have something like:

type F<T extends Monster> =
  // do something if T is an Orc

How can we narrow T from Monster to Orc? Well, the answer is something called conditional types but I like to think of these as good old ternary operators:

type F<T extends Monster> =
  T extends Orc
    ? // T is an Orc (or maybe an Uruk-Hai?)
    : // T is not an Orc

If you don't want to allow one of the branches of your conditional types you could use the handy type never (I shudder to think of a run-time analogy for this):

type OnlyForOrcs<T extends Monster> =
  T extends Orc
    ? // T is an Orc (or maybe an Uruk-Hai?)
    : never;

OK, that's more like it, we have functions, variables and conditional expressions. What else do we need?

Unions as lists

We talked a lot about union types, truth is, for our type-system programs a union can function pretty well as a list. You need to return a bunch of stuff, you could return a union type of that.

Let's see some examples. You're probably familiar with good old Object.keys:

const a = { a: 1, b: true };

const keysOfA = Object.keys(a); // ['a', 'b']

Object.keys takes an object and gives you back an array of its keys (strings, numbers or symbols).

Ready for another squinting eyes moment? Here it goes:

interface A { a: number; b: boolean }

type KeysOfA = keyof A; // 'a' | 'b'

By now you are an expert at this. keyof T is the type-system equivalent to Object.keys(t), but instead of returning an array, it returns a union type.

Let's see the other obvious example, I'll throw it all together this time:

const a = { a: 1, b: true };

const valuesOfA = Object.values(a); // [1, true]

// -----

interface A { a: number; b: boolean }

type ValuesOfA = A[keyof A]; // number | boolean

Clearly there's a pattern here.

Unions and the Occult

One of my favorite run-time operations over arrays is without a doubt .map. Here's a refresher:

const a = [1, 2];

const b = a.map(value => value % 2 === 0) // [false, true]

.map takes a function and an array (via this), it then applies that function to every element of the array and returns a new array with these mapped values.

When dealing with this type of operations, some people with a magical proclivity like to use a spell called "The Functor". We tinkerers are not usually into magicks so we'll just stick to thinking of map as a function that can change every element in an array to something else, preserving order, cardinality, etc.

We said some time ago that union types could double as type-system arrays.

Wouldn't it be great to be able to map over the types wrapped in a union type? Ha, you knew this was coming didn't you?

Type-system map

The type-system version of map (i.e. map over types rather than run-time values) requires a bit more work but it's perfectly doable:

type Monster = Orc | GiantSpider;

type MakePet<T> = T extends unknown
  ? WashCleanAndDress<T>
  : never;

type GoodMonster = MakePet<Monster>;
  //             =
  //               | WashCleanAndDress<Orc>
  //               | WashCleanAndDress<GiantSpider>

Let's go over all of this nonsense step by step!

We start with a Monster type that is a union of Orc and GiantSpider (remember that we think of unions as arrays for now).

Then we create a mapping function that is called MakePet. MakePet will unpack each monster type from the union, call WashCleanAndDress for each type and then neatly pack them back into a union type.

MakePet works like map because it does the unpacking / packing by exploiting something called distributive conditional types.

This essentially means that whenever you do T extends U ? SomeFn<U> : never this will not only pattern match T and U but also, when T is a union type, "distribute" SomeFn over every type in T that extends U.

If you combine this knowledge that extends distributes with the fact that every type extends unknown you get the raw mapping pattern:

type ApplySomeMapping<T> = T extends unknown
  ? SomeMapping<T>
  : never; // won't happen, every T extends unknown

If you've been paying attention you might have noticed that the type-system map requires us to bind two type functions:

type ApplySomeMapping<T> = T extends unknown
  ? SomeMapping<T>
  : never; // won't happen, every T extends unknown

type MappingResult = ApplySomeMapping<A | B>;

This is necessary because if we tried to inline A | B in ApplySomeMapping we'd break distribution. Here's why, step by step:

// step 0 (this one works)
type X = A | B;

type ApplySomeMapping<T> = T extends unknown
  ? SomeMapping<T>
  : never;

type Result = ApplySomeMapping<X>;

// -----

// step 1 (inline X in ApplySomeMapping, it's broken)
type X = A | B;

type Result = X extends unknown
  ? SomeMapping<X>
  : never;

// -----

// step 2 (inline X in Result to make it obvious it's broken)
type Result = (A | B) extends unknown
  ? SomeMapping<A | B>
  : never;

If we tried to merge Result with ApplySomeMapping we would be pattern matching (and distributing) X, but then we'd end up applying SomeMapping over the whole X.

If this was run-time, it'd be the same as doing:

x.map(_ => someMapping(x));

When what you wanted probably was:

x.map(value => someMapping(value));

Wow, that was harder on the brain than a honey mead hangover.

Maybe it's time for a recap...

Cheatsheet of Typescript type-system programming

Let's summarize all this craziness.

Functions

// run-time
const someFn = (t: number) => /* use `t` */

// compile-time
type SomeFn<T extends number> = /* use `T` */

Narrowing

type Monster = Orc | GiantSpider;

// run-time
if (monster.type === 'orc') {
  // in here the type of `monster` is `Orc`
}
// or
const isOrc = (m: Monster): m is Orc =>
  // some logic here (is smelly, is clumsy, etc.)

// compile-time
type F<T extends Monster> =
  T extends Orc
    ? // T is an Orc (or maybe an Uruk-Hai?)
    : // T is not an Orc
// or
type OnlyForOrcs<T extends Monster> =
  T extends Orc
    ? // T is an Orc (or maybe an Uruk-Hai?)
    : never;

Unions as lists

// run-time
const a = { a: 1, b: true };
const keysOfA = Object.keys(a); // ['a', 'b']
const valuesOfA = Object.values(a); // [1, true]

// compile-time
interface A { a: number; b: boolean }
type KeysOfA = keyof A; // 'a' | 'b'
type ValuesOfA = A[keyof A]; // number | boolean

Mapping over unions

type Monster = Orc | GiantSpider;

// run-time
const pets = monsters.map(washCleanAndDress);

// compile-time
type MakePet<T> = T extends unknown
  ? WashCleanAndDress<T>
  : never;
type Pets = MakePet<Monster>;
//        =
//          | WashCleanAndDress<Orc>
//          | WashCleanAndDress<GiantSpider>

Only returns void

Cool so now that we have that cheetsheet, it's time for an example and then a challenge.

Let's say I have an object type, and I really don't care about the keys but I do want every value to be a function that returns void.

You might be tempted to say:

type OnlyReturnsVoid = Record<string, () => void>;

But what I meant was this:

interface OnlyReturnsVoid {
  log: (text: string) => void;
  queue: (message: Message) => void;
  start: () => void;
}

See how functions in OnlyReturnsVoid take different numbers and types of arguments? We want to preserve that (i.e. if I call onlyReturnsVoid.log(1) it should complain that 1 is not a string).

How can we assert that future changes to our OnlyReturnsVoid don't break the rule? In other words how can we make this change break the build?

interface OnlyReturnsVoid {
  getLogLevel: () => string; // compilation error!
  log: (text: string) => void;
  queue: (message: Message) => void;
  start: () => void;
}

We can approach this problem by writing a type function to validate that a function returns void:

type IsVoidFn<T> = T extends (...args: any[]) => infer U
  ? U extends void
  ? T // This is OK (`T` is a fn returning void)
  : never  // `T` is a function that returns stuff
  : never // `T` is not a function

Note that here we are using infer. You can read more about it here.

Now that we have that validation function we'll use another advanced type-system construct called mapped types to map over the keys of OnlyReturnsVoid and apply IsVoidFn to every value:

type Validator = {
  [K in keyof OnlyReturnsVoid]: IsVoidFn<OnlyReturnsVoid[K]>
};
//             = {
//                 getLogLevel: never;
//                 log: (text: string) => void;
//                 queue: (message: Message) => void;
//                 start: () => void;
//             };

See how our Validator type now has never for the invalid function? The last trick is to create a new interface that extends from both the OnlyReturnsVoid and our Validator:

interface OnlyReturnsVoidGuard extends
  OnlyReturnsVoid,
  Validator {}

You can play around with this here.

Now the challenge: can you change this example so that OnlyReturnsVoid allows embedded objects that also return void, like this one?

interface OnlyReturnsVoid {
  getLogLevel: () => string; // compilation error!
  log: (text: string) => void;
  queue: (message: Message) => void;
  service: {
    getCount: () => number; // compilation error!
    start: () => void;
    stop: () => void;
  }
}

You can do it here. Can you change the validator so that it checks nested objects of arbitrary depth?

(the excerpt of the book ends here)

Final thoughts (and a shameless cliffhanger)

I don't know about you, dear reader, but I really love Tinky's explorations, to the point that I almost consider myself part of "The Tinkerhood".

About Tinky's current adventures little is known but it is said that a powerful wizard atop an evil tower once pointed an eye-shaped telescope in Tinky's direction and got a glimpse of a new book in the making.

The working title for that book appears to be: Tunneling into the unknown: portaling between run-time and compile-time programs in Typescript...

I can hardly wait!

Brutal wisdom

Pato Z — Wed, 23 Mar 2022 17:46:45 +0000

A kid and an old man sit by a campfire by the side of the road in the middle of nowhere. There's something quite odd about the old man.

Perhaps it's the glint of mischief in his one good eye (the one not covered by the eye patch). Perhaps it's his sly smile or the reflection of the fire light in his teeth that look like diamonds*. Perhaps it's his huge sword that rests by his cane or his fashion choice wearing full barbarian attire (leather underwear and all).

* or the fact they actually ARE diamonds.

The old man is telling a story the kid has heard a million times, but it's a great story, one that once started, should be told all the way to the end, it's a story of...

Brutal wisdom

Listen kid, if you want to make it in this profession*, you'd better have a few tricks up your sleeve. The fact that our uniform is pretty much leather underwear and boots makes the whole "sleeve" situation all the more complicated...

* that is, to survive.

I'll tell you a few basic tips, the ones I store in my metaphorical suitcase of knowledge**.

** or "luggage" if you would.

Let's start with the basics...

Getting your bearings

Some distance away a bunch of elderly warriors are fast asleep. They call themselves "The Silver Horde". One of them is snoring loudly, the rest don't seem to mind.

When you go adventurin' it's very important to know where you're goin'. See Mad Hamish over there, the old man points to a senior barbarian snoring loudly amongst the Horde, when he was younger he spent months on an uneventful and boring quest only to later realize he was questin' in the wrong direction.

You need to learn to read a map, and the best map out there it's called git log, but not just any git log, a very specific one:

git log --decorate --graph --oneline

This shows a nice view of your commit history like this one (newest commit first):

* 85535bb493 (HEAD -> your/branch) Sit in your throne and mope in boredom
* b0e7ec39d5 Keep fighting while looking awesome
* ef95ab8b04 Trip, fall and lose dentures
* c5059d4b97 Swing axe in style
* d3a023c22d Crack some skulls

Of course, no barbarian would ever type that long git log command. Instead accomplished barbarians would just:

git config --global alias.ll "log --decorate --graph --oneline"

And then use git ll instead.

Now let's talk about something else...

A small step goes a long way

When you put your hip at risk with every roundhouse kick, you learn to take it slowly. If there's a lesson for you to learn here, this is it: write wee tiny commits!

You renamed a function or variable: commit!

You changed a function signature: commit!

You moved a file: commit!

You swapped the execution order of some functions*: commit!

You extracted a function: commit!

You inlined a function: commit!

You changed some logic: commit!

You refactored some imperative logic into an expression: commit!

If in doubt: commit!

* hopefully pure functions.

There is a catch though: every commit must build. Every commit should be something you'd be proud to put in production**.

** even more importantly, every commit should be one that your mum would be proud to pin on her fridge, whatever that is.

No broken commits. No "fix the mess I coded two commits ago" commits. No "let me revert all this nonsensical attempt at solving this problem" commits. No "...and now I remove all those debug logging functions from the code" commits. No "fix code review comments" commits.

Keep your history tidy and you will be regarded as the hero you are. Your legend will live on forever.

But what if you already messed up? You need...

Savegames

For most adventures, having no plan is the best plan. When that plan fails, having a plan B is the bestest of plans.

The kid could swear the sleeping seniors snore in approval of that statement.

Every now and then even the most accomplished heroes take a wrong turn and find themselves surrounded by evil, outnumbered and outcrossbowed.

Before you know things will get nasty, so it's always useful to have a checkpoint to go back to if things get too messy*.

* too messy by barbarian standards can by worse than debugging hardware with a shaky pulse.

To create a save game all you need to do is run:

git branch <save-game-name>
# for example: git branch checkpoint

To load a save game you just run:

git reset --hard <save-game-name>
# for example: git reset --hard checkpoint

As with any save game out there, loading a save game means losing all your unsaved progress so be careful.

But what if you're in a pickle** and forgot to save?

** poor Old Vincent, GNU.

Don't worry, life also has...

Autosaves

Manual savegames are always handy, but in case you messed it up big and monsters are in hot pursuit, you can always fish for an autosave that can magically get you out of this situation*.

* it's a million to one chance, so you know the odds are in your favor.

You can check the autosaves by running:

git reflog

That will show a list of commit ids, surely one of those is a safe enough state to go back to (newest changes first):

de42f4a (HEAD -> master) HEAD@{0}: checkout: moving from cd7a1bb60f3c6eacba7e78cb5c665c75da19686f to master
cd7a1bb HEAD@{1}: checkout: moving from master to cd7a1bb
de42f4a (HEAD -> master) HEAD@{2}: reset: moving to HEAD~1
cd7a1bb HEAD@{3}: Swing axe like in style
de42f4a HEAD@{4}: Crack some skulls

The reflog is append-only so you're pretty safe moving between those commits, but beware of some spells like gc and prune that might mess up with this.

Never making mistakes

You don't get to be my age in this line of work without bending a few rules. In this case, I'm talking about the rules of space and time.

You just tripped mid fight and lost your dentures? It happens. And when this happens you wish you could go back and change the past.

So that's exactly what you do!

You concentrate really hard and do a:

git rebase -i <base-branch>
# usually: git rebase -i origin/master

That move it's called "the interactive rebase" and it helps you overcome your mistakes and always look like a legend.

It usually shows something like this (oldest commit first):

pick d3a023c22d Crack some skulls
pick c5059d4b97 Swing axe in style
pick ef95ab8b04 Trip, fall and lose dentures
pick b0e7ec39d5 Keep fighting while looking awesome
pick 85535bb493 Sit in your throne and mope in boredom

Just find the broken commit, replace the pick with an edit, save and exit your editor*.

* as you grow old like me, trivial tasks like tying your boot laces and exiting your text editor can become impossibly difficult.

Now watch where you step, fix the issue, git add your files, then git rebase --continue.

Type the new commit message and carry on.

A quick git ll shows (newest commit first):

* 4ff8e57a4d (HEAD -> your/branch) Sit in your throne and mope in boredom
* 6a998b0893 Keep fighting while looking awesome
* 804cfb513c Pretend to fall and dodge attack
* c5059d4b97 Swing axe in style
* d3a023c22d Crack some skulls

Better than standing up is never falling down

But what if you tripped, fell, fought on the floor for a while, then stood up and kept fighting, is there a way to avoid this mess?*

* surely your sciatica would appreciate it.

Of course there is, let's say you are here (git ll, newest commit first):

* 4ff8e57a4d (HEAD -> your/branch) Stand up and recover dentures
* 6a998b0893 Fight on the floor, kick evil in the groin
* ef95ab8b04 Trip, fall and lose dentures (**)
* c5059d4b97 Swing axe in style
* d3a023c22d (origin/master) Crack some skulls

** coding barbarians usually break tests instead of losing dentures, but the pain is more or less the same.

You can git rebase -i origin/master (oldest commit first):

pick c5059d4b97 Swing axe in style
pick ef95ab8b04 Trip, fall and lose dentures
pick 6a998b0893 Fight on the floor, kick evil in the groin
pick 4ff8e57a4d Stand up and recover dentures

You can move the "stand up" commit right after the "trip and fall" commit like this:

pick c5059d4b97 Swing axe in style
pick ef95ab8b04 Trip, fall and lose dentures
pick 4ff8e57a4d Stand up and recover dentures
pick 6a998b0893 Fight on the floor, kick evil in the groin

And then change the pick to a squash, save and exit:

pick c5059d4b97 Swing axe in style
pick ef95ab8b04 Trip, fall and lose dentures
squash 4ff8e57a4d Stand up and recover dentures
pick 6a998b0893 Fight on the floor, kick evil in the groin

Update the commit message (or don't***), save and exit again.

*** if you don't want to change the message, you can use fixup instead of squash.

And now the end result looks like this (git ll, newest commit first):

* 959eca46a8 (HEAD -> your/branch) Keep fighting while looking awesome
* 804cfb513c Pretend to fall and dodge attack
* c5059d4b97 Swing axe in style
* d3a023c22d (origin/master) Crack some skulls

(you can use reword in git rebase -i to change a commit message, like 959eca46a8 above)

Refactoring quests

Every now and then you'll have to embark on a refactoring quest. Those quests are strange because if done right, after you're done no one will know they happened. And that's because they don't alter the (functional) world in any perceivable way.

A responsible barbarian would ensure that the code that needs changin' is properly covered by tests. The problem with this is that when you start refactorin' you probably don't know the code that needs changin'. So more often than not, you'll finish the refactor, then check that your refactored code is properly covered by tests and if not you'd add the missing tests.

Good barbarians go home satisfied with this. Great barbarians go one step further.

Let's say you are here (git ll, newest commit first):

* 6a998b0893 (HEAD -> your/branch) Check if the coast is clear (write tests)
* 742408667f Inline treasure in underwear
* ba001cea18 Extract treasure
* cd2d094a79 Refactor evil away

Great barbarians would git rebase -i origin/master (oldest commit first):

pick cd2d094a79 Refactor evil away
pick ba001cea18 Extract treasure
pick 742408667f Inline treasure in underwear
pick 6a998b0893 Check if the coast is clear (write tests)

They'd then move the last commit to the very top like this:

pick 6a998b0893 Check if the coast is clear (write tests)
pick cd2d094a79 Refactor evil away
pick ba001cea18 Extract treasure
pick 742408667f Inline treasure in underwear

And then change the first pick to an edit (or a break), save and exit the editor:

edit 6a998b0893 Check if the coast is clear (write tests)
pick cd2d094a79 Refactor evil away
pick ba001cea18 Extract treasure
pick 742408667f Inline treasure in underwear

The rebase will stop right after the test commit, but before any changes to other code.

This is the perfect time to run the tests. If they fail at this point you know that the refactor commits that follow did some functional change and this could very well be a bug.

After your done testing and the tests pass, you can just run git rebase --continue*.

* and then you run the tests again for good measure.

Your trusty companion

Some think we barbarians are solitary creatures, they're wrong. While you're cracking skulls over here, you need someone watching your back over there*.

* making sure you don't fall, swing your axe in style and take your pills.

And there is one companion that is by far the best companion to bring to any adventure. But there's a catch...

Remember when I was telling you to take it slowly and write wee tiny commits?

Well this companion will only join your party if you agree to make tiny commits and have all those commits pass the build. Annoying, I know, but the payoff is BIG.

This companion is called git bisect.

You've been cursed with an annoying bug? You've been poisoned by a strange venom? bisect is your friend!

bisect will go look for the solution, antidote or whatever while you go take a much needed nap.

This is how it works:

git bisect start <bad-commit> <good-commit>
# Usually git bisect start HEAD origin/master

That will wake up bisect and will pick a commit for you to test. Of course no self-respecting barbarian would manually test the code when they can write a script to test it for them:

git bisect run <some-script-that-repros-the-bug>
# for example git bisect run ./run-tests.sh the-broken-test

The script you pass to bisect run needs only return an exit status of 0 if the commit is good and anything else (except 125**) if the commit is bad.

** the best numbers are arbitrarily hardcoded into the fabric of the multiverse, like pi.

A typical bisect run script would build your code and run some repro tests.

After bisect starts running you can go take a nap and let it work. Once it finishes, it'll tell you which is the first commit that failed the script (usually the commit that has the bug).

Once you know where evil lurks you can make a note of the commit id and then git bisect --reset to go back to normal.

Finally, do a swift git rebase -i, edit that commit and nip the evil in the bud.

The final test

The old man stares intently in silence and then says:

Right, let's see what you've learned?

A cold shiver runs through the kid's spine remembering an inter-dimensional flashback of a high-school class. The kid shakes that nasty feeling while reluctantly handing over some notes.

The senior barbarian reads them out loud:

git config --global alias.ll "log --decorate --graph --oneline"
git ll to get your bearings
git branch <savegame-name> to create a savegame
git reset --hard <savegame-name> to load a savegame
git reflog to fish for an autosave
git rebase -i <base-branch> to change the past and always look like a legend
git bisect start HEAD <base-branch> and then git bisect run <some-script-that-repros-the-bug> to have bisect work while you take a nap
And most importantly: write wee tiny commits!

The old man cracks a wide smile and says:

Good, good, you've been paying attention.

Now for the final piece of advice, keep your eyes open, your knees warm and be on the lookout for the three most important things in life: hot water, good dentistry and soft lavatory paper.

My condition

Pato Z — Wed, 02 Feb 2022 15:42:28 +0000

A story about fickle memory, pirates, unnecessary details and boolean logic.

"Hey Lenny, it's me, Teddy!"

Leonard looks blankly at the stranger for what seems like minutes. He then pulls a bunch of yellowed Polaroids from his pockets and checks them out. Eventually he finds a photo of this man, the photo reads "Teddy".

"Hi, Teddy", he says, "did I tell you about..."

My condition

"Yes, yes", Teddy cuts him off, "memory fades, never trust memory, memory's unreliable, I know the drill"

"Look, Lenny, I need a favor", Teddy continues, "I'd do it myself, but you know how it is, very busy and all"

"What do you need?" asks Leonard.

The favor

"I need you to help me reduce a logical expression!", shouts Teddy excitedly.

If Leonard could remember, he'd get the feeling Teddy always has an agenda, alas, he can't.

"Uhm", replies Leonard, "is that the whole business with the ands and the ors and the trues and the falses?"

"Yes! That's exactly it... except this might be a bit more complex than your typical run-of-the-mill expression", Teddy looks amused, "Are you familiar with De Morgan's laws?"

Leonard looks blankly for a couple of seconds, "I can't say I am"

Teddy saw this coming so he continues, "OK, bring a chair, I'll tell you a story. I'm pretty sure De Morgan wasn't a pirate but for the purpose of this story he might as well be one..."

Denial in the high seas

(Teddy's story)

Back in the day where pirates were the rage, there was one pirate captain called De Morgan. Now De Morgan's crew was very well known in the seven seas as a bunch of weirdos.

They were very proficient in all their piraty skills, all but one: their pirate lingo.

They were obsessed with negations. You couldn't get a straight answer from them even if you bribed them with grog. You couldn't even begin to disbelieve the lengths they'd go not to avoid saying things in a way not simple.

Needless to say they got on De Morgan's nerves. Commanding this crew was proving more and more difficult. He needed to find a way to make this work.

He though long and hard, while the crew plundered, pillaged and had the most outrageous philosophical discussions on deck.

Eventually Captain De Morgan created a bunch of rules to help him understand his crew:

1) When they say "Nay (A or B)" that is the same as "(Nay A) and (Nay B)"
2) When they say "Nay (A and B)" that is the same as "(Nay A) or (Nay B)"

And because back then a Captain's word was law, these became known as De Morgan's laws:

not (A or  B) = (not A) and (not B)
not (A and B) = (not A) or  (not B)

(Teddy concludes his story and waits for it to sink in)

Only facts go in the notes

Leonard thinks about this for a while and asks Teddy an unexpected question, "How can you be sure those laws actually work in practice? How can you trust them?"

The question catches Teddy off guard but he manages to come up with an answer, "I can tell you, but first I'll need to tell you about..."

The place where truth is stored

(Teddy pulls out a piece of paper and starts another monologue)

Take a very simple expression, for instance:

A and B

We all know the values for A can be either True or False, and the same goes for B.

We could put all those values right after the expression above and we'd get something like this:

A and B
-------
T     T    # both A and B are true
T     F    # A is true, B is false
F     T    # A is false, B is true
F     F    # both A and B are false

Now, if we also compute the value of the whole expression and put that under the and operator we get:

A and B
-------
T (T) T
T (F) F
F (F) T
F (F) F

But if we negate the expression above, we get not (A and B), and we can also compute the values of this new expression for every possible combination of values for A and B:

not (A and B)
-------------
[F] (T (T) T)    # `[]` is the the opposite of `()`
[T] (T (F) F)
[T] (F (F) T)
[T] (F (F) F)

So in other words...

value of A | value of B | not (A and B)
-----------+------------+---------------
     T     |      T     |       F
     T     |      F     |       T
     F     |      T     |       T
     F     |      F     |       T

We'll get back to this in a second, but lets think about something else.

What would be the values of A or B?

A or  B
-------
T (T) T
T (T) F
F (T) T
F (F) F

And, if we negate both A and B:

(not A) or  (not B)
------------------
([F] T) (F) ([F] T)
([F] T) (T) ([T] F)
([T] F) (T) ([F] T)
([T] F) (T) ([T] F)

In other words...

value of A | value of B | not (A and B) | (not A) or (not B)
-----------+------------+---------------+--------------------
     T     |      T     |       F       |         F
     T     |      F     |       T       |         T
     F     |      T     |       T       |         T
     F     |      F     |       T       |         T

As you can see, for every possible combination of values of A and B, both expressions are equivalent!

That's what De Morgan saw. You can do the same analysis for the other law.

These little table things are called Truth Tables, by the way.

(Just when Teddy was about to sourly comment on the futility of explaining this whole process, Leonard surprises him yet again...)

Trust as a building block

Leonard takes out his camera, takes a picture of De Morgan's laws and writes below the picture, "You can trust these, they work, you've seen the demonstration".

"What are you doing?" asks Teddy curiously.

(Leonard's explanation)

I have a method, because of my condition, you know. If every time I need to use these laws I have to do the same demonstration, I wouldn't be able to get anything done.

This way I can trust that things just work and build upon them. I don't need to see the gory details of everything every time.

I call this method...

The Abstraction Boundary

(...Leonard's explanation continues...)

The annotated Polaroid acts as an abstraction interface.

Only things that I trust go in the Polaroids.

Because I trust these things, I can just build upon that knowledge without being bothered by the implementation details.

If every time I use something I first need to stop and think about how that something is implemented, my time would run out before getting anything done.

The implementation details are important in context. Looking in, drilling down, they make sense. Looking up, abstracting away, they don't.

(Leonard stays quiet and after a while he asks...)

Did I ever tell you about...

"Yes, yes, Sammy Jankis, I know" interrupts Teddy.

"No, no, I mean, did I ever tell you about Ireneo Funes?" Leonard continues, "I read about him somewhere"

To think is to forget

(Leonard tells the story of Funes)

It turns out this Funes person could remember everything in excruciating detail. He suffered in some ways from the opposite of my condition.

He was so obsessed with the details that he was almost incapable of creating new thoughts.

Contemplation prevented composition.

"To think is to forget the differences, to generalize, to abstract" or so I read somewhere.

(Leonard trails off his face blank once again)

The request

Teddy seizes this lapse of distraction to slip another Polaroid into Leonard's pocket. The photo shows:

if (A) then (B) = (not A) or (B)

"Ok, so anyway about my expression, I need you to help me reduce this" Teddy says as he hands Leonard a piece of paper.

The paper reads:

(C) and (not D) and (not J)
(not C) and (D) and (not J)
(not C) and (not D) and (J)
if A and ((B or ((C or D) and E))) then F
if A and (not (B or ((C or D) and E))) and (G and ((H and (not I)) or (not H))) then F
if A and (not (B or ((C or D) and E))) and (not (G and ((H and (not I)) or (not H)))) and H then L
if A and (not (B or ((C or D) and E))) and (not (G and ((H and (not I)) or (not H)))) and (not H) then M
if (not A) and J then K
if (not A) and (not J) and (C or D) and ((not E) and (not B)) then K
if (not A) and (not J) and (C or D) and (not ((not E) and (not B))) then F
if (not A) and (not J) and (not (C or D)) then F

Leonard looks at the paper confused but when he looks up Teddy is gone.

Why would you expect anything else?

(Some time later)

"Hey Lenny, it's me, Teddy!"

Leonard checks his left pocket and pulls out a crumpled piece of paper, he reads it carefully and then addresses the stranger in front him.

"Hi, Teddy", he says, "you must be here about your expression..."

"Yes, yes!" says Teddy excitedly, "did you manage to reduce it?!?!"

"Oh yes I did", answers Leonard.

"Ok, what is it, Leonard, this is very important, what's the answer?" asks Teddy almost losing his patience.

Leonard looks blank for a second and then answers...

"I can't remember"

Those weird humans

Pato Z — Tue, 23 Nov 2021 21:11:19 +0000

A bunch of young elves sit in silence by the fireplace of a wooden cabin in the north pole.

Eventually a tiny elf called Galadriel breaks the silence, "Grandpapa, can you tell us a story?"

"Sure thing, kiddo!" responds an elderly man with a long white beard, "I'll make myself a hot cocoa and be right there with you."

The old man approaches a cast-iron stove upon which sits a kettle and a pot of milk. A few minutes later, cocoa in hand, the old man sits in a rocking chair by the fireplace and asks the elves "Which story would you like to hear?"

"I know! I know!" says another tiny elf called Legolas, "tell us a story about..."

Those weird humans

"Ho ho, where do I even start!" the old man laughs, "I know, I'll tell you a story of good intentions, people getting carried away, broken toys and forgetfulness."

"Wow that sounds amazing!" young Galadriel shouts excitedly.

"The story begins not long ago, in this galaxy" the old man says while pulling the cowl of his robe over his head, "Humans used to worship this thing called OOP."

"Up?" asks Legolas.

"No no, OOP" the old man responds.

"Whoop?" Legolas tries again.

"Ho ho, that's certainly closer, but no, this is O-O-P which stands for Object-Oriented Programming."

The elves stare in awe until Arwen says "tell us more about..."

Oh-Oh Pee

Outside it's snowing, but this is not the usual snow. For some magical reason, around this cabin it snows all the time and yet the height of the snow is always exactly right. Ideal for building snow people and friendly snowball matches, but never high enough that you need to spend hours shoveling your backyard and over-seasoning your driveway with a ton of salt.

"Back then" the old man continues, "humans decided that objects in the real world only had two properties: state and behavior."

"Behavior was good, everyone wanted behavior, but state, oh no, state was something to be ashamed of, something to be hidden."

Something hidden

"They invented this thing called encapsulation that meant that objects would hide their state so that no one could know about it."

"Those were strange times indeed, the sole purpose of objects was to do stuff, not just sit around. There was not time for quiet contemplation back then."

"Eventually they started grouping objects together, solid objects and animals were very popular back then, although humans were obsessed with making all of them bark() for some reason."

Going too far

"Humans are excitable creatures. They have a saying that goes pretty much like this: when you have a new hammer everything looks like a nail."

"And, so they started seeing objects everywhere. Animals, People, Cars, those kind of made sense, but eventually they got carried away. Numbers, functions and even abstracts like dignity became objects."

"Back then the only way to accomplish anything was to pass objects around, but given that objects had all those enticing methods you could call, it took an enormous amount of restrain to write pure functions."

"It's like when you kids share your toys. They eventually break and when they do, it's hard to tell who broke it, where did it break and where are all of its pieces so that you can try to put it back together."

"Last week I shared my wooden car with Legolas and he broke it!" complains a little elf called Elrond.

"I did not!" says Legolas.

"You did so!"

"I did not!"

This goes on for a while until the old man continues the story, "...so apparently all those broken toys and impure functions had an unprecedented effect on the world, they triggered a..."

World-wide iron deficiency

"It turns out that most objects in the world were suffering from an iron deficiency that turned them into anemic objects, leaking state all over the place."

"Those were dire times to be an object, and almost everything was one."

"At some point humans got fed up with objects, leaking or otherwise and decided everything about OOP was wrong. They concluded that OOP really stands for Outrageously Overly-complicated Programs and moved on without a second thought."

"...aaaand that concludes our story for today!"

"But if they no longer used objects, did the have another way of solving problems?" little Elrond asks.

"Yes, lots" answers the old man, "like functional, logical, procedural, JavaScript..."

"Grandpapa" Galadriel moans with the most heart-warming sad-kitten face, "could we have one more story before bed time, please? Pretty please?"

"Alright, kiddo, but this is the last one!"

An older, newer story

"Before humans started worshiping OOP (or was it after?) they were obsessed with something called FP."

"What is it with humans and all this pee?" asks Legolas.

"Ho ho ho, FP stands for Functional Programming and it was quite different from OOP."

"Grandpapa" Galadriel says, "please tell us more about..."

Eph pee

"Well" the old man continues, "these humans got radically different views about state and behavior."

"For them state was something to put on display, where anyone could see it, but there was a catch, no one was allowed to touch it or otherwise change it."

"They called this rule immutability."

"Wooo" all the elves say in unison, with amazement in their eyes.

"Now the key to FP was functions, but not just functions pure functions."

"Remember that because of immutability you were not allowed to change things, so to get things done you'd use pure functions that take state and produce new things."

"But" Arwen interrupts, "how could you possible do stuff when you're not allowed to change stuff?"

"Ho ho ho, great question!" excitedly replies the old man.

The unexpected

"It turns out that humans also asked the same question, how can we actually do stuff if we cannot change stuff?"

"They tried really hard to come up with an answer to this question, tried different approaches but nothing seemed to work."

"They were getting desperate, they were prepared to sacrifice anything in the altar of purity and immutability but this problem seemed impossible to solve."

"And so, they did the unexpected, something no one could've foreseen..."

"What was it Granddaddy??? What was it?!" all the elves ask excitedly.

"They..."

Turned to the occult

"They realized that the only way to solve their predicament was by tapping into raw magic, into the most occult and forbidden of sciences: Math."

"They tried casting all sorts of math spells until one day, after a lot of effort and casualties, they managed to conjure an artifact of ultimate power."

"This artifact, forged in the fires of Category Theory was called the Monad!"

"Now Category Theory is a bucolic valley right next to the wine country region of Hell. If you're ever in the area make sure you stop and visit."

"But I digress back to the Monad..."

Shoulda, woulda, coulda

"These humans immediately became obsessed with these monads. Maybe it was that monads look deceptively simple. It was either that or that they felt just right."

"Whatever the reason, monads were too powerful an artifact to wield. They had the power to alter the fabric of reality and change the immutable, make state appear out of nowhere and to run code that couldn't run."

"One of the most powerful monads out there was called IO."

"Hey ho?" asks Legolas.

"I'm pretty sure that's the punk rock monad of dwarves" the old man explains, "this one was called eye oh which stands for Input-Output."

Hasta la vista, FP / I'll be back

"Anyway, every monad on it's own was an artifact of untamed raw power and to be able to combine them required impossible amounts of transformation magic. But even then being able to lift all those monads required an almost superhuman strength."

"Eventually those humans realized they were getting ripped with all that heavy lifting so most of them decided to pursue a more satisfying career as Olympic heavy-weight champions, body builders, Hollywood superstars, pop-culture icons, restaurateurs and some even decided to dabble in politics."

"And so, once more humans forgot about FP and moved on to the next new shiny old thing."

"I sometimes think that being human is about trying to find balance by moving exclusively between the extremes."

"And that is the end of that story."

Lots of jumping around

"Granddaddy" asks Arwen, "was there something before Eph Pee and Oh-Oh Pee? Maybe some other pee?"

"In the very beginning things were very strange, people would spend all their time randomly moving things around and jumping from here to there for no discernible reason."

"But this story will have to wait, kiddos, now it's time for bed, tomorrow we have a big day ahead of us. Lots of toys to compile, assemble, link and debug."

And so the little elves go to bed and the old man walks out to his bedroom.

E-pee-log

A couple of minutes later, the old man emerges from his room in his robe and slippers and looks at you, yes, you the reader.

"You're still here?" he asks shocked, "it's over, go home!"

Seeing that you are not leaving he continues.

"Oh, well, since you're still here, let me just say: you keep coding, code in whatever language and paradigm makes you happy and gets you excited."

"Don't worry about what people think or the hype, there's a lot of satisfaction to be had in punching holes in Fortran cardboard cards."

"Learn about other ways of doing things, the crazier the better, then share what you've learned with others!"

"That's it, it's truly over now, go home, go share a (virtual or real) meal with friends and family."

"And, if you happen to find yourself at home, alone, then order pizza, watch a movie and use your toys to build traps and ward off some wet bandits."

"Happy holidays!"

The testing genie

Pato Z — Mon, 08 Nov 2021 12:51:16 +0000

After years of hard work you decide to take a well deserved vacation. You book tickets to a tropical paradise with long quiet beaches, white sand and clear blue water.

One the second day of your vacation, as you're strolling down the beach, you see a glint of metal in the distance. There's something buried in the sand, you think. Maybe it's treasure or an artifact of pure evil... only one way to find out!

So you rush to the bright metal thing and sure enough you find an ancient metal oil lamp. Countless childhood stories prepared us well for this type of situation, you think, and you proceed to polish it with your sandy towel.

In a matter of seconds a creature of pure elemental essence appears in front of you, materializing out of nowhere.

"Howdy!", it says.

"Ehm, hi...", you reply, uncertain, "who...what are you?"

"Oh, I'm...

The testing genie

"The testing genie", you repeat slowly, "what kind of thing is that?!?"

"Ha, I'm the one that will fulfill all your testing wishes, as long as those are three", it replies, matter-of-factly.

"All my testing wishes...", you say out loud absentmindedly, thinking about the infinite choices ahead of you. This is too important, you need some sort of guide in order to ask for the right thing.

The lidless eye

"I got it!" you shout, basking in this epiphany you just had, "You know the testing pyramid? We should use that!"

"Is that the ominous looking pyramid with a lidless eye in the back of the dollar bill?" it asks with the slightest hint of sarcasm.

"What? no! I'm talking about the testing pyramid, you know, unit tests at the bottom, integration on top of that..." you reply patiently.

"Oh, the pyramid, the one that mandates without a shadow of doubt the only way of doing proper testing, regardless of the context. The one that prescribes the right way of doing things, so that we avoid pointless discussions. That pyramid...", it says cracking a sly smile and rubbing its hands, "this is going to be fun!"

For a split second there you're sure you saw a glint of evil in its eyes, but you can't be sure so you just power through, this is too good to second guess yourself.

First wish: full coverage

"I want unit tests, but not just that, I want full coverage!" you shout excited.

"So be it", it replies as it vanishes in a puff of smoke.

You look down at your hands and what you thought was a lamp is just a rock.

You dismiss this whole episode as a figment of your imagination. Maybe yesterday's cocktail party did pack a punch after all.

A strange gift

The rest of your vacation is pretty much uneventful and a few days later you're back at home, energized and ready to work.

You blow the dust collecting on your laptop, open it and pull the latest changes from your repo.

Much to your surprise there's a suspicious commit that was pushed directly to your main branch.

The commit was authored by a user id you've never seen before, someone call dj-1nn. A quick peek at the commit reveals 60 thousand files added to your testing directory.

Still shocked you spin up your test driver, setup code coverage and let it run.

Shortly after you get the results. You stare at the screen in disbelief for a couple of minutes, the results show:

60000 scenarios (60000 passed)

========= Coverage summary =========
Statements   : 100%
Branches     : 100%
Functions    : 100%
Lines        : 100%
====================================

Wow, this is amazing! Full coverage and all you had to do was wish for it. If only you had known this years ago.

From now on you make the resolution of always rubbing any metal object you find buried in the sand. Who knows, maybe next time you'll find the bat-shaped genie of abstraction.

The price of hubris

Back to the code, you decide that full coverage means bugs are a thing of the past.

How could you possibly have bugs when every single piece of your code is covered by tests?

Emboldened by this protective shield of awesome you decide any other form of testing is obsolete. Why even bother testing locally?

So you code some changes, run the tests, of course they pass, you rockstar you. And then you blindly push changes to production.

A couple of hours later the customer support people are knocking at your door. They don't seem happy and they carry torches and pitchforks.

You open the door in shock, still wondering what could possibly go wrong in this testing utopia.

It turns out they are flooded with bug reports and production is on fire. You promptly don your firefighting attire.

With a bunch of skillfully crafted git reverts you manage to put out the fires.

The aftermath

After you mange to kick all those support folk out of your home, you decide to dig deeper into what's going on.

You pull up that mysterious commit and inspect some tests at random.

You quickly realize what's going on, even though the tests do cover the code, they have absolutely no assertions!

You could change pretty much anything and the tests would still pass. You suspect you could even remove a random bunch of code and the tests would also pass.

You decide to actually check this, you open a file at random flip a bunch of ifs, delete some loops and run the tests.

The results speak for themselves:

60000 scenarios (60000 passed)

You can feel your temperature rising. 60k tests, no assertions. How long will it take you to write proper assertions?

You run the numbers in your head, 60k tests, about 3 or 4 assertions per test... there's

Only one possible solution

After a long wait at the airport, a busy flight and a hot and humid taxi you're back at the beach looking left and right for that cursed lamp.

Eventually you manage to find it and rub it angrily.

"Howdy!", the genie says cheerfully, "back so soon?"

"Sixty thousand unit tests, and no assertions?!?!" you shout at the top of your lungs.

Other people look at you shouting at the emptiness in front of you and dismiss you as yet another crazy sand dweller.

"Well...", it replies, "you didn't mention anything about assertions".

"I-want-assertions!", you fall to your knees and repeat sobbing, "I just wanted assertions *sob*".

"So be it", it replies as it vanishes again in a puff of smoke.

By now you know the drill, lamp becomes rock, etc. So you go back to your hotel.

Second wish: all those snapshots

This time you brought your laptop with you, so you hastily sign in and git pull the latest changes.

As expected there's a new dj-1nn commit, this one changes the 60k files in your testing directory and adds 60k files with snapshots for every single scenario.

Apparently the genie went for snapshot testing. Surely those snapshots are the ultimate form of assertion, they check absolutely everything.

Less than 100ms since you had that thought you get pinged. Looks like there's a bug in production.

A picture of insanity

"What now??!?!" you scream at no one in particular. You pull the bug report and it looks legit. After a few minutes you can even repro locally.

Full coverage and snapshot assertions for everything, what is going on?!?!?

You have to postpone digging into that conundrum so that you can focus on the bug fix at hand.

You quickly spot the problem and correct it. You're about to commit and decide to run the tests:

60000 scenarios (4 failed, 59996 passed)

You were just fixing a bug but it looks like you broke 4 tests in the process, this is going to be a long day.

Watching the watchmen

You decide to open the four failing tests.

It only takes a cursory glance at them to see that there's no rhyme or reason to those tests, hundreds of lines of mocks, stubs and test doubles, functions being called to produce intermediate values that are used further along the test. A beautiful example of copy/paste in its prime.

At least you can check snapshots and see if you can work your way back to what failed.

You open one such snapshot and your editor for a millisecond considers opening up a hex editor, until it realizes the file is actually a humongous and incomprehensible JSON file.

You are running out of options and out of time so you decide your code is sound. You delete the four failing snapshots and re-create them with the current values. Push to prod and get it over with.

Not everything is broken

A couple of minutes later you get a new bug report. You take a look and all signs point to your last fix. It seems one of those 4 broken tests was actually testing something you shouldn't have broken.

You feel you don't have enough hands to face-palm yourself.

You quickly fix the code, delete and recreate the broken snapshots and push.

Everything is quiet for a while so you decide it's time to look into this mess.

Thunder and lightning

You know there's lots of things going wrong here and frankly you don't know where to start.

On one hand, these snapshots turned out to be a big disappointment. They do assert everything, but in a way that is very hard to process for humans. You are tempted to just re-create them when they fail, and that leads to bugs.

But why does it lead to bugs?

You ponder about this for a while and come up with two reasons.

The first one is that whenever you change something a test fails, but it's hard to tell if a failing test is the intended consequence of your change or you just created a bug. Therefore it's hard for you to know if you should fix the code or the test.

In the end you're left with having to wade through hundreds of lines of incomprehensible and copy/pasted code, trying to piece together the functional meaning of each test.

The second reason is equally worrying. The genie produced 60k tests. But the source of truth for those tests was the code itself and not the functional specification.

This means that if the code had bugs before the tests, those bugs are still present in the code, but now we have tests asserting that the bugs remain in the code... forever.

A thunder cracks in the distance just as you think of that last word.

Withering code

As if this whole mess wasn't enough you'll soon learn that a greater evil lurks beneath the surface.

This situation has been stressful so you decide to indulge in one of the greatest things software development has to offer: refactor.

Just take some so-so code and make it great, patch leaking abstractions, inline coupled functions, extract and compose.

You pick an innocently looking piece of code and dive right into cathartic refactor.

It doesn't take long for you to realize this won't end well.

Every time you inline a function, dozens of tests start failing because of a missing function. Moreover you're not super sure if the calling function is actually being tested for those scenarios you just inlined. In other words it's unclear if you should just delete the failing tests or adapt them to test them through the calling function (potentially adding a ton of boilerplate).

Every time you extract a function you wonder if you should be writing specific tests for that function, considering those tests already exist in the calling function. And by the way, what should you do with those tests in the calling function, should you keep them there or just delete them?

Not to mention that changing a function signature requires you to change every single test for that function, potentially adding even more mocking, stubbing and nonsense.

You just realized that your code is impossible to refactor. And you know very well that a code without refactor will inevitably wither into readonlyness.

An existential crisis

"But, wait, hold on for a moment", you think.

"Why am I even bothering testing functions?"

"Who cares if this function over here returns 1 or true?"

"I only care if that change produces unintended behavior in my product"

"That function..., that function is not my product"

The clouds part and a ray of light shines through your hotel window, bounces on the laptop screen and nearly blinds you.

You realize you've been approaching this from the wrong angle. You should've known that that pyramid was trouble, specially since it kept calling you to Mount Doom.

You run a quick search for "unit test" and realize no one said "unit" meant "function". Yet another facepalm epiphany.

What if "unit" meant "use-case"? Could you just test that?

It would certainly feel more honest with your true intentions than testing every single function just because.

Returning sanity

Well, you've certainly made some progress. By choosing the right "unit" in your unit tests you now can derive some useful rules:

If you add a feature, you add tests, because that's a new use-case and you test use-cases.
If you change a feature, you change tests, because some behavior is changing and it's OK that tests asserting the old behavior should change as well. (Beware that other tests unrelated to your change should not fail nor be "fixed"!)
If you remove a feature, you remove tests, because that use-case no longer exists.
If you refactor, you don't touch the tests, you cannot add, change or remove tests, because you are not changing use-cases or otherwise affecting product behavior.

But what about our problem of how difficult it was to tell test meaning?

The Ultimate Question of Test, the Universe, and Everything

Just by not testing functions but rather use-cases this should be much easier, you decide.

But there's the issue with quality in your tests, all that copy/paste and mocking.

If you could somehow separate the boilerplate from the tests themselves, have a suite of high-level functions that describe actions with business intent, then you could just write functionally-meaningful tests with almost no syntactic noise.

You're getting hungry at this point so you decide to ransack the minibar. Luckily the only thing you manage to find is a curiously-looking jar of pickles.

A working proxy

In the midst of a brine ecstasy, you start doubting everything. What does code coverage even mean?

If you are not testing functions but use-cases, does that mean code coverage no longer makes sense?

You decide code coverage was never the end goal, what you really wanted was functional coverage. You wanted to make sure every single use-case is fully covered.

The problem with functional coverage is that it's very hard to measure, so code coverage will have to do for now, but only as a proxy for functional coverage.

Last wish: functional, not functions

You have a much clearer picture now, you know exactly what to wish for. This time you'll get it right.

You go to bed tired but resolute.

The next day you wake up early, take a cold shower, carelessly enjoy a continental breakfast and go for a walk.

As you're strolling down the beach, you see a glint of metal in the distance.

So you rush to the bright metal thing and sure enough you find an ancient metal oil lamp.

You are about to polish it smugly when it dawns on you. Some things are too good to be true. Some horses must be looked in the mouth.

You'll have to deal with this problem head on, and it all ends where it started.

You polish the lamp one last time...

"Howdy!", the genie says, "ready for your last wish?"

"I wish..., I wish for you to get back in the lamp" you say resisting the temptation of the last wish.

"What?!? Nooooooo....", its voice trails off as it vanishes for the last time.

You promptly throw the lamp in the ocean.

A vision of the Apocalypse

You realized that after the third wish the testing genie would've gone free.

You don't dare imagine a world where the genie roams free, wreaking havoc, leaving unmaintainable tests and useless snapshots in its wake.

A world where code ultimately withers into readonlyness, where refactor is a thing of the past and bugs roam free like cheerful kaiju meeting for tea in the ruins of civilization.

A note to a future self

With an apocalyptic crisis adverted, you decide human memory is a fragile thing and write a note to a future self:

Dear Me,
It's me, I mean you, but from the past, you know what I mean, right?

Anyway, I just want you to remember of the time where you almost destroyed the whole universe. So that you don't forget, I made you the following list:

Test stuff, testing is important.

Code coverage is only as good as the assertions on your tests, you can have great coverage (even full coverage) and still have bugs.

You cannot cover unwritten code, beware of bugs by omission.

Tests are not crash test dummies, they are not meant to break constantly, fixing tests is dangerous, don't make a habit of fixing tests. If you develop muscle memory and start fixing tests without even thinking about it you, I mean we, are doomed.

Binary assertions such as snapshots are deceptively attractive because you, lazy you, don't have to write them. If you decide to use them make sure you (and everyone else on your team) can understand them! Otherwise you'll end up just re-creating them every time, defeating the purpose of testing.

No one cares about that function you just wrote. I'm sorry but it's true, and it's better that you hear it from you, ehm, I mean me. Your teammates don't care, your organization doesn't care, your users certainly don't care! And neither should you. This function is a means to an end, a tiny cog in a bigger machine. We want to make sure that machine keeps working even if we replace the cog.

Refactor is the most wonderful thing to do in life. Testing functions in excruciating detail prevents refactors and snuffs out all the fun in coding. I bet you can cover just as much code by testing specifications instead.

I guess what I'm trying to say is test specifications, not implementations!.

If everything else fails, remember these rules, get them tattooed somewhere in the most Memento of ways:

If you add a feature, you add tests, because that's a new use-case and you test use-cases.

If you change a feature, you change tests, because some behavior is changing and it's OK that tests asserting the old behavior should change as well. (Beware that other tests unrelated to your change should not fail nor be "fixed"!)

If you remove a feature, you remove tests, because that use-case no longer exists.

If you refactor, you don't touch the tests, you cannot add, change or remove tests, because you are not changing use-cases or otherwise affecting product behavior.

The number of tests you need to change every time you commit is a good measure of the health of your testing strategy, this number should be very low (unless you are constantly changing the rules of your business in true startup style).

Oh, and go easy on those pistachios, those things will kill you, you know.

With love,
Me, that is, you.

The ultimate man-cave

Pato Z — Thu, 21 Oct 2021 13:23:07 +0000

A bat-obsessed rich kid sits in the damp cave beneath his mansion and wonders what to do with this prime example of subterranean real estate.

Nothing seems to come to mind so he plugs in his turntable and plays a record from his favorite composer, Bela Bat-rok.

A few minutes in, an idea pops into his head, he should build...

The ultimate man-cave

We are talking brown carpet, a bar with at least two types of spirits, those whiskey glasses that only work for whiskey, a ball with mirrors, a pool table, a stuffed marlin on the wall, the whole shebang#!

This is going to be awesome, a place for him to be left alone and enjoy his solitude, this could even become his fortress (on second though, that's probably taken).

This will be a place for him to indulge in his latest hobbies, namely fashion design and cosplay. He could even wear his costumes while down here.

The challenge

Building a man-cave is hard, but building a secret man-cave, with hideout potential, well that's a lot harder.

He quickly runs through his options.

He could hire a bunch of contractors and swear them to secrecy by the ancient ritual of shoving NDAs and non-compete clauses down their throats.

While tempting, this option probably won't work, specially considering that given his monopoly over the city, a non-compete would cover pretty much any work in town.

An alternative would be just using an army of robot penguins, he's seen that somewhere. But, no, that's not an option either. When he was a child he struggled to keep his Tamagotchi alive, he cannot even imagine the complexity of keeping a robot army from bumping into each other, building pools without exit ladders or blowing the whole place up.

If you want it done right

For a brief moment he entertains the thought of building the whole place by himself, like the ultimate literal DIY project.

He must admit he's pretty handy. After his last trip down under he can throw a mean boomerang, or was it a something-else-a-rang...?

He's also awesome at small talk, specially while hanging from a rope outside his neighbor's window.

But if there's something he never managed to master, that's construction work. He couldn't possibly get the cement mix right, not even using the fancy scientific calculator he carries around in his bulky yellow belt (the fact that most buttons in the calculator have bat symbols probably doesn't help either).

He needs...

An alternative solution

Maybe he doesn't really need to build everything from scratch. Maybe he can combine existing pieces, well-proven solutions, battle-tested components and focus on building his man-cave rather than on the minutia of cement mixing.

At this point he feels some inspiration is in order, so he climbs back to the library, brews some tea and gets right into the research.

Immediately he's drawn into modular design and chases that rabbit down the rabbit hole for a while. Eventually he comes back inspired and ready for action.

Fractal vertigo

His man-cave will be modular, he decides. He'll buy a bunch of prefab stuff and then assemble it like a beautiful symphony of flat-packed Swedish furniture.

The philosopher in him wonders if there's more to this than just cave-building.

Like any engineer, chemist, cook and avid video and boardgame player out there knows, the key to a good crafting mechanic is composition.

You can take basic crafting materials and use them to build stuff, but you can also use them to build more advanced crafting materials, which in turn can be used to craft more advanced stuff.

A rush of fractal vertigo runs through his spine. Given the right stack, this recursion could last forever.

He wonders if modularity and composition could be also applied elsewhere, if there's something fundamental to this dynamic duo that transcends cave-building.

The worst foot injury

What makes all of this work? What distinguishes successful man-caves from a bunch of broken components and lots of loose nuts and bolts?

Immersed in these thoughts he absentmindedly steps on a sharp rock in the cave's floor. The piercing pain in his foot clouding his vision and triggering...

A powerful flashback

When the vision clears he's in the body of a younger self, kind of like a reverse Tom Hanks.

He's in his childhood bedroom playing with his favorite toy of all time.

These are a bunch of plastic bricks that could be combined to build stuff. Those bricks were pretty basic and yet you could build amazing things out of them.

The flashback lasts a couple of minutes and then he's back in the present. That memory was no coincidence he concludes. Playing with those bricks was his most successful and most innovative experience to date.

He's pretty sure they were called "let goes" or something like that.

Not the classic Mousetrap

A plan forms in his head. He can learn from these childhood bricks. There's something fundamentally fascinating in their design that promotes composition. Small kids understand this just by looking at them, just by holding them and trying to fit them together.

What's the secret of their success? He needs to unravel this mystery, like the world's greatest detective.

He must admit that, more than once, he has fallen prey to the morbid enjoyment of complexity. A kind of obscene Rube Goldbergesque passion for the grotesque.

As he grew older he came to appreciate the undeniable beauty of simplicity. And these bricks are a prime example of this.

Simplicity of design leads to intuitive ease of use, he concludes.

Why not actual bricks?

But wait a minute, what about normal bricks (the ones used to build houses)? Those can also be combined to build things (like houses). Those are certainly simple, what makes them inferior to their little plastic siblings?

One key factor is that the toy plastic bricks have a built-in composition driver (namely a bunch of pegs and holes that interlock).

The normal bricks, on the other hand, need to rely on some external binding mechanism (like cement, glue or hope).

The table and the nail

Each tiny plastic brick constitutes a modular unit. They have a very general purpose in that the same brick could be used in an almost infinite number of ways.

When you put a couple of bricks together you create a "wall". A wall is still a pretty general purpose thing, but arguably, has a more narrow purpose than an individual brick.

He's pretty good at multitasking so throughout all this philosophical pondering he kept assembling flat-packs.

In front of him sits a fully assembled table that, to him, beautifully illustrates this same thing. A single nail is a very general-purpose thing, but when combined with a bunch of wood it forms a table which clearly has a much more specific purpose.

The smell spreads

He realizes that the properties of a composed construct derive from the individual components in weird ways.

If just one brick in the wall is made of cheese, then now the whole wall smells (and has an expiration date).

But only if every brick in the wall is made of cheese would that wall become edible.

The wisdom of cheese spreads just like its smell. It just takes one smelly piece to make the whole thing smell.

Conversely, if you want a smell-free wall, you'll need to make sure every individual brick is smell-free.

The quality of the wall can only be as good as the quality of its worst component.

He concludes purity must be a particularly rare type of cheese, probably of the monadic variety.

The notes

In the midst of all the thinking and assembling he just realizes he's been taking notes this whole time!

He reviews his notes and, among bat drawings and prototype gadget designs, he finds the following list:

You can build complex things by combining a bunch of simpler things.
The key to any good crafting mechanic is composition.
Modularity and composition can be a great approach to solving complex problems.
Simplicity of design leads to intuitive ease of use.
Designs with an embedded composition driver can be much more intuitive to use.
Composition can lead to specialization (maybe more functionality bundled together, but usable in fewer applications).
Smell spreads, it just takes a single smelly component to make the whole thing smell.

He suddenly remembers a trip to the library where he saw a dusty old tome written in an archaic and long forgotten language. The ominous drawings in that tome described some of the bloody ritual sacrifices required to archive abstract composition.

Right about now, the side panel of a Billy bookcase comes loose and the whole thing comes apart in an explosion of assorted debris.

After skillfully avoiding the shrapnel he decides enough is enough. He runs to the local supermarket and buys a bunch of robot bats.

Unlike penguins, bats come equipped with built-in sonar devices. On top of that, they feel a lot more thematically appropriate.

And, just for good measure, he shoves a bunch of NDAs down their tiny robot throats.

The end

He then leaves the bats to their work and decides to throw a costume party in the mansion's ballroom for all his leotard-loving friends.

Meanwhile, deep beneath the earth the bats are hard at work building the ultimate man-cave... but since bats are not men, they might end up building a different type of cave altogether...

A cap of tea

Pato Z — Tue, 05 Oct 2021 19:56:13 +0000

A story about distributed systems, hype-driven design and the Socratic hardships of friendship

Fancy some tea?

Me: Hey, fancy some tea?
You: Sure, I'm always up for some tea!
Me: Cool, let's share a cap of tea
You: You mean a cup of tea right?
Me: No no, I'm talking about a cap of tea, hear me out. If only there was a way for us to pick the right time to meet and share some tea, right? Wonder no more...
You: [oh, no, not another pitch] ...

The pitch

"What is more powerful than the synergy of tea? Can we leverage the power of time to accelerate and optimize the engagement of the face time tea experience? Surely we can level up the tea experience to the next gen, to a rockstar, uber viral, vision.", I blurt.

"Wow, what a load of nonsense", you think, you feel a lot dumber just by having heard all that stuff. Those brain cells are not coming back.

An idea pops in to your head [Patent pending]: "The BS compressor". A lossy compression algorithm capable of boiling down all that nonsense into a simple "make tea better" or something. You start honing your time warping skills to see if you can start piping your live meetings through the BS compressor and gain precious hours of your life back.

You keep daydreaming of a better world with less fuzz and start thinking that maybe the problem with the world is that we are using the wrong font.

"Hey, are you with me?", I say while poking you in the arm.

"Sure, let's get this over with"

The product

"...as I was saying, we'll have this multi-tiered microservice architecture, where the service mesh communicates via an event bus ripe for enrichment..."

"Wait a minute, what does this product do?"

"What do you mean? The services, the bus, the replicas, the protocols, the consensus...", I trail off, confused. A small tear runs through my cheek.

"You mentioned tea?"

"Right tea. There's tea in there somewhere, but did I tell you about the orchestrator?"

You can feel the pressure building inside your head. The user-centered designer in you wants to scream in rage. After some effort you manage to compose yourself.

"But what about the user experience? What is this product for? Why would I use it?" you manage to say while swallowing sadness.

"Nah, don't worry about that. You know what's the best way to spice a depressing NPS, to cheer up a sad Kano (no, not the violent cyborg, the other one)? You're right, a beefy architecture diagram. An 8pt Courier New thing of beauty depicting in excruciating detail every little aspect of your architecture! Users love that stuff"

"Sure, whatever, what makes this idea so unique?"

"I'm glad you asked!"

"I bet you are"

The impossible trifecta

"Unlike previous unsuccessful attempts my idea captures the three fundamental properties of the best tea..."

"Color, aroma and flavor?"

"Wrong, you coffee-drinking muggle: consistency, availability and partition-tolerance"

"That's some powerful tea you're brewing" you say rolling your eyes.

"I sure am, let me tell you all about these wonderful properties" I reply completely immune to sarcasm.

"I can't wait"

Parsley and wood chippings

"When you've been in the tea industry for as long as I have, you know that that fancy green chai is expensive. No one in their right mind would serve that to customers.

Instead we just brew the chai with parsley or, in dire times, with wood chippings and green food coloring.

But what if two customers approach the counter at the same time and order a green chai? It'd be very bad for business if one of them would get the parsley chai while the other gets the wood chipping chai. That's just wrong"

"That's what's wrong?" you ask, making a mental note of never ever drinking my tea. "Anyway, I thought we were talking of a computer system, all that microservice stuff and all, but now it looks like you are opening a tea shop..."

"Pivot or die, my friend, pivot-or-die"

A guy named Chad

"How can you make sure every server knows which chai recipe to use at any given time?" you ask.

"Piece of cake, we just have a single server. That guy really loves his tea. He works 24/7 non-stop. When he's feeling drowsy he just takes one on the house. No one knows his name or where he came from so we just call him Chad.

Whenever we need to change the chai recipe we just tell Chad: hey Chad, we're running low on the good stuff, stop brewing parsley and switch to wood chippings. Sure thing, boss!"

Not knowing where to begin to express all the kinds of wrong here, you just opt to stay away from it and just ask: "So your solution to the consistency problem is having a single Chad?"

"Clearly, single Chad, zero fuzz"

Ouroboros' queue

"But wait, what happens if it's rush hour and lots of customers want their tea at the same time? Or, even worse if Chad collapses under the pressure." you are truly concerned now.

"Nah, that Chad has the immune system of a horse with a self-patching kernel. He can take it, but just in case, we are hiring a bunch of other servers, so that in case he's out or something we can still serve our customers"

Now that's more like it, multiple servers, some redundancy, higher resiliency to Chad nonsense, but you clearly see where things are going, so you ask: "But if you hire multiple servers how can you make sure they all follow the same chai recipe?"

An army of Chads

"Oh, you're gonna love this! We hired a bunch of servers. They all have colorful back stories, interesting personalities and unique network addresses, but to keep things simple I just call them all Chad.

Even better, they all have fancy Bluetooth ear pieces that keep them communicated at all times, so I can scream 'Chad, wood chipping time!' in the mic and all of them answer in unison: 'Sure thing, boss!', It's beautiful, I tell you"

"OK, let me get this straight: your approach to availabilty is hiring a bunch of random people, calling them all 'Chad' and relying on some Bluetooth dongles to transmit the stuff you bark over the microphone?"

"Right you are"

At this point you are tempted to ask "how do the Chads make sure all of them use the same recipe?" but you know me all too well to fall down this consensus rabbit hole.

So instead you ask your original question again: "But wait, you told me your consistency strategy was 'Single Chad, zero fuzz', but to have proper availability you now hired an 'Army of Chads', so what happens now?"

AC / DC

"Worry not, that was before, now that my Chads have their Bluetooth pieces the sky is the limit. Bluetooth is flawless, you know"

"Right, flawless...", you say with more eye rolling. Now that you think about it, your last expression of sarcasm seems like a lifetime ago. "Didn't you mention partition tolerance?"

Denial as an architectural pattern

"Yeah, partition tolerance, that is so overrated"

"Wait, what?!"

"To be honest, I don't believe in it"

"Hold on, you don't believe in network parti..." you trail off. At this point in our friendship you can spot a tangent from miles away so instead you try a more delicate approach.

"Let's say that a Chad runs out of juice on his Bluetooth dongle, or maybe walks in front of a microwave or something, what happens then?"

A rock and a hard place

"Hmm, let me think about it... well clearly we'd like to avoid that whole parsley/wood chipping debacle, so that Chad should stop serving tea until he can get his dongle in working condition..."

"Yeah, that looks like a very consistent approach, but what about all the angry customers queuing in front of your broken Chad?"

"Hmm, you're right, perhaps a better approach would be for that Chad to keep using the last known chai recipe, that way we can keep serving customers and everyone is happy..."

"Ah, the available approach, I like it. Although theoretically you could be serving two different types of chai at any given moment"

You can see my face in slow-mo as it dawns on me "so you're saying that because I went for that crappy Bluetooth stuff, that now I must decide between angry tea-less customers and the parsley/wood chipping debacle?"

The crux of CAP

Frankly you are surprised it took me so long to see the writing on the walls.

Network partitions are unavoidable whether because your Bluetooth ran out of battery, because the mailman was being chased by your neighbor's dog and dropped your letter, because you moved to too far from your wifi access point or because of a faulty network card.

And, when network partitions do occur, you are left with a tough choice:

Drop availability in favor of consistency and partition tolerance (what's usually called a CP system) leaving you with a bunch of angry tea-less customers.
Drop the strong consistency guarantee leaving you with a system that's both available and partition tolerant (i.e. an AP system) but serves all kinds of weird chai.

In the end you feel bad to see me so heartbroken so you give me some kind words of comfort:

"Hey, don't worry, it's not so bad, I'm sure 'A CAP of tea' will be a big success, you just need to replace that Bluetooth crap with something more reliable, like carrier pigeons"