Forem: jocrah

Debugging as a Controlled Experiment

jocrah — Sat, 03 Dec 2022 07:16:50 +0000

What possibly changed?

As with almost everything in life, there is the inherent need to know which factors influenced a specific outcome, good or bad. It is part of how we sometimes find out which foods and actions the body does not react well to. A similar concept applies to how we learn to relate well with friends and people from diverse backgrounds.

When there are too many varying factors, it becomes difficult to find out which particular one is causing a good or adverse effect. It could be one of them, a combination of them, all of them, or none of them. Do you catch my drift? It can be quite stressful when trying to figure out the cause of something among a thousand possible factors.

All we simply want to do is understand what is going on.

Being the second write-up of The Human Aspect of Debugging series, this focuses on some tips I have learnt and picked up when trying to nail down the cause of a code malfunction.

Victory. Defeat. Victory.

During the 18th century, when Britain and Spain were at war, a British commander, Commodore George Anson, led a squadron of eight ships on a mission. Their mission was to disrupt and capture the Spanish Empire's possessions, and it was fairly considered a success^[1].

In an unexpected turn of events, only 188 men out of the original 1,854 were recorded as having returned from the voyage. This was not due to losing men to an ambush by enemies. Instead, most of them were alleged to have died from a disease known as scurvy , now known to be caused by a lack of vitamin C^[2]. At the time, the concept of vitamins was unknown.

In an attempt to find out the cause of the strange fatal illness, James Lind, a Scottish doctor, performed one of the first recorded controlled experiments. He focused on the diet of British soldiers and divided a sample number of them into 6 groups. He then gave them the same diet but made a unique addition to each group's diet. Cider for one group, oranges for the next and so on. After months of observation, it became clear citrus fruits made all the difference.

All other things being equal

As observed in Lind's approach, the concept of a controlled experiment is aimed at minimising the effects of variables other than the independent variable to increase the reliability of the results.

A very helpful thing I've found to do when debugging is narrow down the surface area. This means that to discover the cause of a bug, one should try not to have too many unknowns. It leads to a larger area for the bug to hide.

This concept plays a major role in reproducing bugs. In James Lind's case, among many other factors, his aim was to try to pinpoint the deciding factor for preventing the disease. In the same vein, going back to the bug story shared in the previous section, I had to test a scenario where:

the Shopify site had only our app installed. Outcome: Widgets showed perfectly.
the other app was installed before my app. Outcome: Widgets did not show.
my app was installed before the other app. Outcome: Widgets showed perfectly.

That said, establishing a pattern that results in a bug does not guarantee a fix to the bug but is very essential. Once a bug is reproducible, the chances of fixing it increase enormously.

Instead of changing a lot of things at a go, it is helpful to make small changes while observing the behaviour. It is worth noting that, applying the concept of a controlled experiment means different things in different use cases.

Sometimes, bugs happen in the wild. That is, they can happen in environments which might be outside the reach of very helpful debugging tools. This can be a great source of frustration since there's not much to go on. In such cases, building on little clues and making little changes in the available environment and observing step-by-step helps to build up confidence and understanding of what the focus should be on.

Additionally, a few generic approaches I use are outlined below when it comes to debugging written code.

Commenting out parts of the code

When there is a lot that has changed about the code and there is no idea what could possibly be wrong, gradually commenting out bits and pieces of the code while testing the outcome can be vital. It helps to incrementally grow into understanding what might be the cause and what is surely not the cause.

Alternating between original and updated code

I have had experiences where a related existing feature just stopped working when new code changes were introduced. In such cases, it is only natural to focus on the new changes to spot what introduced the issue. However, the longer we spend investigating and not finding the reason, the more we are likely to get frustrated and impatient. It could be so elusive that we start believing it was already an existing bug. Surely, that is a possibility and should not be ruled out.

A handy way of confirming or eliminating such a possibility is to revert to the main version of the code and test out the same scenario.

The benefit of this is that, if the feature works well for the original code version for the same scenario, we are motivated to look at the new changes a bit more critically.

On the other hand, if the bug is found to have existed before the code updates. In a fair number of cases, bug investigations have eventually ended up revealing deep-rooted long existing issues.

And that's a wrap

In this write-up, we learn that, when debugging, making small incremental changes and observing behaviour could be helpful in figuring out what might be causing a bug. It also helps to improve understanding of what some sections of the code are really doing.

Thanks for reading! Any thoughts and further experience on this are highly welcome.

Wikimedia Foundation. (2022, November 2). George Anson's voyage around the world. Wikipedia. Retrieved December 1, 2022, from https://en.wikipedia.org/wiki/George\_Anson%27s\_voyage\_around\_the\_world
James Lind - Wikipedia. (2022). Retrieved 5 July 2022, from https://en.wikipedia.org/wiki/James\_Lind#Prevention\_and\_cure\_of\_scurvy

Debugging: Navigating Hunches and Assumptions

jocrah — Mon, 21 Nov 2022 18:37:28 +0000

The universe's plot

I used to banter with a friend whenever they had a code issue and were on the verge of giving up. Feeling worn out and discouraged, they would describe the problem while implying that they had done everything right but the code was still misbehaving. After listening to their plight, one of the things I would jokingly say is,

Oh, that is unfortunate. Let's file an issue with the Node.js team since it must be from their end 😁.

We would laugh it off and soon enough, they would end up finding out what was missing from their code. Spoiler. It was not the runtime (yet anyway).

Times like these arise for many developers. Times when we are so sure that everything about our code should work perfectly. In fact, during those times, we are almost ready to go out on a limb for our code. We insist we have done everything right and that it is perhaps the programming language or framework or the universe at large conspiring to hold us back from greatness. (It sometimes is the language or framework indeed. Don't rule it out simply because I implied so 😊.)

There are many helpful tools for stepping through code to help know what might be causing a malfunction. However, in this debugging series, our focus will be on behavioural approaches that could help in finding the cause of bugs more efficiently. This in turn will help reduce the usual frustration that comes with debugging.

This article addresses how to conveniently deal with hunches and assumptions when something isn't working right with our code logic.

How it goes

Sometimes, when our code is not working, we naturally have certain spots in the code we are already not confident about. As a result, they are unofficially ranked highest on the suspects' list and are the starting points.

In a similar vein, other times, it is actually because we are most certain about specific parts of the code such that we rule them out altogether and probe more into any other part whether related or not.

Having a hunch or assumption about the cause of a malfunction is inevitable. They just show up. This could be due to prior experience or sometimes due to a much less logical reason. However, if it turns out well, it cuts the whole bug fanfare short.

Is this a good or bad thing? Let's see.

Two sides of the coin

When it comes to using a hunch or assumption in debugging, a case can be made that, exploring the wrong place for the cause of a bug is technically not a waste of time. This is because finding the cause of the bug is as essential as finding what is not the cause. It helps to clear stuff off the suspect list to help zero in on the cause. However, there are times we are so convinced there is possibly no other cause even when a hunch or assumption is yielding nothing. Debugging based on a random hunch with no basis makes one spend time and energy chasing shadows.

Debugging this way leads to exhaustion and loss of desire to continue. This is not to say finding the cause of a bug should necessarily be a quick affair. The main aim here is to increase the chances that we are focusing our time and energy on the right things.

When it comes to what we think might be causing a bug, it requires more substance. Our suspicions need to be supported by initial facts and confirmation from the source code or documentation.

Exhibit A: A hunch borne out of earlier curiosity

A couple of years ago, I was part of a team in charge of designing widgets for websites. This included integrating with sites on eCommerce platforms like Shopify, Wix and Bigcommerce. We had successfully launched and our widgets were functioning properly across all platforms. All except one Shopify website. The website had 'decided' not to show the widgets at all. What made it even more challenging was that every other Shopify site was playing nice so there was no other site to draw a pattern from.

Stuck with the issue for a few days, I remembered something I had been curious about some weeks before and had a strange feeling it might be playing a role. It was about how code bundled by Webpack had a global variable, webpackJsonP, used for loading other scripts (chunks) of the code when needed. These chunks are the result of a concept called code-splitting which splits code into smaller different parts based on the code structure. It helps to load only the needed CSS and/or Javascript on a site instead of loading an enormous amount of code that might not be required at the time.

Not too much information, I hope 😅. Let's proceed then.

At the time, I remember being curious about what would happen to the webpackJsonP variable if two separate app codes bundled by Webpack were on the same website. I had no idea that I would learn the answer to that the hard way a few weeks later.

Following up on the hunch, I found out another third-party app installed on the site had also Webpack for bundling their code. Due to this, the app had first rights to the global webpackJsonP variable because it had been installed before ours. As a result, it was loading the scripts of the other app and not ours. Our app had no chance.

The fix was to configure Webpack to customize the name of the global variable^[1] when bundling our app code and voilà!

Working with hunches the right way

Given that they can be unintentional, there is no correct method to have an assumption or hunch in and of itself. However, acting on an informed one can go a long way.

An essential initial approach to validating hunches and assumptions is to try to reproduce the bug with the theory and information at hand. In my case, to confirm my initial hunch that another app was playing a role in the widget malfunction, this is what I tried: I created a sample Shopify site and installed our app. Widgets appeared and worked flawlessly. Next, I uninstalled our app, installed the other app in question, and then reinstalled our app. Result? The network requests would not even show any javascript being loaded for our app. Hunch confirmed. Note that this did not mean a fix had been figured out but was a giant step in the right direction. This ties into the next article in this series we will be talking about.

Additionally, one should infer from the source code itself, NOT a memory of the code, even if one is the sole programmer on the project in question. Inferences can also be made from past experiences or something that caught one's attention while testing something unrelated.

In conclusion, getting a logical and factual basis for any hunch or assumption about what could be causing a code malfunction helps to meet fewer dead ends and reduces frustration.

Thanks for reading! Any thoughts and further experience on this are highly welcome.

This is no longer an issue in Webpack 5 onwards.