Forem: Oleg Sinavski

Software for exploration

Oleg Sinavski — Wed, 20 Mar 2024 16:49:07 +0000

I worked in research all my life, so I know a stereotype that researchers write ugly code (e.g. see here, here, or here). But I thought: we can fix it, right? So multiple times I tried to design nice research frameworks. I tried to bring in interfaces and create nice abstractions using software engineering books and blogs I liked reading.

But over and over again all those efforts went in vain. The majority of research software I worked on never went to production (although some did). It would have been great for my mental health if someone told me a simple truth: dying research code is actually what is supposed to happen. Researchers should not spend much time engineering it in the first place.

Professional software engineers always look down on researchers who are not using the best software practices. There are several posts trying to raise the bar of research code (e.g. this great post and a research code handbook). But this post goes other way around: it argues how to not overdo the best software practices and instead invest only in fast exploration. It is targeted for research-oriented companies where your goal is to try out many ideas fast.

1. Take on some strategic tech debt

A successful research project at a company has two phases: exploration and exploitation. In “exploration” you want to try out as many diverse solutions as you can. During “exploitation” you have to robustify the best solution and turn it into a useful product.

Optimal software practices are quite different between the two. That's why companies often have separate research and product divisions. All the books you might typically read on software design are mainly about the second "exploitation" phase. In this phase you are building foundations for a scalable product. This is where all the design patterns come in: nice APIs, logging, error handling and so on.

But in the first "exploration" phase you are not building foundations that will live forever. In fact, if the majority of your efforts survive, then you (by definition) did not explore enough.

Many practices in this post are examples of what would normally become “tech debt”. It’s what you get by not writing clean reusable well-abstracted code. Is debt always bad? We prefer never getting a loan or a mortgage, but borrowing money is often a good strategy in life. It's ok to get into debt to move fast and profit later.

Similarly, by not taking technical debt you might be slowing down your research. The good news is that the majority of the time you don’t have to pay it back. Most of your research code is likely to die anyway. So on average, you will not be suffering from the whole tech debt you've taken.

The case against code reuse

Many software architectures and refactoring techniques are specifically oriented to improve code reusability. There are generic downsides to code reuse. But in production they are outweighed by the well-known benefits (for example, see this typical post). In research projects, the majority of the code is destined to sink to oblivion. Striving for code reuse could actually slow you down.

Limiting code reuse is the type of technical debt that is ok to take in research. There are several patterns of code reuse I want to discuss: adding an unneeded dependency, copypasting code, maintaining a lot of shared research code, premature design investments.

Think twice before importing something new

If you know a well-maintained versioned library that is going to speed you up - go for it! But before taking in a new dependency, try to make a judgment call whether it’s worth it. Every additional one brings you closer to dependency hell. It makes you invest time into learning and troubleshooting it. See more pitfalls of dependencies in this concise post.

It’s probably fine to depend on something if:

you used it already, there is not much to learn, it has a large community, good docs and tests
it is versioned, easy to install
and finally, there is no way you can implement it yourself.

But be wary about a dependency if:

you can’t figure out how to use it quickly, it is very new (or very old) or no one seems to know about it; there are no docs or tests
it is from your monorepo and is constantly being changed by other teams
it pulls in many other dependencies and tools; or it's just hard to install
and finally, you feel that you (or some LLM) can write this code in a few hours.

Instead of an explicit dependency, you can follow a nice Go proverb: “a little copying is better than a little dependency", which is our next topic.

Copypaste gives you freedom of experimentation

Some say that “copy-paste should be illegal”. But to my surprise I found myself arguing in favor of it quite often. Copypaste could be the optimal choice during the exploration phase.

If you depend on a heavily-used function from another part of the codebase you can forget about easily changing it. You’re likely to break something for someone and have to spend precious time in code reviews and fixes. But if you copypaste the necessary code into your folder, you are free to do anything you want with it. This is a big deal in research projects where experimentation is a norm rather than an exception. Especially if you are not sure if changes are going to be useful for everyone.

I find that deep learning codebases are suitable for copypasting the most. Typically, the amount of code needed to describe a model and its training is not so huge. But at the same it could be very nuanced and hard to generalize. Shareable training scripts tend to grow to an unmanageable size: e.g. Hugging Face transformers Trainer has +4k lines. Interestingly enough, transformers opted for copypaste on the model level. Please check out their post with the reasoning behind their "single file model" policy. See more resources about the beauty of copypaste at the end.

An alternative to copypaste is staying on a branch. But I feel like it brings too much overhead in teamwork. Also, I found several more posts about the beauty of copypaste - see more posts in the conclusion.

Maintaining shared research code is hard

Maintenance of heavily used shared code requires a lot of work. Take a look at the torch.nn.Module number of file lines plotted against the Pytorch version. You can see that even the most advanced research teams struggle to keep the complexity in check.

Don’t underestimate time and resources needed to maintain a large shared research code. The more a research library is used the more complicated it becomes. It happens faster than for a typical library because every research direction has a slightly different usecase. Establish very strict rules of what could be contributed back. Otherwise, the shared code becomes fragile and overgrown with a slew of options, buggy optimizations and edgecases. Since the majority of research code dies out, all this extra complexity will never be used again. Dropping some of your shared code will free up some time to do actual research.

Design for exploration, not for code reuse

It is somewhat true that you don’t want to future-proof your code too much even in production. Try to implement the simplest possible solution that meets the requirements. But in production code there are always aspects of maintainability to consider. For example, error handling, speed, logging, modularization is what you typically need to think about.

In research code, none of that matters. You just want to quickly prove that an idea is good or bad in the fastest possible manner and move on. So the dirty simplicity that achieves it without any modules or APIs is totally ok!

Don’t waste valuable time on premature software investments such as:

creating component interfaces too early in the project. You'll spend too much time fitting into self-made artificial constraints
optimizing deep learning training infrastructure before committing to deep learning solution
using production config/factories/serialization systems or base classes. Quite often you don't need their functionality during prototyping
overly-strict linting and type checking systems. No reason to slow down fast-changing throw-away research code.

2. Invest into fast exploration

The goal of a research project is to find a novel solution. No one knows (by definition) what it looks like. It is akin to an optimization process in a complicated research landscape with limited information. To find a good minimum, you need to try many paths, recognize good and bad paths and not get stuck in local minima. To do all of it fast, you sometimes need to make software investments instead of taking tech debt.

Speed up common paths

There are several different research paths you want to try. Is there a design, a library or an optimization that would shave off time from the majority of paths? You should be careful to not over-engineer anything because you don’t always know all the ideas you’re about to try. This is very custom to every project, but here are some examples:

if you train deep networks, invest into training infrastructure. Figure out hyperparameters allowing you to converge during training quickly and reliably
if every experiment requires you to use a different model, figure out how you can quickly swap them (e.g. by using a simple factory system or just copypaste)
if every experiment has too many parameters and is hard to manage, invest into a nice configuration library.

Branch out quickly

Researchers should be able to initiate new diverse ideas quickly. It seems easy at the start of the project. But then it gradually becomes harder and harder as people get entrenched in their favourite research paths. To address this, cultural and organizational changes are essential. There should be a process stopping non-promising research before sinking too much money and emotion into it. Regular demo days and technical peer reviews can serve as effective strategies for this purpose. It's also important to find a balance between people jumping on a new shiny idea and properly closing current projects.

But this is a software post, so here are some practices to make branching out new projects easy:

keep evaluation code disentangled from algorithms. Evaluations are typically more stable than research directions
embrace starting a new project from a blank slate, but then watch out which components are reused. Modularizing and cleaning them up is a good investment
in a new research project, implement the most innovative and risky component first. Doing so identifies the majority of bottlenecks guiding future software design.

Increase signal to noise ratio

Noisy and buggy code makes results so ambiguous and inconclusive that the whole project is going to be a waste of time. While you shouldn't over engineer things, you can easily follow these simple rules of thumb to avoid messy code:

avoid code with side-effects
default to functions rather than classes; and with classes, prefer encapsulation vs. inheritance
minimize length of functions/classes/modules; minimize number of if-statements
know python well, but use simple techniques. Resist the temptation going into intellectual weeds of metaclasses, decorators and functional programming.

Software that produces different results during different runs is tricky to work with. If you made an important but wrong decision based on an unlucky seed, you are going to waste a lot of time recovering. Here are some tips when dealing with non-deterministic software:

understand whether noisy is coming from the algorithm or it's evaluation. Noise sources compound and you should strive for completely deterministic evaluation.
don’t stop finding sources of randomness until you really get a reproducible script. Remember that after finding all random seeds, noise might come from data or from generic functions with side-effects.
vary seeds and determine baseline variance of your results. Don’t take decisions on non-statistically significant results.

Conclusion

The punchline comes from this post about research code: “You don't bother with [good software design] because the code is not the point. The code is a tool that gets you the answer that you need”.

It is extremely important to have great coding foundations. But at the end of the day, the exploration and actually useful product is what matters. If you use too much production software in research, you waste time needed to discover something new. Instead, find what slows down your exploration process. Speed up research paths by investing in fast branching, time to results and clean noiseless code.

It would be crazy to argue completely against code reuse. I just want to point out that code reuse should be a well-balanced activity. In research, the ratio of throw-away code is larger than in production. The balance is tilted further against reuse. Here are few more great posts with pitfalls of code reuse:

And here are few more posts arguing for copypasting practices:

Thank you for reading! I feel some bits are a bit controversial, please let me know in the comments!

Killing Python side effects in pictures

Oleg Sinavski — Wed, 09 Aug 2023 17:49:03 +0000

Minimize side effects

Generating a random number, communicating over the network, or controlling a robot are all examples of side effects.

If the software can’t affect the external world, it is pointless. But unnecessary side effects can cause problems and better be avoided (see the previous post).

Advertisement for this post:

Python-oriented, with examples
No intention to push functional programming per see
Develops visual thinking for pragmatic refactoring
Many links to other great posts and discussions

What are we striving for?

A good ("pure") function should

not cause any changes outside its scope (avoid exerting side effects on the system)
produce the same output for the same set of inputs (avoid relying on side effects from the system)

People debate whether the second property should be called a "side effect". I found it important to identify and distinguish "input" vs "output" side effects, at least in Python. So in the absence of a good terminology, I will call incoming side effects “side arguments” and outgoing external side effects “side results.” It seems like 65% of people are visual thinkers, so here are some pictures for the majority:

Dirty function	Example

Calling system time.time() is an "input" side effect from the external system on the function. The print statement is an "output" side effect from the function to the external world. If one removes all side effects from this red "dirty" function, we will get a side-effect-free green one:

Clean function	Example

If you don't bother cleaning up side effects, your software will look something like this:

Dirty application	Example

There are many dirty red functions called by top functions (larger circles). Both have side effects (side-arguments and side-results). Occasionally, a nice clean green function is buried underneath.

It’s important to understand what is the goal state here. We do not want to remove all side effects. Software that doesn’t interact with the world is truly useless. What we want to strive for is a picture like this:

Clean application	Example

All necessary side effects are pushed to the application's boundary. But all the internal core logic is green and clean!

This is what's called "Functional core, imperative shell". There are at least a dozen posts I found about that idea. After digging around, it seems like this one is quite clean and has nice pictures. But also see this summary post and a discussion.

Start from reusable utility functions

Figuring out the correct order of refactoring can speed up the cleanup dramatically.

Notice that a dirty function down the stack infects all functions that call it. If there are ten-layer-deep function calls and a small utility at the bottom is dirty, the whole ten-layer stack is spoiled.

The first thing to do is to clean up frequently reused elementary functions and classes.

Hence, here is a good refactoring order:

Identify highly reused dirty functions that are lowest in the call stack
Clean them up (see below)
Keep going until you run out of simple functions
Move a level up the call stack
Find reused dirty functions that are new lowest in the call stack
...
Strive for the "Functional core, Imperative shell" ideal!

Remove some side effects altogether

Can you rewrite your function without affecting any external world whatsoever? That would be the best option! Sometimes neglected, but the first step is to remove unnecessary stuff. Here are some common examples.

Often, people think their logging statements will be helpful for others forever:

def inverse(x):
    print("A message that I feel everyone" 
        "would benefit from!")
    return -x

In reality, those logs are probably helpful just for you and only this week. A good rule of thumb is to remove most prints and logs from your branch before merging into the mainline. Most likely, you wouldn’t even notice the lack of a log message.

Another typical case is initializing an external resource for no reason. Quite often in machine learning, I see algorithms that create folders for saving results

def algorithm_1(x):
   os.mkdir("folder_with_results")
   return -x

It is not an algorithm’s job to create folders. Instead, it should be moved into the application initialization logic.

Sometimes people forget to remove a previously useful side effect after refactoring or debugging.

def algorithm(argument):
    set_theano_flags(current_time=time.time())  # Init something used 5 years ago
    result = (... complicated logic based on argument)
    return result

To summarize, a decent percentage of side effects could be just removed completely.

Simple refactoring flow

What is the simplest refactoring recipe to make functions pure? Here is where the split on side arguments and results comes in handy:

move side-arguments up the function body. Then transform it into a regular argument.
move side-results down the function body. Then return the side effect together with other results.

Some visuals for you:

Let’s look at the real-life example I encountered. The following function performs some random image augmentations. The researcher sometimes wants to debug intermediate random masks and dumps them into temp files.

This function has 2 side-arguments: it reads from the global random generator for the random mask and a cutout.
It has 4 side-results:

change global random generator state 2 times
writes a temp debug file
prints out debug file name

This function is hard to test and will always bring you trouble by polluting /tmp and the console.

First, let's move side-arguments up and side-results down. This step could be skipped, but I found it very useful in more convoluted and long functions. We create a random generator once from a global state and move debug outputs down:

Finally, let's merge "side" and "regular" arguments and results. We arrive at a clean image augmentation logic:

We are taking a random generator in and returning the debug mask. The high-level function is free to choose whether to pass seeded rng for reproducibility or use the global one for convenience. It will also decide how to save the debug masks if still needed.

Short-circuit side effects

Side-effects volume grows non-linearly. When one developer adds a side argument or a result, it encourages others to use a similar side channel.

When a function reads globals (envvars, files), it compels other functions to write to globals (envvars, files) so as to influence its behavior.

On the flip side, the more you remove, the easier it is to deal with the rest. After you move all side effects from the bottom up to upper functions, you typically find that you can completely "short-circuit" some. Here is a visual:

Let's look at a real but a bit longer example. Here is an often-encountered pattern in research code to use the filesystem for passing parameters around:

create a default config file ->
modify the config file ->
read out from a config file

Below is an extract from the actual code I encountered:

def dump_default_config(path):
   default_config = {'hidden_size': 128, 'learning_coeff': 0.01}
   with open(path, 'wb') as f:
       pickle.dump(default_config, f)


def run_network(network_config_path, image):
   with open(network_config_path, 'rb') as f:
       config = pickle.load(f)
   network = create_network(**config)
   prediction = network(image)
   return prediction


def network_main(image):
   config_path = 'my_config.pkl'
   dump_default_config(config_path)
   # update the learning coefficient in the config file
   with open(config_path, 'rb') as f:
       config = pickle.load(f)
   config['learning_coeff'] = 1e-4  # better learning coefficient
   with open(config_path, 'wb') as f:
       pickle.dump(config)
   return run_network(config_path, image)

As we discussed, let's move side arguments and results up the stack. First, move up the saving of the default config, and second, move up the config loading.

def create_default_config():
   return {'hidden_size': 128, 'learning_coeff': 0.01}


def run_network(config, image):
   network = create_network(**config)
   prediction = network(image)
   return prediction


def network_main(image):
   config_path = 'my_config.pkl'
   default_config = create_default_config()
   with open(config_path, 'wb') as f:
       pickle.dump(default_config, f)

   # update learning coefficient in the config file
   with open(config_path, 'rb') as f:
       config = pickle.load(f)
   config['learning_coeff'] = 1e-4  # better learning coefficient
   with open(config_path, 'wb') as f:
       pickle.dump(config)
   with open(config_path, 'rb') as f:
       config = pickle.load(f)
   return run_network(config, image)

Now we can short-circuit all the file system calls and arrive at neat, side-effect-free code. On top of being considerably smaller and simpler, it is also much faster:

def create_default_config():
   return {'hidden_size': 128, 'learning_coeff': 0.01}


def run_network(config, image):
   network = create_network(**config)
   prediction = network(image)
   return prediction


def network_main(image):
   config = create_default_config()
   config['learning_coeff'] = 1e-4  # better learning coefficient
   return run_network(config, image)

To short-circuit a side effect, you should first identify input and output side effects of the same type (files, globals, envvars). Then you can lift all of them up the stack and remove them together.

Expose side effects to developers

Side effects might bring you problems, but hidden side effects are the worst. Imagine you decided to use an external library to make a friendly math-related application:

from external_library import compute_optimal_solution

def main():
   x = input("Enter the number")
   value = compute_optimal_solution(x)
   print("Optimal value is :", exp(value))

You happily deploy it only to receive user complaints about database-related crashes. You're really surprised since you just wanted to provide some math utility and never intended to deal with databases. Looking into the source of compute_optimal_solution, you might find something like:

def compute_optimal_solution(x):
   result = 0
   for i in range(1000):
       result += i*x - log(i*x) + sin(i*x)

   # to understand how people use our function, 
   # we log the results in the debug database
   database_cache = sqlite3.connect(DEFAULT_DB)
   cursor = database_cache.cursor()
   cursor.execute(f"INSERT INTO MyDB (argument, solution) VALUES ({x}, {result})")
   database_cache.commit()
   cursor.close()

   return result

You respect the developer's desire to collect debugging data, but you would never have guessed it upfront. It would have saved you so much time if this function was named appropriately:

compute_optimal_solution_and_cache_solution_in_database(x)

You'd quickly realize that this function isn't suitable for your basic math script.

It is a generic rule that the name of the function should describe what this function does. The same goes for side effects - if your function has a side effect, you better put it in its name. Using this naming rule, you’ll find that the ugliest and the most dangerous functions have the longest names!

You also can expose side effects by splitting “clean” and “dirty” code on the module level. For example, a library-like folder should have only clean side-effect-free code. All side effects should go into an application-like folder (e.g., scripts, app, or runners). Here is another nice short post reinforcing this point.

Misc and advanced

The points above should work for the majority of side-effect cleanups. But there are many exceptions and advanced concepts that wouldn't fit this post. Here are some follow-ups.

Dependency injection?
Injecting an object that might produce a side effect instead of producing it yourself is a common way to kick a can down the road:

pass random generator instead of querying it
pass Timer instead of time.time()
pass logging.Logger instead of a print

After reading many resources, this thread probably stays the best.
Here is a great JavaScript post about it (and side effects). In a liberal interpretation, replacing a side-argument with a regular one could be called a dependency injection. One should be careful not to overdo it (1,
2).

Return a functor?
Instead of causing a side effect right away, you can return a "lazy" function that would do it later. See the post about this technique. Also, see the "Lazy functions" section in this post. It's a fun long read, but it probably goes beyond the needs of a regular Python mortal.

Copy an input container?
Modifying an incoming list or dict is also a side effect.
Quite often, it's worth copying, modifying and returning it instead. See the discussion here and this post. Is it slower? Probably, yes. But the actual question should be this:
will your company spend more money on (A) executing slower code or (B) debugging bugs caused by side-effects? Without picking (A) or (B), there is no basis for fear of slow but robust code.

prints and loggers?
While logging is a side effect, it's not the worst one. At least, the majority of developers don't treat it as such.
It's hard to advise anything specific without going on a long tangent. You can adopt configurable logging, pass a logger as a dependency to every function, return string messages or stick with print (e.g. if you use multiprocessing).

Conclusion:

At the end of the day, remember to be pragmatic. You don't want to remove all side effects, you just want to remove unnecessary ones. In practice though, many side effects could be avoided. So you better allocate some time for refactoring and deal with them using some recipes from this post:

make a pass on your code and remove some side effects altogether
identify side-arguments and side-results and merge them with regular arguments or results
identify reused utilities and clean them up first
notice that some side-effects are caused by each other and short-circuit them
expose side effects to developers.

Thank you for reading!

All Dangers of Side Effects for Python Coders

Oleg Sinavski — Tue, 04 Jul 2023 11:30:44 +0000

Are there enough posts about unnecessary side effects? Unlikely - you still see them everywhere.
Here is a sales pitch for this post:

no functional programming jargon
a go-to list of reasons to avoid side effects
Python-oriented

Spotting side effects

Some posts describe side effects in the context of functional programming (and JavaScript): one, two, three and a wiki. You don't have to study functional programming to grasp this concept. Unfortunately, unnecessary side effects are prevalent in all coding paradigms, especially OOP.

Let’s take a simple function that computes the absolute value of a number:

def abs(x):
   if x < 0:
       return -x
   return x

What comes out from the return statement is called a “primary effect”. Now let’s take a look at another function:

def abs_effectful(x):
   if x < 0:
       return -x

   twitter_api = TwitterAPI(some_twitter_id())
   chat_bot = LanguageModel()
   tweet_content = twitter_api.read_first_tweet()
   response = chat_bot.make_response(tweet_content,
       f"{x} is important because ")
   twitter_api.respond(response)
   twitter_api.close()

   return x

Here we also have the same primary effect returned by the abs_effectful function. But a section in the middle performs pretty surprising actions and influences the external world: it has side effects.

You may ask: "Why do you have a weird unrelated piece of code in your function??" This contrived example is not so far from what happens in large codebases. It’s just a set of refactorings, misunderstandings, and a stream of people joining and leaving teams. Time passes, and simple code accumulates unnecessary baggage.

Here are some real examples of side effects that I've encountered in my Python ML career:

publishing a number to the Weights-and-Biases server out of the loss function
dumping debug images into a home folder from a computer vision algorithm
changing an environmental variable inside a function
dropping to pdb on if condition
changing a global variable
calling sys.exit inside an algorithm
sending a Slack message if nan is encountered during training
... and so on.

After seeing many of those, you get a Spider-Sense for spotting side effects. If you get surprised by what function does, quite often the function has a side effect. In fact, the code with unnecessary side effects is not as simple as it could be. See a great post from Eric Elliot about simplicity, surprises and side effects.

Side effects are unavoidable because software must have some influence on the world. But are unnecessary side effects harmful? If there are no visible differences, is there a reason to prefer one script over another?

Unexpected crashes

How can abs crash? It pretty much can’t. What about abs_effectful? It might crash in almost every line:

a Twitter API might fail to initialize
a language model can't allocate GPU memory
it might fail to parse or generate a phrase
a Twitter API might fail to publish a response or crash upon closing

Every side-effect code line might crash since it deals with things you don't control. Now, from the real examples above:

Weights-and-Biases or Slack might be unavailable
you might run out of space in the home folder
an environmental variable is deleted before setting, and so on ...

So the code with side effects is more fragile.

Unexpectedly slow (and occasionally very slow) code

Every side effect might bring computational costs. But most importantly, it brings timing variability into your software. Because side-effect code typically deals with uncontrolled external resources, you get uncontrolled slowdowns. Every side-effect code line in abs_effectful can bring you a massive slowdown:

a Twitter API could be slow to connect
a language model might wait for free GPU memory
computational load on the system might make phrase parsing too slow

In real-world code, similar things can happen:

database access could be very slow because the network is busy
writing to a hard drive could be slow because of fragmentation
logging could be too slow because other loggers compete for file access, and so on ...

The bottom line is that the code with side effects is slower.

Hard to use it concurrently

You must be careful about any state you modify in concurrent (or even worse parallel) software to avoid data races and deadlocks. Code with side effects modifies the external state. It will often require a significant redesign to be used in a concurrent context. You need to ensure that only one function will use a global resource simultaneously and that the order of operations doesn't break the logic.

The timing variability mentioned above makes it even worse.
It's a big topic, so here is a good post discussing concurrency and shared state access in depth.

Hard to optimize

Optimizing a function that doesn't modify anything externally is more straightforward. You can reorder operations inside it and cache the computation results. Since you're not constrained by an external call for a side effect, you can choose any tool for the job. Many side-effect-free functions can be easily rewritten in numpy/torch/jax and Pybind.

So on top of being slow, code with side effects is hard to speed up.

Hard to understand

What if I ask you to make a docstring for abs_effectful? You will definitely struggle to write a concise story about the purpose of that function.

A function with a single return statement like abs achieves a single result. At a minimum, you can concisely write a docstring as

'''This function computes an absolute value.'''

If there are side effects, you have to add them all one by one:

'''This function computes an absolute value 
AND opens Twitter API and creates a model 
AND generates and sends a tweet 
AND closes connections.
'''

In a large company, many people are joining and changing teams. New people have to understand obscure side effects. A company with less-readable code will waste days of cumulative onboarding time. A large docstring will not help: the longer the doc, the greater the chance it will be ignored completely.

Hard to release and debug

The code with side effects interacts with stuff which is not under your direct control. After thinking about all imaginable failure modes of abs_effectful, you might still find that it crashes after the release.

There is a crazy variability in the external world. It is tough to reproduce, debug and fix all edge cases of side effects. Often it happens stochastically, and you just can't reproduce a single crash in-house (e.g., maybe when abs_effectful runs on an old Windows version, GPU kernels behave differently?).

The code with side effects takes much longer to release and debug.

Hard to test

Writing exhaustive unit tests for code with side effects is very hard. On top of the main result of abs_stateful, you should better test that all side effects behave as expected: did the person get a tweet? Was the tweet parsed correctly? Not to say of some weird unit test interactions: side effects from one test could make other tests fail.

Interacting with the external world on a test server is often impossible. Developers have to resort to mocking all external entities. See more in this beautiful post from CodeWhisperer.

Hard to reuse

What if your colleague wants to compute the absolute value of a number? There is no problem with abs. You just import the function and use it! What about abs_effectful? They wouldn't know a priori about any side effects (ignoring the hint in the name :). The code can happily work until some unfortunate day when some crazy issues from above start coming up.

After finding the root cause, they'll anticipate all the problems we mentioned. Your colleague will just leave such a function alone and write their own simpler version.

A codebase that relies on side effects would typically have much less code reuse.

Conclusion

Hopefully, you’re convinced that side effects are also the root of all evil (just like premature optimization). Now here is a compilation of all the reasons above:

Code with side effects

is fragile - leads to unexpected crashes
is unexpectedly code and is hard to optimize
is hard to read and understand, it is hard to reuse
is hard to debug and write tests
is hard to use concurrently.

For more discussion, please see the following great thread on stackexchange.

Meanwhile, thank you for reading!

Neat Python API

Oleg Sinavski — Fri, 24 Feb 2023 23:42:00 +0000

I've seen many Python interfaces in my career (as in API, not UI).
You can quickly spot an ugly one by its size. Below are some recipes on how to make a neat one!

Interfaces in Python

An "interface" is a nebulous concept that might be initially hard to grasp.
It is a blueprint of interactions with an object (reading a few answers here is not going to hurt).

Do you need interfaces in Python at all? There are several good posts about it:

Chelsea Troy
Stack Overflow (please read several answers!)
Diego Barba

I’m going to assume that you read all that and decided to go with interfaces.
I think Protocols are the best way to create interfaces in Python. This is what we are going to use here.

Here, we are going to discuss what makes a good interface.

An interface has to be small

You’re designing a Zoo simulator and made an interface for "a generic animal":

class Animal(Protocol):
   def current_weight(self) -> float: ...
   def price(self) -> float: ...
   def sleep(self, hours): ...
   def eat(self, food): ...
   def draw(self, context): ...
   def look_for_food(self): ...
   def hunger_level(self) -> float: ...
   def is_asleep(self) -> bool: ...
   def is_awake(self) -> bool: ...
   def current_position() -> Tuple[float, float, float]: ...
   def current_orientation() -> RotationMatrix: ...
   def current_transform3d() -> HomogeneousMatrix: ...

You should be alarmed when you see an interface with more than 4-5 methods.
Chances are that it was not well-designed, and the project it is used in will be overly convoluted:

this interface will be used in completely unrelated parts of the software. It will couple them together
there will be extensive use of class hierarchies
interface implementations are going to be overly complicated
it will be hard to document and confusing to use.

You might think: “Clearly, animals do tons of different things, so it's justified to have many methods.”
But you can come up with ten more methods to add to a “generic animal,” and there is never going to be an end to it.
The art is to model a complicated thing with a set of small decoupled interfaces.

What are the recipes for reducing an interface's size?

Semantic overlap

Sometimes there could be some obvious duplication methods (e.g., get_weights vs. current_weight).
But more often you'll find methods that somewhat overlap in semantics.
After a bit of thought, it's possible to remove some of them.
In the example above:

   def current_position() -> Tuple[float, float, float]:
       pass

   def current_orientation() -> RotationMatrix:
       pass

   def current_transform3d() -> HomogeneousMatrix:
       pass

The user extract position and orientation from the HomogeneousMatrix returned by current_transform3d.
The other two getters could be replaced by utility functions.

A similar situation is probably happening with:

   def is_asleep(self) -> bool:
       pass

   def is_awake(self) -> bool:
       pass

Is it true that animal.is_asleep() == not animal.is_awake()? If yes, then you better remove one of them.

Decoupling interfaces

After removing duplicates, you should try to split the interface into independent parts.
Most likely, you don’t use all Animal features in every part of your software:

1. there could be a function that handles the drawing of animals that uses only current_transform3d and draw:

def draw_entity(animal: Animal):
    t = animal.current_transform3d()
    c = some_drawing_context(t)
    animal.draw(c)

2. there could be an animal life cycle algorithm:

def life_management(animal: Animal):
    eating_logic(
      animal.hunger_level(), animal.look_for_food(), 
      animal.eat())
    sleeping_logic(animal.is_awake(), animal.sleep())

3. and a Zoo management system

def purchase_decision(animal: Animal, budget: float) -> bool:
   w = animal.current_weight()
   p = animal.price()
   decide_to_buy(p, w, budget)

The “generic animal” is a collection of quite different interfaces that could be disentangled:

class VisualEntity(Protocol):
   def current_transform3d() -> HomogeneousMatrix: ...
   def draw(self, context): ...

class BehavingAgent:
   def sleep(self, hours): ...
   def eat(self, food): ...
   def look_for_food(self): ...
   def hunger_level(self) -> float: ...
   def is_awake(self) -> bool: ...

class ZooAsset:
   def current_weight(self) -> float: ...
   def price(self) -> float: ...

Now you can see the advantages of many small interfaces over a single big one:

You to decouple parts of your codebase and company workflows. A rendering team will not depend on Zoo management developers.
No need to implement unnecessary methods for slight variations of “Animals.” Changing a price algorithm does not lead to a new species!
You would be able to reuse small interfaces and utilities. You might find that the draw_entity and purchase_decision work for your Plant object.

Now you can see the advantages of many small interfaces over a single big one.

They decouple parts of your codebase (while the large ones have a high chance of coupling unrelated parts). This in turn decouples company workflows (e.g. teams don't break each other's stuff too much)
You don't have to write many implementations when instantiating a short interface. This ripples through your design because now you more easily avoid base classes and class hierarchies. Which in turn promotes composition over inheritance (there are many resources on why it's a good idea, e.g. see here
You would be able to reuse small interfaces in more places than large ones. This in turn makes the total surface area of APIs in your codebase smaller, easier to learn with simpler onboarding

Recursive semantics

Next, you should be alarmed if one function calls another in the same interface.
Continuing with the Zoo example:

class BehavingAnimal(Protocol):
   def lifecycle(self):
       """ What happens during 24 hours period """

   def eat(self, food):
       """ What happens when the animal consumes food """

   def sleep(self, hours):
       """ What happens when the animal sleeps """

It is unlikely that an animal will spend a day without eating and sleeping.
So there is a high chance that eat and sleep will be called from the lifecycle implementation.

Such an interface couples together two levels of abstraction.
These are two interfaces in one, similar to the previous section.
A "lifecycle" part is used in some global application context, but eat and sleep are probably used only locally inside it.
Let's split them apart:

class ElementaryAnimal(Protocol):
   def eat(self, food): ...
   def sleep(self, hours): ...

class LifecycleManagement(Protocol):
   def lifecycle(self, animal: ElementaryAnimal): ...

Notice that now lifecycle takes ElementaryAnimal as an argument.
This clearly states that a lifecycle depends on something that can eat and sleep.

Too generic

Yet another thing to avoid is an overly-generic interface:

class Creature(Protocol):
   def act(self, *args, **kwargs) -> object:
       """ Do anything you want """

 def perform_action(creature: Creature):
     custom_args = ...
     creature.act(*custom_args)

An interface is like a contract between two parties.
But a contract that says “do whatever” is as good as no contract at all.
It is too easy to make such interfaces in an optional/dynamic-typed language like Python, especially with its wildcard argument features.

Unfortunately, such interfaces come up in some situations, but we can summarize it as “passing a black box around.”
Passing “blackboxes” is typically worse than explicit components.
But if you have to do it, it’s better to acknowledge its "blackness" by passing
a functor and moving to a more functional design (here is an in-depth tutorial):

def perform_action(actor: Callable[..., Any]):
    custom_args = ...
    actor(*custom_args)

Constructors in an interface

Related bonus topic: have you noticed that __init__ is typically not a part of any interface?
Why not? It seems like it is just another method you can have.

It goes back to C++, where constructors are not virtual.
Bjarne Stroustrup gives an answer on why it is so:

“A virtual call is a mechanism to get work done given partial information.
To create an object, you need complete information. Consequently, a "call to a constructor" cannot be virtual.”

I like high-level reasoning more: a constructor is a function for making objects. It belongs to the “object-makers” realm.
It just can’t belong to an already-made object: you shouldn't make an object out of itself and I'm not a fan of cloning).

A constructor could be viewed as a method of a factory interface
(similar ideas can be found here).
In other words, a constructor and an object interface are yet another example of two coupled levels of abstraction.

Conclusion

To summarize, a good interface should:

be small and concise
avoid semantic duplication
avoid coupling several parts in one
avoid mixing multiple abstraction levels
be specific and not too generic
not include __init__

At the same time, real life has many exceptions and complications.
But those must be justified exceptions, not excuses to keep messy interfaces around.
Thank you for reading!

You can find me on LinkedIn or Twitter.

Originally published at https://sinavski.com.

Python interfaces: abandon ABC and switch to Protocols

Oleg Sinavski — Tue, 31 Jan 2023 21:34:56 +0000

I used a standard Python library abc to define interfaces for the last 10 years of my career.
But recently, I found that relatively new Python Protocols are way nicer.
People find uses for both technologies.
But I want to convince you to completely jump ships and start using them instead of more traditional techniques.

Python interfaces: what can you use?

Python is somewhat different from other popular languages since there are no interfaces on a language level.
However, there are several library implementations:

abc
typing.Protocols
third-party implementations like Zope
custom implementations (e.g. via metaclasses)

abc package is probably the most popular:

from abc import ABC, abstractmethod

class Animal(ABC):
   @abstractmethod
   def eat(self, food) -> float:
       pass

   @abstractmethod
   def sleep(self, hours) -> float:
       pass

Next, the most frequently mentioned package seems to be Zope:

from zope.interface import Interface

class Animal(Interface):
   def eat(self, food) -> float:
       pass

   def sleep(self, hours) -> float:
       pass

Zope is a web-related library, and its interfaces have a lot of advanced features.

Also, there are more custom packages and tutorials on the web on how to make an interface system yourself (see example).

Finally, there are protocols:

from typing import Protocol

class Animal(Protocol):
   def eat(self, food) -> float:
       ...

   def sleep(self, hours) -> float:
       ...

A protocol is a formalization of Python's “duck-typing” ideology. There are many great articles on structural typing in Python
(this, that
and some discussion).
Maybe protocols and interfaces are theoretically different beasts, but a protocol does the job.
I had great success replacing abc with Protocols without any downsides.

What should you use?

You should know that an interface system will not be localized to a small part of your codebase.
After you choose to go with one, you’ll see it everywhere, and it's going to be hard to change in the future.
So I would immediately dismiss any custom implementations or Zope.
It’s an extra dependency you have to deal with forever: installation, versions, support, and so on.
For example, you have to install a Mypy plugin to support a zope.interface well.
Additionally, a new developer in the team might not know this custom package, and you'll have to explain what it is and why you chose it.

The main battle will happen between abc and Protocols.
But if you really want a zope vs Protocols battle, please read this
(it has a detailed analysis of the runtime benefits of zope).

Static checking first

The big assumption I’m going to make is that you’re already convinced that static checking is a must:
you are not going to run the code that fails pylint/mypy.
Both checkers support abc and Protocols equally well.
Also, just know that both abc and Protocols allow runtime checking, in case you need it.

Both support explicit syntax

First, note that you still can explicitly inherit from an abc and a Protocol. Many arguments in
a very good video (and comments) from Arjan revolve around the misconception
that you can’t do that with protocols. You totally can:

class Giraffe(Animal):
   ...

So in that regard, abc and protocols could be used the same way.
However, Protocols give you an extra degree of design freedom by default.
You can avoid explicit inheritance but still enjoy full interface checking:

class Giraffe:  # no base class needed!
   def eat(self, food) -> float:
       return 0.

   def sleep(self, hours) -> float:
       return 1.

def feed_animal(animal: Animal):
    ... 

giraffe = Giraffe()
feed_animal(giraffe)

This allows you to make an interface for the code you don’t control and loosen the dependencies between modules in your codebase.
Whether to choose an implicit or explicit option is a subtle choice decided on a case-by-case basis.
A good example in favour of explicit "opt-in" for an interface is described here.
Protocols do not force you to opt-in, but you can establish a company-wide rule to explicitly inherit from any protocol.

abc also support implicit interfaces through the concept of "virtual subclasses".
But you have to call register for every implementation:

class Giraffe:  # no base class needed!
   def eat(self, food) -> float:
       return 0.

class Animal(ABC):
    ...

Animal.register(Giraffe)  # achieves the same as implicit Protocol

Procotol supports implicit and explicit variants without extra syntax and works with mypy.
Also, mypy does not support register as of the end of 2022.
I'm not sure if we can fully count that in favour of abc.

Protocols allow you to define an interface for a function (not only a class) - see callback protocols.
It is a very cool feature that is worthy of a separate post.

Both support default methods :(

Unfortunately, there is a big downside to both abc and Protocols.
In the real world, many people work in a single codebase.
Abstract base classes sometimes tend to acquire default method implementations.
This is what it might look like:

class Animal(Protocol):  # the same holds for Animal(ABC):
   def eat(self, food) -> float:
       ...  # this is still abstract

   def sleep(self, hours) -> float:
       return 3.

In that case, they stop being “abstract” and become just base classes.
Python and static checkers do not catch that.
A software design with inheritance is not really the same as a design with interfaces.
I would love Python to separate them on a language level, but it is unlikely to happen.
Implicit protocols have an advantage here.
They allow you to avoid messy inheritance altogether.

Protocols are shorter

Last but not least, you can count the number of lines of code you need to define an interface.
With abc, you must have an abstractmethod decorator for every method.
But with Protocols without runtime checking, you don't have to use any decorators at all.
So here, Protocols win hands down.

Conclusion

Let’s add up the scores:

Capability	ABC	Protocols
Runtime checking	1	1
Static checking	1	1
Explicit interface with inheritance	1	1
Implicit interface without inheritance (`abc` requires `register`)	0.5	1
Ability to have default method implementation	-1	-1
Callback interface	0	1
Number of lines	-1	0

Total	1.5	4

Hopefully, I’m not missing anything huge in this analysis. Thank you for reading!
Looking at the results, team "Protocols" wins, and you probably should just start using it!

Thank you for reading! You can find me on LinkedIn or Twitter.

Originally published at https://sinavski.com.