Forem: Xavier Barbosa

Some notes about the right usage of memoization in Python

Xavier Barbosa — Tue, 14 Oct 2025 19:01:59 +0000

Memoization

In computing, memoization is an optimization method that speeds up programs by caching the results of costly function calls and reusing those stored results when the same inputs appear again.Retry

Python’s `functools.lru_cache` and `functools.cache`

functools — Higher-order functions and operations on callable objects

Simple lightweight unbounded function cache. Sometimes called “memoize”.

Returns the same as lru_cache(maxsize=None), creating a thin wrapper around a dictionary lookup for the function arguments. Because it never needs to evict old values, this is smaller and faster than lru_cache() with a size limit.

For example:

@cache
def factorial(n):
    return n * factorial(n-1) if n else 1
>>> factorial(10)      # no previously cached result, makes 11 recursive calls
3628800
>>> factorial(5)       # just looks up cached value result
120
>>> factorial(12)      # makes two new recursive calls, the other 10 are cached
479001600

The cache is threadsafe so that the wrapped function can be used in multiple threads. This means that the underlying data structure will remain coherent during concurrent updates.

What are the places I have seen reused in recent years?

Avoid re-re-re-load costly yaml files

@cache
def get_glossary_data():
    return load_yaml_file("nomenclature/glossary.yml")

Implement some Multiton on stuff like localized i18n catalogs

@cache
def get(locale: str, *, domain: str) -> Translations:
    ... # a lot of heavy work
    return Translations(...)

Compile once slow jmespath expressions

@cache
def compile_query(expression: str) -> ParsedResult:
    return jmespath.compile(expression)

Extend an object from another domain

@cache
def load_settings(entity_id: str) -> Settings:
    return SettingsFactory(entity_id).load_settings()

Maybe there are other ways to use it in the code source?

Things to be aware of, things to avoid, and things to think about

🌞 When it is a pure function

from lib.i18n import get_locale
@cache
def get_heavy_stuff():
    for _ in range(0, 1_000**2):
        ...
    return "funny"

nothing to say. Just use it 😎

🚨 When implementation uses contextual content under the hood

For example

from lib.i18n import get_locale
@cache
def get_that_thing(text):
    locale = get_locale()  # ⛔️🚨🚔👮🏽📢
    return TheHeavyThing(text, locale=locale)

In this example, locale is an hidden parameter. Refactor:

from lib.i18n import DEFAULT_LOCALE
@cache
def get_that_thing(text, locale: str = DEFAULT_LOCALE): # 👼🌴🌞
    return TheHeavyThing(text, locale=locale)

🐒 Monkey patches that change constants

🐒🌴🥥

# lib.some.thing
MAGIC_VALUE = 42
@cache
def get_that_power():
    return MAGIC_VALUE ** 2
def am_i_strong_enough(me):
    return get_that_power() < me

# tests.lib.some.thing_test
from lib.some.thing_test import get_that_power
def test_get_that_number_1(monkeypatch):
    monkeypatch.setattr("lib.some.thing.MAGIC_VALUE", 1)  # 🌴🥥
    assert get_that_power() == 1
...
def test_am_i_strong_enough():
    assert am_i_strong_enough(10) is False  # 🫨😨🫨😨

Avoid to patch constant whenever it’s possible, it will make you trouble. 🫨

By refactoring the code you should be able to get by, and so should your colleagues.

😰 Using @cache to decorate methods

class DumbestCalculatorEver:
    @cache
    def double(self, x): return x * 2
DumbestCalculatorEver()  # never garbage collected
DumbestCalculatorEver()  # never garbage collected
DumbestCalculatorEver()  # never garbage collected

Don’t or do it wisely on a singleton. Because the object will become immortal, and may leads to memory leaks.

🤔 Cache entities properties naively

@cache
def is_settings_metasyntactical(settings: Settings) -> bool:
    return settings.search("foo || bar || baz || `false`")
settings1 = Settings({"id": "xxx", "foo": True})
settings2 = Settings({"id": "xxx", "bar": False})
settings3 = Settings({"id": "xxx", "baz": True})
assert is_settings_metasyntactical(settings1) is True  # ???
assert is_settings_metasyntactical(settings2) is False  # ???
assert is_settings_metasyntactical(settings3) is True  # ???
assert hash(settings1) == hash(settings2) == hash(settings3)  # the truth

What can we do for it? I don’t think that there is an existing mechanism to handle this case properly, nor there is a single way to handle this case.

This has caused tests to be flaky, but might also be causing bugs in production.

Some ideas to handle entities properties case

Given this function

@cache
def is_settings_metasyntactical(settings: Settings) -> bool:
    return settings.search("foo || bar || baz || `false`")

We expect that this function behaves like a Settings property, which would be equivalent to:

class Settings:
    @property
    def is_metasyntactical(self):
        return self.search("foo || bar || baz || `false`")

By chance, it is unlikely that Settings config is changed during a script resolution.

Moreover cached settings are always cleared a the end of a script / call.

There are several ways of handle this case:

Idea 1: always clear this cache at the end of a script / query

try:
    return resolve_the_query()
finally:
    is_settings_metasyntactical.clear_cache

Idea 2: let the entity register external attributes? for example:

class Settings:
    cache = {}
def is_settings_metasyntactical(settings: Settings) -> bool:
    cache_key = is_settings_metasyntactical
    try:
        result = settings.cache[cache_key]
    except KeyError:
        result = settings.cache[cache_key] = settings.search("foo || bar || baz || `false`")
    return result

Because we always clear settings at the end of a script / call, the cached data will be destroyed with entity instances.

Idea 3: use an external cache using weakref

The weakref module allows the Python programmer to create weak references to objects.

A weak reference to an object is not enough to keep the object alive: when the only remaining references to a referent are weak references, garbage collection is free to destroy the referent and reuse its memory for something else.

from weakref import WeakKeyDictionary
_cache = WeakKeyDictionary()
def is_settings_metasyntactical(settings: Settings) -> bool:
    try:
        result = _cache[settings]
    except KeyError:
        result = _cache[settings] = settings.search("foo || bar || baz || `false`")
    return result

We rely on python to cleanup the cache during garbage collection.

Data-Oriented Programming is dope

Xavier Barbosa — Mon, 03 Jan 2022 06:07:38 +0000

Photo by Ravi Roshan on Unsplash.
This post was published first in https://techblog.deepki.com/data-oriented-programming/. This is an English translation by the same author.

Data-oriented programming (DOP) is not a new concept. It's a paradigm that can be used by developers in any programming language, it's purpose is to reduce complexity of information system that they are designing.

Yehonathan Sharvit explains that in his book Data-oriented programming.
The book explores tenets of this paradigm, as a dialog between two people.

The narrator is a junior javascript developer, he develops a Library Management System of a client. Initial features are easy, the software is coded in object oriented. But when the client asks for some new features at the very last moment, everything becomes complicated. He seeks support from a veteran developper, Joe.

Throughout these pages Joe demonstrates difficulties he encounters, shows him how to confront them. In fine, he teaches him a new way to organize his source code that is more easy to decipher and to evolve.

Examples in the book are in Javascript, I wanted to introduce my interpretation in Python on a small part of these rules: code and data separation.

In the book, the heroes talk about user management. The narrator had to design two types of users, the librarian and the member:

Once this logic implemented, the client requested him to add super members, then VIP members.
It achieves to be this UML class diagram:

It's really hard to manage for the young narrator. Although everything is perfectly logic, the classes hierarchy is hard to work with, mixing inheritance with dependencies.

Joe understands affirms that is "feelings" are due to « Data encapsulation has its merits and drawbacks: Think about the way you designed the Library Management System. According to DOP, the main cause of the complexity of systems and their lack of flexibility is because code and data are mixed together in objects »

That's what Yehonathan Sharvit fights all along the book: he depicts the difficulty to just understand something and be able to upgrade it without difficulties.

Complexity is a thing that have been accumulated insidiously. When it's not kept under control, implementing new features can take weeks instead of days. But DO comes with a radical approch to fight this complexity. To achieve this, data and code must be separated:

In order to explain this separation, here is my implementation in python.

I've followed technics that are described in the book. I started from the client specifications, I've made a list of names that seem to represent Entities of the system, and another list of everything that look like a feature. Then I've organized what I've found:

Two kind of users: members and librarians
Users log in to the system via email and password
Members can borrow books
Members and librarians can search books by title or by author
Librarians can block and unblock members
Librarians can list the books currently lent by a member
There could be several copies of a book

Entities classified by groups:

Features put in several code modules:

On this basis, I will implement book lending.

The catalog's data part:

$schema: "https://json-schema.org/draft/2020-12/schema"
properties:
  lendings:
    additionalProperties:
      type: object
      properties:
        id: { type: string }
        user_id: { type: string, format: uuid }
        book_item_id: { type: string }
      required: [id, user_email, book_item_id]
  propertyNames: { type: string, format: uuid }
required: [lendings]

The user_management's data part:

$schema: "https://json-schema.org/draft/2020-12/schema"
properties:
  members_by_id:
    type: object
    additionalProperties:
      type: object
      properties:
        is_blocked: { type: boolean }
      required: [is_blocked]
    propertyNames: { type: string, format: uuid }
required: [members_by_id]

Here I've used JSON Schema, because data does not have to be contained in rigid structures. Only the keys are relevant and need to be specified. In DO, data requires to obey three other rules:

all types are generic
all types are immutable
shape of data and data schema are separated

Here is a mock that validates these two schemas:

library_data = {
    "catalog": {
        "books_by_isbn": {
            "9781234567897": {
                "title": "Data Oriented Programming",
                "author": "Yehonathan Sharvit",
            }
        },
        "book_items_by_id": {
            "book-item-1": {
                "isbn": "9781617298578",
            },
            "book-item-2": {
                "isbn": "9781617298578",
            }
        },
        "lendings": [
            {
                "id": "...",
                "user_id": "member-1",
                "book_item_id": "book-item-1",
            }
        ],
    },
    "user_management": {
        "members_by_id": {
            "member-1": {
                "id": "member-1",
                "name": "Xavier B.",
                "email": "xavier@deepki.com",
                "password": "aG93IGRhcmUgeW91IQ==",
                "is_blocked": False,
            }
        }
    },
}

By convention, dict are used like some Mapping, and I forbid myself to update them.

Please note that examples will use the classes+static method form in order to make this article readable. In a production code, the modules+functions form is the way to go.

And now the code part:

from __future__ import annotations

from typing import Tuple, TypeVar
from uuid import uuid4

T = TypeVar("T")


class Library:
    @staticmethod
    def checkout(library_data: T, user_id, book_item_id) -> tuple[T, dict]:
        user_management_data = library_data["user_management"]
        if not UserManagement.is_member(user_management_data, user_id):
            raise Exception("Only members can borrow books")
        if UserManagement.is_blocked(user_management_data, user_id):
            raise Exception("Member cannot borrow book because he is bloqued")
        catalog_data = library_data["catalog"]
        if not Catalog.is_available(catalog_data, book_item_id):
            raise Exception("Book is already borrowed")
        catalog_data, lending = Catalog.checkout(catalog_data, book_item_id, user_id)
        return (
            library_data | {
                "catalog": catalog_data,
            },
            lending,
        )


class UserManagement:
    @staticmethod
    def is_member(user_management_data: T, user_id) -> bool:
        return user_id in user_management_data["members_by_id"]

    @staticmethod
    def is_blocked(user_management_data: T, user_id) -> bool:
        return user_management_data["members_by_id"][user_id]["is_blocked"] is True


class Catalog:
    @staticmethod
    def is_available(catalog_data: T, book_item_id) -> bool:
        lendings = catalog_data["lendings"]
        return all(lending["book_item_id"] != book_item_id for lending in lendings)

    @staticmethod
    def checkout(catalog_data: T, book_item_id, user_id) -> Tuple[T, dict]:
        lending_id = uuid4().__str__()
        lending = {"id": lending_id, "user_id": user_id, "book_item_id": book_item_id}
        lendings = catalog_data["lendings"]
        return (
            catalog_data | {
                "lendings": lendings + [lending]
            },
            lending
        )

As we can see, code is a series of pure functions.
Functions that modify a state return a new state object rather than upgrading the previous state.

In each module, functions are made simple and easy to test. They can be reused in any context, like the main module. Globally, they are composed with other existing function. It becomes very easy to adapt them for the client's needs.

And now, which path will data lead if my alter-ego borrows another copy of the book?

library_data, lending = Library.checkout(
    library_data,
    user_id="member-1",
    book_item_id="book-item-2",
)

Two things occur:

Data is systematically transmitted to every function calls. This object is quite opaque, each level use only a fragment that he knows without worrying about the remaining:

# 1. injects data into Library.checkout module
library_data, lending = Library.checkout(library_data, ...)

# 2. extracts data from user_management
user_management_data = library_data["user_management"]

# 3. uses this data fragment into UserManagement module
if not UserManagement.is_member(user_management_data, ...):
    ...
if UserManagement.is_blocked(user_management_data, ...):
    ...

# 4. picks catalog data
catalog_data = library_data["catalog"]

# 5. uses this data fragment into Catalog module
if not Catalog.is_available(catalog_data, ...):
    ...
... = Catalog.checkout(catalog_data, ...)

When a function is about to change a state, it returns a new version of data. Every level of the call stack must returns a new version of data:

# 1. handles the request in Catalog.checkout
lending = ...
lendings = catalog_data["lendings"]
# 2. creates a new version of catalog_data
catalog_data = catalog_data | {
    "lendings": lendings + [lending]
}
# 3. interception of the new catalog_data by Library.checkout
catalog_data, ... = Catalog.checkout(...)
# 4. creation of a new version of library_data
library_data = library_data | {
    "catalog": catalog_data,
}

Then, this new version of data can be exposed to whole system.

In my example, I don't talk about data consolidation. I suggest you to read the book which gives informations concerning this subject.

Is it pythonic?

Broadly speaking, this paradigm fits well in Python if we shelve the object oriented capabilities of the language.
The notion of modules in data-oriented are naturally superimposed on modules in Python, which facilitates adherence.

Borrowings of functional languages such as map(), filter() as well as functions from standard operator module also contribute to make this paradigm quite natural in Python.

In our example, standard typing will not work. However, it is quite easy to do custom typing, such as:

from __future__ import annotations

from typing import Any, Mapping


class M(Mapping[str, Any]):
    def __or__(self, other: dict) -> M:
        return {**self, **other}  # type: ignore

    def __hash__(self) -> int:
        ...

# which can be used as well in the source code

def my_func(members_data: M[str, M], member_id: str):
    member = members_data[member_id]

Code purity

In my implementation my functions are not really pure because I have used exceptions. However, this disgression is acceptable if it is applied under a certain condition. Indeed, exceptions are only used to express illegal operations, in a throw early, catch late way. Using them this way contributes to the readability of the code. The higher layers of the system will know how to deal with them.

For example in a Flask application:

@app.post("/checkout")
def checkout_view():
    ...
    try:
        ..., lending = Library.checkout(library_data, user_id, book_item_id)
        # la fonction 'checkout' peut lever des exceptions
        ...
        return jsonify(lending), 201
    except Exception as error:
        result = {"error": str(error)}
        return jsonify(result), 400

The author is also clear that embracing DO comes at a cost. For example, the fact that DO is relatively agnostic of any programming language undermines the guarantees offered by object modeling (or other tools such as code analysis that some IDEs allow). However, he sometimes offers alternatives for this, such as JSON Schema used here.

What I presented to you was just a preview of DOP using pure Python. The author gives a lot of details about unit tests, data structures, state management, structural sharing, atomicity, transformation pipeline, etc.

I highly recommend you to read Data-oriented programming by Yehonathan Sharvit, and please follow his approach in his blog.

Finally

The author is a multilingual developer and without citing it, many concepts come from the Clojure language. According to its defenders, Clojure is the easiest programming language in the world because it has almost no syntax or grammar and it was designed by Rich Hickey in such a way as to facilitate code changes.

This language can be inspiring for other languages. To convince yourself, you can consult this other talk Design, Composition, and Performance Short by Rich Hickey.

It makes me happy to see some of these principles reused in other languages. Indeed, languages must nurture on each other.

Bref, making functional code in Python is dope.