Make Your Python Code 10x Faster with These Tricks

Jesús Becerra — Thu, 22 May 2025 16:11:18 +0000

1. `slots`

class Person:
    __slots__ = ['name', 'age']

Why use this trick?:

Reduce memory usage: Python usually uses a __dict__ to store object attributes, which consumes more resources and is slow.
Faster attribute access: With __slots__, Python use a static array, eliminating the find key overhead.
Useful when you create millions of instances Data Processing, ETL, Data Science.

2. Generator Expressions (`(x for x in ...)`)

sum(x**2 for x in range(1_000_000))

Why use this trick?:

Avoid creating temporary lists: sum([x**2 for x in ...]) first creates a list in memory, then applies the sum function.
Reduce memory usage: Useful when you create large datasets.

3. Memoization with `functools.lru_cache`

from functools import lru_cache

@lru_cache(maxsize=None)
def fib(n):
    return fib(n-1) + fib(n-2) if n > 1 else n

Why use this trick?:

Avoid recalculate results: Store in cache the values for the same parameters.
Pure functions (same input → same output): Ideal for recursive algorithms like Fibonacci, search in tree.

4. Vectorization with NumPy

import numpy as np
arr = np.array([1, 2, 3])
result = arr * 2

Why use this trick?:

Optimized operations in C: NumPy execute operations in contiguous memory blocks (arrays in C), without loops in Python.
Bypassing the GIL: Operations are applied directly to the data without Python loops.

5. Cython compilation

# example.pyx (Cython)
def sum(int a, int b):
    return a + b

Why use this trick?:

Converts Python code into C: Cython compiles code into native C extensions.
Static types improve efficiency: Declare Types (int, float), avoid python's dynamic typing.
Faster mathematical computations: 100x Faster math operations.

6. `collections.defaultdict` for Dictionaries

from collections import defaultdict
count = defaultdict(int) # Initialize values by default
count["word"] += 1

Why use this trick?:

Avoid if key in dict checks: In a normal dict, you need check if the key exist first. defaultdict do it for you.
Less calls to functions: More efficient that dict.setdefault() or try/except KeyError.

7. Concatenation with `str.join()`

"".join(str_list)

Why use this trick?:

Strings are immutable: In Python, each += create a new string object (This process is O(n²)).
join() pre-assign memory: Join each item in a only operation (This process is O(n)).

Performance Improvements

Trick	Improvement	Ideal Use Case
`__slots__`	~20-30% more faster	Handling mulpliple object instances
Generators	2-5x (in-memory)	Big data processing
`lru_cache`	10-100x (velocity)	Recursively called functions
NumPy	10-1000x	Matrix operations
Cython	10-100x	Heavy mathematical loops
`defaultdict`	~2x	Counting
frecuencies
`str.join()`	10-100x (big N)	Mass text concatenations

Compatibility by Python Version

1. `slots`

Compatible Versions: Python 2.7 and all Python 3.x versions (3.0+).
Details: Works the same across all modern versions.

2. Generator Expressions (`(x for x in ...)`)

Compatible Versions: Python 2.4+ (but optimized in Python 3.x).
Improvements in Python 3:
- Generators are more memory-efficient in Python 3.
- range() in Python 3 is already a generator (in Python 2, it was a list).

3. `functools.lru_cache`

Compatible Versions:
- Python 3.2+ (included in the standard library).
- Python 2.7 requires pip install functools32.
Changes in Python 3.8+:
- New user_function parameter for customization.

4. Vectorization with NumPy

Compatible Versions: Python 2.7 and Python 3.x (requires NumPy ≥ 1.16 for Python 3.7+).
Recommendation: Use NumPy ≥ 1.19 for Python 3.9+ (better support).

5. Cython

Compatible Versions:
- Python 2.7 (supported up to Cython 0.29.x).
- Python 3.x (requires Cython ≥ 3.0 for Python 3.11+).
Important:
- Python 3.11+ has optimized the interpreter, reducing the performance gap between native Python and Cython in some cases.

6. `collections.defaultdict`

Compatible Versions: Python 2.5+ and all Python 3.x versions.
No significant changes across versions.

7. String Concatenation with `str.join()`

Compatible Versions: Python 2.x and 3.x.
Optimization in Python 3.9+:
- Internal improvements in string handling (PEP 584).

Resume

Data Structures: __slots__, defaultdict.
Memory & CPU Efficiency: Generators, NumPy, Cython.
Algorithmic Optimization: lru_cache, str.join().

Forem: Jesús Becerra

Make Your Python Code 10x Faster with These Tricks

1. __slots__

Why use this trick?:

2. Generator Expressions ((x for x in ...))

Why use this trick?:

3. Memoization with functools.lru_cache

Why use this trick?:

4. Vectorization with NumPy

Why use this trick?:

5. Cython compilation

Why use this trick?:

6. collections.defaultdict for Dictionaries

Why use this trick?:

7. Concatenation with str.join()

Why use this trick?:

Performance Improvements

Compatibility by Python Version

1. __slots__

2. Generator Expressions ((x for x in ...))

3. functools.lru_cache

4. Vectorization with NumPy

5. Cython

6. collections.defaultdict

7. String Concatenation with str.join()

Resume

1. `slots`

2. Generator Expressions (`(x for x in ...)`)

3. Memoization with `functools.lru_cache`

6. `collections.defaultdict` for Dictionaries

7. Concatenation with `str.join()`

1. `slots`

2. Generator Expressions (`(x for x in ...)`)

3. `functools.lru_cache`

6. `collections.defaultdict`

7. String Concatenation with `str.join()`