Forem: Dmitry Posokhov

Asyncio Events and Conditions: The Secret Life of Python's Traffic Signals

Dmitry Posokhov — Mon, 07 Oct 2024 06:13:27 +0000

Ever wondered how your Python coroutines manage to play nicely together without causing a traffic jam? Let's dive into the whimsical world of asyncio events and conditions—the unsung heroes keeping your asynchronous code from turning into a circus (unless you want it to).

Events: The Green Light Everyone's Waiting For
Think of an asyncio.Event as a traffic light in your code. Coroutines line up and wait patiently (or not so patiently) for the light to turn green before zooming off.

Imagine you're at a crosswalk with a group of people (coroutines). The pedestrian signal is red (event is unset), so everyone waits. The moment it turns green (event.set()), the entire group moves forward in unison. No one wants to be that person dodging cars because they went ahead on a red light.

import asyncio

async def pedestrian(event):
    print("Pedestrian is waiting for the signal.")
    await event.wait()
    print("Pedestrian crosses the street.")

async def traffic_light(event):
    print("Traffic light is red.")
    await asyncio.sleep(2)
    event.set()
    print("Traffic light turns green.")

async def main():
    event = asyncio.Event()
    await asyncio.gather(pedestrian(event), traffic_light(event))

asyncio.run(main())

Output:

Pedestrian is waiting for the signal.
Traffic light is red.
Traffic light turns green.
Pedestrian crosses the street.

Conditions: The VIP Pass at the Club
asyncio.Condition is like that bouncer at the exclusive club. Not only do you need the club to be open (with condition), but you also need to meet certain criteria (await condition.wait()).

Picture trying to get into a trendy nightclub. The club (condition) has limited capacity, and the bouncer only lets you in if someone else leaves (a condition is met). You might wait outside, practicing your best dance moves (or checking your phone awkwardly), until you get the nod.

import asyncio

async def clubber(condition, name):
    async with condition:
        print(f"{name} wants to enter the club.")
        await condition.wait()
        print(f"{name} enters the club.")

async def bouncer(condition):
    await asyncio.sleep(2)
    async with condition:
        print("Bouncer signals someone can enter.")
        condition.notify()

async def main():
    condition = asyncio.Condition()
    await asyncio.gather(
        clubber(condition, "Alice"),
        clubber(condition, "Bob"),
        bouncer(condition)
    )

asyncio.run(main())

Output:

Alice wants to enter the club.
Bob wants to enter the club.
Bouncer signals someone can enter.
Alice enters the club.

In this example, only one clubber gets to enter the club because the bouncer only signals one person (condition.notify()). The other clubber remains waiting indefinitely. If you want everyone to get in (party time!), you can use condition.notify_all().

import asyncio

async def clubber(condition, name):
    async with condition:
        print(f"{name} wants to enter the club.")
        await condition.wait()
        print(f"{name} enters the club.")

async def bouncer(condition):
    await asyncio.sleep(2)
    async with condition:
        print("Bouncer signals everyone can enter.")
        condition.notify_all()

async def main():
    condition = asyncio.Condition()
    await asyncio.gather(
        clubber(condition, "Alice"),
        clubber(condition, "Bob"),
        bouncer(condition)
    )

asyncio.run(main())

Output:

Alice wants to enter the club.
Bob wants to enter the club.
Bouncer signals everyone can enter.
Alice enters the club.
Bob enters the club.

Wrapping Up the Party
By now, you should have a clearer understanding of how asyncio events and conditions work. They're the behind-the-scenes coordinators ensuring everything runs smoothly.

So the next time your asynchronous code doesn't resemble a pile-up on the freeway, you know who to thank!

Beyond Zipping Up a Coat: Beneficial use of itertools.zip_longest()

Dmitry Posokhov — Fri, 09 Aug 2024 18:33:27 +0000

Bet you've probably encountered the frustration of losing data trying to zip two lists with different length in Python.
itertools.zip_longest is here to save the day. Here I want to explore how to use zip_longest(), compare it with the standard zip, and delve into practical scenario where it shines.

What is itertools.zip_longest?

The zip_longest() function from the itertools module in Python allows you to zip multiple iterables, filling in the shorter ones with a specified value (default). This ensures that no data is lost, even if the iterables are of different lengths.

Practical Example

Consider a scenario where you’re trying to seat students in a classroom, but the number of students and the number of available desks don’t match. You want to ensure that every student has a seat, and every seat is filled as much as possible.

If you use zip(), as soon as you run out of students or desks, the pairing stops. This means that some desks might remain empty or some students might be left standing.
With zip_longest(), you can pair every student with a desk, and if you run out of desks, you can note that extra students need to stand. Alternatively, if there are more desks than students, you can mark the extra desks as "empty". Every student is accounted for, and you know exactly which desks are left unoccupied.

Consider an example where the number of desks exceeds the number of students, using both zip() and zip_longest().

Using zip()

students = ['Alice', 'Bob']
desks = ['Desk 1', 'Desk 2', 'Desk 3']

# Using zip to pair students with desks
seating_zip = list(zip(students, desks))

print("Seating with zip:")
for student, desk in seating_zip:
    print(f"{student} is assigned to {desk}")

Output:

Seating with zip:
Alice is assigned to Desk 1
Bob is assigned to Desk 2

With zip(), the pairing stops as soon as the shorter list (students) is exhausted. Desk 3 remains unassigned, and there’s no indication that it is unused.

Using zip_longest()

from itertools import zip_longest

students = ['Alice', 'Bob']
desks = ['Desk 1', 'Desk 2', 'Desk 3']

# Using zip_longest to pair students with desks
seating_zip_longest = list(zip_longest(students, desks, fillvalue='Empty Seat'))

print("\nSeating with zip_longest:")
for student, desk in seating_zip_longest:
    print(f"{student} is assigned to {desk}")

Output:

Seating with zip_longest:
Alice is assigned to Desk 1
Bob is assigned to Desk 2
Empty Seat is assigned to Desk 3

With zip_longest(), every desk is accounted for, even if there aren’t enough students to fill all the seats. In this case, Desk 3 is paired with "Empty Seat" indicating that this desk remains unoccupied. This approach is particularly useful when you need to keep track of all resources, ensuring nothing is left out.

👍 Advantages of zip_longest():
Preserves Data: Ensures no data is lost by filling shorter iterables.
Flexibility: Allows specifying a custom fill value.
Comprehensive Pairing: Useful in data processing tasks where alignment of different-length iterables is necessary.
👎 Disadvantages of zip_longest():
Padding May Be Unwanted: In some cases, padding might introduce unnecessary complexity.
Memory Usage: Might use more memory if dealing with large iterables and large fill values.

👍 Advantages of zip():
Simple and Efficient: Works well with iterables of equal length.
Less Memory Usage: No padding means potentially less memory overhead.
👎 Disadvantages of zip():
Data Loss: Truncates to the shortest iterable, losing data from longer ones.

Conclusion

zip_longest() is a powerful tool in Python’s arsenal, especially when working with iterables of different lengths. It ensures data integrity by filling in missing values, making it ideal for various data processing tasks. While zip() is simpler and more memory-efficient, zip_longest() provides the flexibility needed in many practical scenarios.

Do Not Make This Mistake with Python Descriptors

Dmitry Posokhov — Mon, 22 Jul 2024 08:46:27 +0000

Descriptors provide a powerful way to manage attribute access (like as Django models style 😀), but when used in multiple instances of a class, they can lead to unexpected behavior. In this short article, I'll explain what happened in my work and how to avoid this common issue.

What are Descriptors?
Descriptors are special objects in Python that manage the access to another object's attributes. They implement the methods __get__, __set__, and __delete__. When used correctly, descriptors can greatly enhance the functionality of a class.

The Problem I Noticed:
Recently, while reviewing a colleague's code, I noticed an issue with use of descriptors to manage an attribute across multiple instances of a class. Here’s a simplified version of code:

class MyDescriptor:
    def __init__(self):
        self.value = None

    def __get__(self, instance, owner):
        return self.value

    def __set__(self, instance, value):
        self.value = value

So what is wrong?
Try to use this descriptor in class:

class MyClass:
    attr = MyDescriptor()

Then create a couple of instances of this class:

obj1 = MyClass()
obj2 = MyClass()

And so what? Ok, try to set values to 'attr' in both instances:

obj1.attr = 10
obj2.attr = 20

So, what we see when we try to use 'attr' from first one instance:

print(obj1.attr)  # Expected: 10, but it was 20
print(obj2.attr)  # Expected: 20

Surprisingly (or not 😏), changing the value in obj1 also changed the value in obj2. We are expecting each instance to have its own value, but it turned out that the same descriptor instance was shared across all instances of the class.

Understanding the Issue:
The issue arises because the descriptor is a class-level attribute. This means that the same descriptor instance is used for all instances of the class. Consequently, setting the value on one instance affects all other instances.

The Solution:
To ensure that each instance has its own value, we need to store the value separately for each instance. This can be achieved using a dictionary to hold values for each instance. Here’s the revised code:

class MyDescriptor:
    def __init__(self):
        self._values = {}

    def __get__(self, instance, owner):
        if instance is None:
            return self
        return self._values.get(instance, None)

    def __set__(self, instance, value):
        self._values[id(instance)] = value

class MyClass:
    attr = MyDescriptor()

obj1 = MyClass()
obj2 = MyClass()

obj1.attr = 10
obj2.attr = 20

print(obj1.attr)  # Now: 10
print(obj2.attr)  # Now: 20

By using a dictionary to store the values, each instance now correctly maintains its own value without affecting others.

Conclusion:
Descriptors are a powerful tool in Python, but they come with their own set of challenges. When using descriptors, especially in multiple instances of a class, it’s crucial to ensure that each instance maintains its own state. Using a separate storage mechanism like a dictionary can help avoid unexpected behavior.