Forem: Adarsh Punj

Memory management in Python: Interning & Optimization

Adarsh Punj — Sat, 14 Oct 2023 17:21:18 +0000

Originally published at Pythongasm

Introduction

Behind Python’s simplicity lies a series of thoughtful decisions that makes it so user friendly.
In this article, we will look at some of these decisions, and understand how memory is managed in CPython.

`is` operator

Let’s start with the is operator.

We'll be using shell/IDLE for all these operations:

>>> a = [1,2,3]
>>> b = [1,2,3]

>>> a is b
>>> False

This is simply because a and b are pointing to two different locations in the memory.

We can verify this using id().

>>> id(a) == id(b)
>>> False

We can confirm that is operator is doing nothing but just comparing the memory addresses of two objects if we look at its implementation:

int Py_Is(PyObject *x, PyObject *y)
{
    return (x == y);
}

Source: CPython on GitHub

Use is operator instead of comparing memory addresses using id(), which is a costlier operation.

Now, if we do b = a, then b starts pointing to same memory as a, hence a is b is True.

This is also true for all singletons in Python:

>>> a = None
>>> b = None

>>> a is b
>>> True

Singletons are objects that only exist once in the memory. If a variable is bind to a singleton object, it will point to this one and only memory.

However, integers are not singletons:

>>> a = 1234
>>> b = 1234

>>> a is b
>>> False

Great, let’s try that again with a different integer:

>>> a = 5
>>> b = 5

>>> a is b
>>> True

Oops, this looks weird. Actually, Python treats smaller integers a bit differently. When you start a Python environment, some integers are already allocated into memory. Hence, when you do a = 5, Python smartly tags a to an already allocated memory address. Same goes for b = 4.

“smaller integers” here means integers in the range [-5, 256].

IDLE vs `.py` file

We know that integers are immutable objects (can’t be changed at a memory level), so doesn’t it make sense to reuse all immutable objects?

They are not going to change anyway — so why not just allocate some space when it’s created for the first time, and whenever a new assignment is made with that object, simply point to that address.

That’s what Python often does. This is called interning.

Let’s rerun the same code in a .py file:

a = 1234
b = 1234

k = a is b

>>> k
>>> True

This shouldn’t be surprising because when we compile a .py file, the whole code is parsed by Python (unlike shell or IDLE where each line is executed separately), and Python is able to intern values when required.

Interning responsibly

However, we shouldn’t count on this behaviour of interning. This doesn’t mean the memory management is weird and unpredictable.

The goal here is efficiency, regardless of how it is achieved — interning or not.

In some cases, the time spent for interning is a good trade-off but it might not be a great choice, when you have a big big string which probably isn’t going to be reused or compared. So it's better to save the time.

Since this is so subjective, and depends on the context, Python gives you the flexibility to choose if you want to intern a value manually, with it’s sys.intern() API.

Conclusion

We saw some choices Python makes to make things efficient. These are implementation level details, and not a guarantee which your code should rely on; rather the takeaway here is understanding low level design.

For example, interning -5 is a choice, and so is interning all strings of __len__() = 1. These might change over time.

There’s lot more to this topic, e.g. garbage collection, which will be covered in upcoming articles.

🔔 Desktop notifications in 10 lines of Python code

Adarsh Punj — Mon, 09 Oct 2023 21:12:22 +0000

This article was originally published at Pythongasm

Introduction

We often use the print function to see the progress of a script, or simply for debugging purposes. Rather than constantly looking at the terminal, and waiting for an output to pop up, we can use push notifications as a way to track progress while working on something else.

Our approach to make this happen will be very simple — we will use operating system’s built-in commands for notifications, and just execute them through Python. Few lines of code, no third party modules.

MacOS

The following terminal command executes AppleScript (Apple’s own scripting language) for notifications:

osascript -e 'display notification "Your message goes here" with title "Title"'

Try running this command via terminal, and you will see a notification like this:

Let’s break down the script to understand how this works:

We can customise/parameterise and execute this command with a simple Python snippet:

import os
title = "Success"
message = "File downloaded"
command = f'''
osascript -e 'display notification "{message}" with title "{title}"'
'''
os.system(command)

Linux

Linux-based operating systems offer an even simpler command:

notify-send "Your message" "Title"

Just like we did it for macOS, we can pass this command in os.system() and that’s it.

Making it cross-platform

By this, I just mean making sure the same Python file works on both macOS and Linux. Python has a built-in module called platform, which we will use to know the host machine's OS.

>>> import platform
>>> platform.system()
>>> 'Linux'

platform.system() returns 'Darwin' when run on macOS.

Now, all we need to do is put conditionals, check the platform and execute the command accordingly. We can put all of this together, and define a push() function like this:

def push(title, message):
    plt = platform.system()

    if plt=='Darwin':
        command = f'''
        osascript -e 'display notification "{message}" with title "{title}"'
        '''
    if plt=='Linux':
        command = f'''
        notify-send "{title}" "{message}"
        '''
    else:
        return

    os.system(command)

Conclusion

This was a quick and easy way to push desktop notifications with Python. There are many third-party libraries out there made specifically for this purpose. This tutorial only covered unix platforms, however, we have a Python way to do this on Windows as well. Check out the next section for relevant links.

🔥 Create GPT-3 bot out of your own PDFs using Python

Adarsh Punj — Tue, 08 Aug 2023 17:25:05 +0000

Read the latest version of this article here

Introduction

Traditionally most chatbots were a tree of if-else blocks. Some had a layer of sophistication where users’ intent was captured, to decide which function should be triggered. However, the tech was never good enough to resolve an actual query — until now.
After GPT-3, we've seen how large language models are able to formulate near-human answers.

In this tutorial, we'll index documents, create a GPT-powered bot on top of it, and finally expose it as an API.

Context + LLM

These LLMs like ChatGPT aren't trained on enterprise data. They are general purpose models, created to chat and complete sentences.

To solve this problem, you don’t need to fine-tune these models. Instead, the better solution is to pass some knowledge along with the question. Let me give you an example.

We're "what is the top speed?" to ChatGPT:

Both answers are correct, however, the second one is more useful for the customer.

The next challenge here is to get the “knowledge”, and use it to set the context for the LLM, along with the question. For this, we can use a retriever. Imagine you’ve database of all possible information about the car. You can perform a lexical search and find the rows that might have the answer. Then, pass these rows along with the customer’s question to the LLM.

Here's how the complete flow will look like —

In the above example, we scanned the DB for the words "top" and "speed", concatenated the results together. For a better experience, we also added some instruction (to keep the answer concise) with the original question the customer asked. Put everything together, passed to the LLM and boom — we have the answer.

Indexing

In this exercise, we’ll build a simple bot that answers questions about the Mercedes-Benz A-Class Limousine.

Get their brochure here

Now companies don't have such pristine tables about the product information. They are available in the form of documents like the one above.

So, the first step is to create a database or a search index. To extract the text, we’ll use PyPDF2:

pip install PyPDF2

from PyPDF2 import PdfReader
import csv

SOURCE_FILE = "The Mercedes-Benz A-Class Limousine.pdf"
OUTPUT_CSV = "Mercedes-Benz-A-Class-Limousine.csv"

reader = PdfReader(SOURCE_FILE)

n_pages = len(reader.pages)

header = False # file is empty, so no header row is present
for n in range(n_pages):
    page = reader.pages[n]
    text = page.extract_text()

    with open(OUTPUT_CSV, "a+") as f:
        writer = csv.DictWriter(f, fieldnames=["page", "content"])
        if not header:
            writer.writeheader()
            header = True # header row added

        writer.writerow({"page": n + 1, "content": text.lower()})
        f.close()

page	content
1	the new mercedes-benz a-class limousine.
2	*please read the disclaimer2 nearest showroom contact us live chat design technology ...

This creates a CSV file. To KISS, we'll use this CSV file + pandas as our index with our own version of retriever.

Retriever

Let's create a class Retriever which will be responsible for retrieving results based on customer query. The class will have the following functionalities:

Removing stopwords Stopwords are common words like "a", "the", "in", etc., that don’t contribute to the search; negatively impact it more often than not. For example, if someone asks you "what are the top features of the car", you'll ignore most of the words in the question, and only look for the word "features".

stopwords = [...] 
return [word for word in words if word not in stopwords]

Filtering top rows Once we have extracted all the rows with matching keywords, we'll prioritise those with higher frequency of the keywords. This also becomes important considering the token limit of LLMs.

df = pd.read_csv('Mercedes-Benz-A-Class-Limousine.csv')

# add a column frequency which will contain the sum of frequencies of keywords words
df["freq"] = sum([df['content'].str.count(k) for k in keywords])

# sort the dataframe by frequency of keywords
rows_sorted = df.sort_values(by="freq", ascending=False)

Retrieve The actual retrieve function which will be responsible for giving us the context based on customer query.

query = "top speed"

# select all rows where the content column contains the words "top" or "speed"
df = df.loc[df['content'].str.contains("top|speed")]

return f"{''.join(top_rows['content'])}"

Note that this is a very basic implementation of a retriever. The standard practice is to use something like ChromaDB or Azure Cognitive Search for indexing and retrieving.
Also, you can get the complete list of stop words this GitHub gist.

Let's put everything into the class:

import pandas as pd


class Retriever:
    def __init__(self, source, searchable_column):
        self.source = source
        self.searchable_column = searchable_column
        self.index = pd.read_csv(source)

    @property
    def _stopwords(self):
        return [
            "a",
            "an",
            "the",
            "i",
            "my",
            "this",
            "that",
            "is",
            "it",
            "to",
            "of",
            "do",
            "does", 
            "with",
            "and",
            "can",
            "will",
        ]

    def remove_stopwords(self, query):
        words = query.split(" ")
        return [word for word in words if word not in self._stopwords]

    def _top_rows(self, df, keywords):
        df = df.copy()
        df["freq"] = sum([df[self.searchable_column].str.count(k) for k in keywords])
        rows_sorted = df.sort_values(by="freq", ascending=False)
        return rows_sorted.head(5)

    def retrieve(self, query):
        keywords = self.remove_stopwords(query.lower())
        query_as_str_no_sw = "|".join(keywords)

        df = self.index
        df = df.loc[df[self.searchable_column].str.contains(query_as_str_no_sw)]
        top_rows = self._top_rows(df, keywords)

        return f"Context: {''.join(top_rows['content'])}"

Now, we can easily instantiate this class :

r = Retriever(
   source="Mercedes-Benz-A-Class-Limousine.csv", searchable_column="content"
)
context = r.retrieve(query)

LLM

The next step is to pass this context to an LLM. We'll use SOTA gpt-3.5-turbo model. You'll need an OpenAI API key for this, which you can get here.

Here's a sample OpenAI API Request:

OAI_BASE_URL = "https://api.openai.com/v1/chat/completions"
API_KEY = "sky0urKey"

CONTENT_SYSTEM = "You're a salesman at Mercedes, based on the query, create a concise answer."

def llm_reply(prompt: str) -> str:
    data = {
        "messages": [
            {"role": "system", "content": CONTENT_SYSTEM},
            {"role": "user", "content": prompt},
        ],
        "model": "gpt-3.5-turbo",
    }

    response = requests.post(
        OAI_BASE_URL,
        headers={"Authorization": f"Bearer {API_KEY}"},
        json=data,
    ).json()

    reply = response["choices"][0]["message"]["content"]

    return reply

>>> llm_reply("hello")
"Hello! How can I assist you today?"

messages is a list of dictionaries. Each dictionary has two keys — role and content. The "system" role sets a role for the bot, and works like an instruction. We'll pass our messages under role "user".

Let's ask our first question about the A Class Limousine:

query = "What is the size of alloy wheels?"

r = Retriever(
   source="Mercedes-Benz-A-Class-Limousine.csv", searchable_column="content"
)
context = r.retrieve(query)

reply = llm_reply(f"{context}\nquery:{query}")

>>> reply
"The size of the alloy wheels is 43.2 cm (17 inches)."

🎉 Congrats! You successfully created your a GPT-powered bot from a PDF file.

FastAPI application

Let's expose our function as an endpoint using FastAPI.

We're also mounting a static directory, and rendering templates to integrate this API with an UI.

`requirements.txt`

fastapi==0.78.0
pandas==1.5.3
pydantic==1.9.1
requests==2.27.1
uvicorn==0.17.6

`bot.py`

import pandas as pd
import requests

OAI_BASE_URL = "https://api.openai.com/v1/chat/completions"
API_KEY = "sky0urKey"
CONTENT_SYSTEM = "You're a salesman at Mercedes, based on the query, create a concise answer."

class Retriever:
    ...

def llm_reply(prompt: str) -> str:
    ...

`main.py`

from fastapi import FastAPI, Request
from fastapi.templating import Jinja2Templates
from fastapi.staticfiles import StaticFiles
from bot import Retriever, llm_reply
from pydantic import BaseModel


class ChatModel(BaseModel):
    query: str


templates = Jinja2Templates(directory="./")

app = FastAPI()

app.mount(
    "/static",
    StaticFiles(directory="./", html=True),
    name="static",
)


@app.get("/")
def root(request: Request):
    return templates.TemplateResponse("index.html", context={"request": request})


@app.post("/chat")
def chat(cm: ChatModel):
    query = cm.query

    r = Retriever(
        source="Mercedes-Benz-A-Class-Limousine.csv", searchable_column="content"
    )
    context = r.retrieve(query)
    prompt = context + "\n" + "Query:" + query
    return llm_reply(prompt=prompt)

We will use uvicorn to run the app:

uvicorn main:app --port 8080 --reload

You can go to localhost:8080/docs and send a sample request from Swagger UI or simply send a curl request:

curl -X 'POST' \
  'http://127.0.0.1:8080/chat' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "query": "What is the size of alloy wheel?"
}'

Now that the API is live, you can connect it to your frontend. For this tutorial, I used this beautiful design on codepen (Credits: abadu).

Demo

Your browser does not support the video tag.

Link to the complete source code is attached below.

Challenges & Next steps

Retriever: As we discussed, there's scope of improvement in the retriever. For e.g., you can implement a function for spell check, or integrate the whole thing with some cloud based service like Azure Cognitive Services.
Ring fencing: Improve the prompt/system instructions so that the bot is constrained to answer only Mercedes-related questions, and politely declines other questions.
Chat history: The current bot doesn't send historical messages. Also, it doesn't recognise follow-up questions. For example, I should be able to ask "what other devices" (follow-up question) after asking "can i connect my iphone?". Hence, you should identify follow-up questions, and also send chat history.

References & Further Readings

🚀 5 Python One-Liners You Should Know

Adarsh Punj — Tue, 01 Aug 2023 17:50:35 +0000

Format JSON

cat file.json | python -m json.tool

The | operator is a "pipe" between the two commands, hence the output of cat is sent to Python's json.tool

Save a string to a file

print("Hello, World!", file=open('file.txt', 'w'))

The default value of file is sys.out, which leads it to print to the console.

Hidden user input

import getpass
 pwd = getpass.getpass("ENTER PASSWORD:" )

getpass is a built-in Python library

Validate spellings [EN]

Let's see if the word "hello" is a valid English word.

if "hello".casefold() in open('/usr/share/dict/words/').read(): return True

Most of the unix-based machines have a "words" file in the system that is basically the English dictionary with 200k+ words. We can check if the given word exists in the dictionary to validate the spelling. The name and location of this file might be different for you.

Start a quick, local server

python -m http.server 8000

This starts a local server on port 8000 from the current working directory

Source: Snekletter; weekly code nuggets.

Decorators in Python

Adarsh Punj — Tue, 01 Aug 2023 05:30:00 +0000

For the latest version, read this article at Pythongasm

Introduction

Imagine you've a cake machine that produces really nice cakes. Now, all of a sudden your customers want you to decorate your cakes with a cherry. Fair demand but if I was you, I wouldn't want to disassemble my cake machine to add this feature of putting cherries on the top.

This is not just because I'm too lazy to revisit the internals of the machine (at least this is not the sole reason) but because doing this will violate many engineering principles:

Coupling — I shouldn't do things that make the system tightly coupled to each other
Single responsibility — The cake machine has single responsibility; and it should continue to be its sole responsibility
Flexibility — If the customers will ask for berries instead of cherries, I will have to do it all over.

We can address all these issues if we just make one machine, which will just decorate the cake produced by the cake machine with a cherry. Let's call it Decorator machine.

It doesn't only solve aforementioned problems but now you've a portable machine which you can use for decorating other products with cherries as well.

Decorator is a design pattern in Python that reduces code duplication and increases decoupling.

Usage

I've been working on a system that has a lot of compute intensive machine learning functions:

@cpuseconds
def train():
    """
    function to train a model
    """
    …

@cpuseconds
def predict(data, model):
    """
    function to predict using a model
    """
    …

We want to see how much of CPU seconds (code execution time on the machine) are being consumed by each customer and on what functionality.

As we already saw in our cake-machine example, editing these serene functions is not a good idea. So let us make a decorator that does that.

import time

def cpuseconds(func):
    """
    decorator method
    to calculate time elapsed
    in execution of func
    """
    def wrapper(*args, **kwargs):

        start_time = time.time()
        response = func(*args, **kwargs)
        time_elapsed = time.time()-start_time

        with open('resource-util.txt','a+') as f:
            f.write(f"{func.__name__},{time_elapsed}\n")

        return response

    return wrapper

Now, we can just decorate the existing functions with this decorator functions:

@cpuseconds
def train():
    …

@cpuseconds
def predict(data, model):
    …

When you run these functions, you'll have a file with records about time utilisation of each function call.

Note that you've changed the behaviour of the function without modifying anything inside the original function. Plan and simple.

However, there's more to this story. And before we proceed further, lets print the doc string of our original functions using dunder method __doc__.

>>> train.__doc__   
decorator method
to calculate time elapsed
in execution of func

Oops. This isn't what we expected. To understand why, we need to dive a little deeper and see how decorator has been implemented in Python.

How it's made

Decorators work on the concept of first class functions. It just means that functions in Python are treated like any other object.
You can pass functions as arguments of other functions like you pass strings, integers, and any other Python object.

In the above example, by using cpuseconds decorator we're actually passing our "train" function in the function cpuseconds that actually returns another function with some added functionalities. It's almost identical but…with a cherry on the top.

So it means that @cpuseconds on top of train translates to:

train = cpuseconds(train)

We are pointing the train variable to a new function. Note that the output of the new function is a function, which hasn't been executed yet.

This means we're not actually calling our original train function, since the variable train has been reassigned to something else — something that coincidentally happens to possess some characteristics of original train function (e.g. returning the same response) but not the train function itself.

Hence, it's not surprising to see that change in train.__doc__ or even train.__name__.

@functools.wraps

Fortunately, python has a built in decorator called @functools.wraps. You can use this decorator to decorate the wrapper function. This will copy certain attributes (like __doc__, __name__, etc.) of the original function to the wrapper function, thereby preserving the info.

from functools import wraps
def cpuseconds(func):

    @wraps
    def wrapper(*args, **kwargs):
        …

    return wrapper

Now let's try again:

train.__doc__
function to train a model

@wraps also comes in handy when chaining multiple decorators.

Conclusion

This was in-depth article on workings of a decorator and its real world use cases. There are other topics we didn't touch in this article e.g., class based decorators, arguments inside decorator functions, etc.

Create a menubar app for macOS, just using Python

Adarsh Punj — Sun, 10 Oct 2021 08:49:23 +0000

Introduction

While Python is great for building a lot of things, macOS apps are certainly not one of them. I was wondering if it’s possible to build a menu bar app for macOS using Python. I found out that it’s not only possible — it’s “ridiculously uncomplicated”.

In this tutorial, we’ll be building a realtime macOS app for stock prices — all using Python.

Let’s start by installing the dependencies.

rumps
requests
py2app

Rumps

Rumps generates PyObjC apps (specifically menubar apps) from simple python code. To test the rumps module, run the following code:

import rumps
def hello(sender):
    print(f"Hello from {sender.title}")

app = rumps.App("Hello World")
app.menu = [rumps.MenuItem("Weird Menu Item",callback=hello)]
app.run()

The hello() function is executed when the menu item is clicked.

>>> Hello from Weird Menu Item

To add more menu items, we can just add more elements to the app.menu list. The argument sender represents the MenuItem that has set the callback.

A cleaner way to get the same result is by using rumps.clicked decorator:

@rumps.clicked("Weird Menu Item")
@rumps.clicked("Saner Menu Item")
def hello(sender):
    print(f"Hello from {sender.title}")

app = rumps.App("Hello World")
app.run()

We will stick to the decorators approach for rest of the tutorial.

Stock price data

There are many sources to get stock quotes. We will be using Finnhub’s API (auth key based, free to use).

GET https://finnhub.io/api/v1/quote?symbol=AAPL&token=YOUR_API_KEY

Sample response:

{
   "c":119.49,
   "h":119.6717,
   "l":117.87,
   "o":119.44,
   "pc":119.21,
   "t":1605436627
}

You can register at https://finnhub.io/register for a free API key. At the time of writing this the rate limit is 60 requests/minute.

Building the StockApp

Initiate a class named “StockApp” as a subclass of rumps.App class, and add some menu items using the rumps.clicked decorator:

import rumps
class StockApp(rumps.App):
    def __init__(self):
        super(StockApp, self).__init__(name="Stock")

    @rumps.clicked("MSFT")
    @rumps.clicked("AAPL")
    def getStock(self, sender):
        self.title = f"{sender.title}"

if __name__ == '__main__':
    StockApp().run()

Now it’s time to integrate StockApp with Finnhub API. Let’s tweak getStock() like this:

import requests
import rumps

class StockApp(rumps.App):
    def __init__(self):
        super(StockApp, self).__init__(name="Stock")

    @rumps.clicked("MSFT")
    @rumps.clicked("AAPL")
    def getStock(self, sender):
        response = requests.get(f"https://finnhub.io/api/v1/quote?symbol={sender.title}&token=YOUR_API_KEY")

        stock_data = response.json()['c']
        self.title = f"{sender.title}:{stock_data}"

if __name__ == '__main__':
    StockApp().run()

The getStock() method updates the title when you select a stock symbol from the menu.
However, we don’t want to get the price by a click event. We need a function to continuously update the price of the selected stock, say after every few seconds.

To do this rumps has a Timer class, and you can decorate a function with rumps.timer() to set a timer on the function.

@rumps.timer(1)
def do_something(self, sender):
   # this function is executed every 1 second

At the launch, we can set some default menu item, say “AAPL”. This option can be changed with a click event while the timer-decorated function will keep updating the price for the currently selected menu item.

@rumps.clicked("AAPL")
@rumps.clicked("MSFT")
def changeStock(self, sender):
   self.stock = sender.title

@rumps.timer(5)
def updateStockPrice(self, sender):
   # fetch stock quote and update title

Not to complicate this but since the app will be sending network requests, we’d need to handle API requests on a different thread so the app UI keeps running while the request is in process.

import threading

@rumps.timer(5)
def updateStockPrice(self, sender):
   thread = threading.Thread(target=self.getStock)
   thread.start()

def getStock(self):
    # code to send API request

Putting it all together

Here’s how it will look at full implementation. We have added icons, made the title more catchy, and added a functionality for user input (using rumps.Window).

import threading
import requests
import rumps

class StockApp(rumps.App):
    def __init__(self):
        super(StockApp, self).__init__(name="Stock")

        self.stock = "AAPL"
        self.icon = "icon.png"
        self.API_KEY = "YOUR_API_KEY"

    @rumps.clicked("Search...")
    @rumps.clicked("MSFT")
    @rumps.clicked("TSLA")
    @rumps.clicked("NFLX")
    @rumps.clicked("FB")
    @rumps.clicked("AAPL")
    def changeStock(self, sender):
        if sender.title!="Search...":
            self.title = f" 🔍 {sender.title}"
            self.stock = sender.title
        else:

            # Launches a rumps window for user input
            window = rumps.Window(f"Current: {self.stock}","Search another stock")
            window.icon = self.icon
            response = window.run()
            self.stock = response.text

    @rumps.timer(5)
    def updateStockPrice(self, sender):
        thread = threading.Thread(target=self.getStock)
        thread.start()

    def getStock(self):
        response = requests.get(f"https://finnhub.io/api/v1/quote?symbol={self.stock}&token={self.API_KEY}")

        if response.status_code!=200:
            self.title = "API Error."
            return

        stock_data = response.json()

        current_price = stock_data['c']
        previous_close = stock_data['pc']
        change = current_price-previous_close

        try:
            changePercentage = abs(round(100*(change/previous_close), 2))

            if change<0:
                marker = "🔻"
            else:
                marker = "🔺"

            self.title = f" {self.stock}: {str(response.json()['c'])} {marker}{changePercentage}%"

        # Finnhub returns 0 for non-existent symbols
        except ZeroDivisionError:
            self.title = "Invalid symbol, set to AAPL"
            self.stock = "AAPL"

if __name__ == '__main__':
    StockApp().run()

To run the app you will need to have an “icon.png” file in the same directory. You can download it from the links below or else just remove the icon from the program. Also, don’t forget to assign your Finnhub API key to self.API_KEY.

Download icon.png
Download icon.icns
Converting it to .app
Now that the app is ready, we just need to generate a shippable macOS app. We can use py2app to do this.
You need to have a setup.py file in the same directory. Other than that you can also add an icon for your application. macOS app icons have the file type .icns

StockApp
  |__ app.py
  |__ setup.py
  |__ icon.png
  |__ icon.icns

The setup.py file:

from setuptools import setup

APP = ['app.py']
DATA_FILES = ['icon.png']
OPTIONS = {
    'argv_emulation': True,
    'iconfile': 'icon.icns',
    'plist': {
        'CFBundleShortVersionString': '1.0.0',
        'LSUIElement': True,
    },
    'packages': ['rumps','requests']
}

setup(
    app=APP,
    name='Stock',
    data_files=DATA_FILES,
    options={'py2app': OPTIONS},
    setup_requires=['py2app'], install_requires=['rumps','requests']
)

Finally run the setup:

python setup.py py2app

You can find your in the newly-created dist folder. Open the app, and see it in action!
If you are getting an error at the runtime, launch the app via terminal so you can go through the traceback.

open Stock.App/Contents/MacOS/Stock

Conclusion

As we have seen it’s easy to create simple menu bar apps like this. A whole lot of things can be built on top of this as it gives you the power to trigger Python functions so easily. We can make a music controller, a server monitor, to see if a program is running, stopwatch, CPU meters, flight position trackers, Internet speed tests, just to name a few.

Introduction to threading and multiprocessing: Concurrency & Parallelism in Python

Adarsh Punj — Sat, 19 Jun 2021 20:24:31 +0000

Introduction

Most of us have come across terms like multithreading, parallel processing, multiprocessing, concurrency, etc., though each of these terms has its own meaning. In a broader sense, we need these because we want to avoid some kind of latency (or have an illusion of doing so) in the execution of regular programs.

To do this, we try to write code that doesn’t necessarily runs in order (non-sequential) and all this boils down to two different concepts — concurrency and parallelism.

Concurrency

I came across concurrency when I was trying to download ~5000 images from the web. I had collected image URLs from Flickr, and these images had to be passed on to a team doing annotation (labelling).

This is how a sequential program to download images would look like:

import requests

def download_image(URL, filename):
    image = requests.get(URL)

    with open(f'{filename}.png','wb') as f:
        f.write(image.content)


flickr_URLs = [
    'https://live.staticflickr.com/6022/5941812700_0f634b136e_b.jpg',
    'https://live.staticflickr.com/3379/3492581209_485d2bfafc_b.jpg',
    'https://live.staticflickr.com/7309/27729662905_e896a3f604_b.jpg',
    'https://live.staticflickr.com/8479/8238430093_eb19b654e0_b.jpg',
    'https://live.staticflickr.com/5064/5618934898_659bc060cd_b.jpg',
    'https://live.staticflickr.com/3885/14877957549_ccb7e55494_b.jpg',
    'https://live.staticflickr.com/5473/11720191564_76f3f56f12_b.jpg',
    'https://live.staticflickr.com/2837/13546560344_835fc79871_b.jpg',
    'https://live.staticflickr.com/140/389494506_55bcdc3664_b.jpg',
    'https://live.staticflickr.com/5597/15253681909_0cc91c77d5_b.jpg',
    'https://live.staticflickr.com/1552/24085836504_3d850f03e7_b.jpg',
    'https://live.staticflickr.com/7787/26655921703_ee95e3be8e_b.jpg',
    'https://live.staticflickr.com/423/32290997650_416303457b_b.jpg',
    'https://live.staticflickr.com/4580/37683207504_053315d23f_b.jpg',
    'https://live.staticflickr.com/3225/2454495359_92828d8542_b.jpg',
    'https://live.staticflickr.com/7018/6640810853_22634c6667_b.jpg',
    'https://live.staticflickr.com/7681/17285538029_363c8760ea_b.jpg',
    'https://live.staticflickr.com/7630/16622584999_0654c8d564_b.jpg',
    'https://live.staticflickr.com/6160/6207543047_da2c66c2f6_b.jpg',
    'https://live.staticflickr.com/2921/14251116921_a97d7a46ce_b.jpg'
]

for url in flickr_URLs:
    filename = url.split(‘/')[-1]
    download_image(url, filename)

It does the job but we spent most of the time waiting for the source URLs to respond. When we scale this program to 5,000 images this “wait time” becomes humongous.

The above program sends a request to a URL, waits until the image loads (gets response from server), writes it to disk, and only then sends a new request until the list exhausts.

However, rather than waiting for the first URL to load, shouldn’t we send a new request in the meantime? Once we receive some response from a previously sent request, we can write the corresponding image to the disk. By doing this we are not letting the latency block our main program.

We can achieve this by starting a new “thread”, along with the main thread using built-in Python module called threading.

Here’s how you create a thread:

thread = threading.Thread(download_image, args=[url, filename])
thread.start()

So that’s one thread. You need to create many threads, so let’s loop over. Here’s what’s the threaded version of this program would look like:

import threading
import requests

def download_image(URL, filename):
    ...

flickr_URLs = [...]

threads = []
for url in flickr_URLs:
    filename = url.split('/')[-1]
    thread = threading.Thread(target=download_image, args=[url, filename])
    thread.start()
    threads.append(thread)

for thread in threads:
    thread.join()

We call .join() on a thread to join it to the main thread — telling Python to wait for a thread to terminate before moving further down in the file.

In this program, we created 20 threads. But how many threads are too many? If we have 5,000 URLs, should we start 5000 threads?

First of all, you should know that you can start multiple threads but they won’t be running simultaneously. It’s just that while one thread is waiting for some I/O operation, another one starts working in the meantime. It might look like the juggler is juggling two balls but in reality, at any given point he only has one ball in his hand. Source: Library of Juggling

Since your OS continuously switches thread to thread, deciding which one should run at a given time, managing hundreds of threads will eat up a big chunk of resources.

So a simple workaround is this:

MAX_THREADS = 10
threads = []
for url in flickr_URLs:
    if len(threads)>MAX_THREADS:
        for thread in threads:
            thread.join()

        threads = []

    filename = url.split('/')[-1]   
    thread = threading.Thread(target=download_image, args=[url, filename])
    thread.start()
    threads.append(thread)

for thread in threads:
    thread.join()

We’re only starting MAX_THREADS at once, waiting for them to terminate (we're doing that by calling.join()), and then starting the next MAX_THREADS threads.

However, there’s a modern way of doing this which we will see later in this article.

Some implementations of Python like PyPy, IronPython can run multiple threads simultaneously but this isn’t the case with the default implementation, that’s CPython.

Notice how we didn’t need CPU power to speed up this task of downloading images. These are I/O bound tasks — it’s like cooking food, for example.  If you’re preparing a dish, and at some point you need to preheat your microwave oven, you won’t be looking for more manpower to speed up your cooking. You’re better off utilising the preheating time into…maybe chopping down some veggies.

Python cutting down veggies while the oven finishes up preheating

That’s concurrency. But what if a task is bottlenecked by the CPU, rather than networking and IO? That brings us to parallelism.

Parallelism

Now suppose you’re done with cooking, and it’s time to do the dishes. Can you apply the concept of concurrency here? Pick the knife and start cleaning it, switch over to the bowl you used for pouring milk, start washing it, and then move on to the plates, then back to the knife you were washing some time ago.

At best, it won’t make any difference to the execution time of this task. And in most of the cases, doing this will make the process slower as you are taking some time in switching back and forth to different utensils (yes, multithreading can also slow down tasks).

If you’re looking to expedite this task, you need manpower. Some friend who washes the bowl while you cleanse knives and forks — more friends, the better.

This is how a CPU heavy task looks like, where you can use multiple CPU cores to speed up the task. Let’s try to apply this using Python.

We’re trying to find out product of prime numbers upto a number n using a function productOfPrimes, such that productOfPrimes(10) -> 210 (product of 2, 3, 5, 7)

import time

def productOfPrimes(n):

    ALL_PRIMES_UPTO_N = []

    for i in range(2, n):

        PRIME = True
        for j in range(2, int(i/2)+1):
            if i%j==0:
                PRIME = False
                break

        if PRIME:
            ALL_PRIMES_UPTO_N.append(i)


    print(f"{len(ALL_PRIMES_UPTO_N)} PRIME NUMBERS FOUND")

    product = 1
    for prime in ALL_PRIMES_UPTO_N:
        product = product*prime

    return product

init_time = time.time()
LIMITS = [50328,22756,39371,44832]

for lim in LIMITS: 
    productOfPrimes(lim)

fin_time = time.time()
print(f"TIME TAKEN: {fin_time-init_time}")

Nested loops, lots of division — pretty heavy on the CPU. The execution took ~10 seconds on 1.1 GHz quad core Intel i5 processor. However, out of 4 cores, we just used one.

To manage multiple cores and processes, we use this module called multiprocessing in python.

Let’s see if multiprocessing improves this result. Syntactically, it’s quite similar to how we started threads:

import multiprocessing

def productOfPrimes():
    …

if __name__ == "__main__":  
    processes = []
    LIMITS = [50328,22756,39371,44832] 

    for lim in LIMITS:
        process = multiprocessing.Process(target=productOfPrimes, args=[lim])
        process.start()
        processes.append(process)

    for process in processes:
        process.join()

This does the same thing in a little over 4 seconds. productOfPrimes was simulatenously executed on multiple cores available in the CPU.

Now let's talk a bit about numbers. How many processes can a quad-core CPU can execute simultaneously? Shouldn't it be only 4? Yes, but that doesn't mean the OS can't hold more than 4 processes in memory. There's a difference in executing processes and just holding them in memory.

Run this bash command to see the number of processes running on your system:

ps -e | wc -l

So if you start 20 processes from Python, it won't throw an error saying you don't have enough cores. The OS will just manage these 20 processes over whatever cores are available.

Pool of Threads and Processes

We’ve seen how we can implement the concepts of concurrency and parallelism using threading and multiprocessing modules. However, we’ve a sexier, more Pythonic way of doing this using the concurrent.futures module (ships with Python).

import concurrent.futures

def download_image(URL, filename):
    ...

flickr_URLs = [...]
with concurrent.futures.ThreadPoolExecutor() as executor:
    results = executor.map(download_image, flickr_URLs)

for result in results:
        print(result)

We can do something similar using ProcessPoolExecutor:

import concurrent.futures

def productOfPrimes(n):
    ...

LIMITS = [...]

if __name__ == "__main__": 
    with concurrent.futures.ProcessPoolExecutor() as executor:
        results = executor.map(productOfPrimes, LIMITS)

for result in results:
    print(result)

Threads, Processes, OS

Technically, threads run inside a process. When we create 4 threads, they share the same process, thus the same memory, and a lot of other OS level stuff (process control block, address space, etc.). The same is not true for processes. Each process has memory space of its own and run independently.

On the other hand, processes can run simultaneously — unlike in multithreading where the OS just keeps switching over and over to manage latency inside the same process.

Conclusion

We’ve seen how we can implement concurrency and parallelism in Python which are fundamentally very different, and have use cases of their own. There are more things to talk about like problems with threading, GIL, asynchronicity, etc.

Optimization: Instagram in 4000 bytes of memory

Adarsh Punj — Thu, 30 Jan 2020 10:14:56 +0000

A few days ago, I came across this project: St4kOverFlow. This is Stack Overflow micro optimized into 4k bytes.

I found this idea really interesting and tried to implement the same with Instagram. I started making an Instagram-like webpage, though with limited features.

You can search for profiles, explore hashtag pages, zoom-in images, etc.

DEMO

At first, all I wanted was to get it working. So I wrote an HTML document in around 200 lines (HTML/CSS/JS), without worrying about memory usage.

The page was ready and working expected.

Now it was time to make it smaller.

As a good first step, I started with renaming variables. profileURL became p. Almost all of the variables in the document are single character alphabets (Yes, terrible). It includes the names of attributes like id, class, etc. Doing this reduced the file size but it was far from small.

Styling (CSS):

CSS takes a lot of memory (comparatively). I tried to use as little CSS as possible. Also, In-line styling is far more memory-efficient. In fact, I wrote a few CSS attributes multiple times rather than defining a reusable CSS class, and it ended up saving me a few bytes.

A little change in styling makes a significant impact on memory.
Changing {text-align:left;float:left;} to {text-align,float:left;} saves 8 bytes, and there were multiple instances where this was done. So, I was able to fairly good amount of space in the CSS part.

JavaScript:

After renaming variables there was not a lot of scope to save space in the scripting part. I did some improvements in the logical part, though. I read this article JavaScript Semicolon Insertion: Everything you need to know, and it helped me omit a few semi-colons out of the document. However, most of them had to be restored at the end (as the document had to be contained in one line).

SVG:

There was no way I could use logos available online. The logo itself would consume 25% from the memory.

I was able to create my own version, thanks to the Instagram logo that is just a rectangle and two circles.
This wasn't the best Instagram logo but it was fine considering the fact that it was only 234 bytes.

<svg width="32" height="32" style="stroke:black;fill: white;stroke-width:2;"> <rect x="2" y="2" rx="6" ry="6" width="24" height="24"/> <circle cx="14" cy="14" r="6"/> <circle cx="21" cy="7" r="1" stroke-width="1" fill="black" /></svg>

I referred to this w3 article, and it was enough.

Finally, I wanted to attach a link to the Github repository. Luckily u.nu exists, and it shortened the long-ass GitHub URL to just u.nu/h1fw.

Issues and improvements:

The current webpage displays just 12 latest posts from a profile (which can be fixed with some tweaks), and there is no support for video.
Other than this, the document still has more room for optimization.

I hope you enjoyed reading this article.
Do read Python One-Liners You Should Know, if you are interested in Python.

Forem: Adarsh Punj

Memory management in Python: Interning & Optimization

Introduction

is operator

IDLE vs .py file

Interning responsibly

Conclusion

🔔 Desktop notifications in 10 lines of Python code

Introduction

MacOS

Linux

Making it cross-platform

Conclusion

🔥 Create GPT-3 bot out of your own PDFs using Python

Introduction

Context + LLM

Indexing

Retriever

LLM

FastAPI application

requirements.txt

bot.py

main.py

Demo

Challenges & Next steps

References & Further Readings

🚀 5 Python One-Liners You Should Know

Format JSON

Save a string to a file

Hidden user input

Validate spellings [EN]

Start a quick, local server

Decorators in Python

Introduction

Usage

How it's made

@functools.wraps

Conclusion

Further Readings

Join snekletter 🚀

Create a menubar app for macOS, just using Python

Introduction

Rumps

Stock price data

Building the StockApp

Putting it all together

Finally run the setup:

Conclusion

Further Readings and At)tributions:

Introduction to threading and multiprocessing: Concurrency & Parallelism in Python

Introduction

Concurrency

Parallelism

Pool of Threads and Processes

Threads, Processes, OS

Conclusion

Further Readings and Attributions

Optimization: Instagram in 4000 bytes of memory

DEMO

Styling (CSS):

JavaScript:

SVG:

Issues and improvements:

View code on Github

`is` operator

IDLE vs `.py` file

`requirements.txt`

`bot.py`

`main.py`