Forem: Derrick Zziwa

Safe Threading with SQLAlchemy in FastAPI

Derrick Zziwa — Fri, 14 Feb 2025 19:07:32 +0000

When building FastAPI applications that use SQLAlchemy for data persistence, you often need to perform background tasks—such as updating records or processing data—without blocking the main request. A common approach is to offload work to background threads. However, threading in Python comes with its own challenges, especially regarding SQLAlchemy sessions.

In this article, we are going to understand the limitations of using threads with SQLAlchemy sessions, and then we’ll discuss how to safely address these challenges.

What Are Threads?

Threads are a way to perform multiple operations concurrently within a single process. They allow your application to handle tasks in parallel—such as processing I/O-bound operations—without waiting for one task to finish before starting another.

Usage in FastAPI:

In a FastAPI application, you might use threads to run background tasks so that the main API request isn’t delayed. For example, sending an email, updating a record, or doing non-critical processing.

Limitations of Threads

Not Truly Parallel (CPython):
Due to the Global Interpreter Lock (GIL), threads in CPython are most effective for I/O-bound tasks. CPU-bound tasks might not gain much speed-up.

Shared State and Concurrency:
Threads share the same memory space, which can lead to complications when mutable shared state is not managed carefully.

Thread Safety Issues:
Not every component is thread-safe. In particular, SQLAlchemy sessions are not thread-safe—using the same session in multiple threads can lead to unpredictable behavior and errors.

The Solution: Isolate SQLAlchemy Sessions in Threads

Instead of passing the existing session to a background thread, you should create a new session within that thread. This ensures that each thread has its own isolated session and avoids the discussed pitfalls.

Implementation Steps

Define a Session Factory:
In your application, use SQLAlchemy’s sessionmaker to create a session factory. For example:

# database.py
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker

DATABASE_URL = "postgresql://user:password@localhost/dbname"
engine = create_engine(DATABASE_URL)
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)

Use the Factory in Background Tasks:

When you need to run a background task (e.g., updating a record), create a new session inside that task:

# background_tasks.py

from database import SessionLocal
from config.logger_config import logger

def update_record_background(param):
    db = SessionLocal()  # Create a new session for this thread
    try:
        # Perform your database operations here
        # e.g., update a record or commit a transaction
        db.commit()
    except Exception as e:
        db.rollback()
        logger.error(f"Error in background update: {e}")
    finally:
        db.close()

Launch the Background Task:

In your FastAPI endpoint or wherever the background task is needed, start a new thread without passing the main session:

import threading
from background_tasks import update_record_background

# When triggering the background task:
threading.Thread(target=update_record_background, args=(param_value,)).start()

Benefits of This Approach

Thread Safety:
Each thread operates on its own session, eliminating race conditions and ensuring data consistency.

Isolation:
Background tasks do not interfere with the main request’s session, preventing unexpected state changes.

Scalability:
This approach makes your application easier to scale, as each task uses an independent session.

We’ve identified the core issues when sharing SQLAlchemy sessions between threads: thread safety concerns and unexpected state changes. The solution is to create a new session for each background task, ensuring isolation and stability in your FastAPI applications.

Implementing this pattern not only resolves errors like IllegalStateChangeError but also leads to cleaner, more maintainable code.
This strategy is applicable to any FastAPI application using SQLAlchemy and can be adapted for more complex asynchronous tasks using task queues if needed.

OpenSource Models Surpass GPT3.5 on Several Benchmarks

Derrick Zziwa — Sun, 23 Jul 2023 16:34:05 +0000

Unleashing the Power of Open Source AI to Redefine Possibilities

Stability.AI, in an audacious move to push the boundaries of AI capabilities, proudly presents FreeWilly - a groundbreaking open-source project aimed at challenging the mighty GPT-3.5. Developed in collaboration with CarperAI lab, FreeWilly emerges as a formidable Large Language Model (LLM) with exceptional reasoning abilities, poised to revolutionize the world of natural language processing.

Empowering Open Source AI Research

FreeWilly's mission is to democratize AI research and foster open collaboration. Both FreeWilly1 and its successor FreeWilly2 are released under open-source licenses, encouraging the AI community to explore, innovate, and contribute to the future of language models. As staunch advocates of open access, Stability.AI is committed to making advancements in AI accessible to all.

A Formidable Challenge to GPT-3.5

Built on the strong foundation of the LLaMA 65B and LLaMA 2 70B models, FreeWilly1 and FreeWilly2 were carefully fine-tuned using a new synthetically-generated dataset. Stability.AI's team employed the Supervised Fine-Tune (SFT) approach in standard Alpaca format to refine these models to their peak performance.

Red-Teaming and Community Collaboration

Stability.AI leaves no stone unturned to ensure the safety and ethics of FreeWilly. Thorough internal red-teaming has been conducted to ensure the models remain polite and harmless. However, the community's participation in the red-teaming process is encouraged, as diverse perspectives can help strengthen the models and identify potential areas of improvement.

A Data-Driven Journey

Drawing inspiration from Microsoft's trailblazing "Orca: Progressive Learning from Complex Explanation Traces of GPT-4" paper, Stability.AI took a novel approach to data generation. By creating a variant dataset with 600,000 meticulously curated high-quality instruction data points from Enrico Shippole's datasets, FreeWilly sets new standards in training efficiency. Rigorous filtering and careful curation ensured that evaluation benchmarks did not influence the models, allowing for a fair and robust evaluation.

Unleashing Unprecedented Performance

Internal evaluation using EleutherAI’s lm-eval-harness, along with AGIEval, reveals FreeWilly's exceptional performance in various domains. From intricate reasoning to linguistic subtleties, FreeWilly showcases unparalleled proficiency in tasks like Law and mathematical problem-solving.

Leading the Open LLM Leaderboard

FreeWilly's exceptional results were not only validated by Stability.AI researchers but also independently reproduced by Hugging Face on July 21st, 2023. These benchmarks catapult FreeWilly to the forefront of the open LLM leaderboard, making it a powerful contender in the AI landscape.

An Open Future of Possibilities
_

FreeWilly1 and FreeWilly2 mark a transformative step forward in the world of open-source Large Language Models. By enhancing natural language understanding and enabling complex tasks, these models offer limitless possibilities and applications. Stability.AI extends its heartfelt gratitude to its team of passionate researchers, engineers, and collaborators whose unwavering dedication led to this remarkable milestone.

Source: https://lnkd.in/dNcY_9jW

The Dangers of Using the Same Database for Development and Production

Derrick Zziwa — Sun, 23 Jul 2023 07:42:00 +0000

In the world of software development, maintaining a clear distinction between the development and production environments is crucial for the success, stability, and security of an application. Unfortunately, some organizations overlook the importance of using separate databases for development and production, unwittingly exposing themselves to a myriad of potential dangers. In this article, we will delve into the various risks associated with using the same database for both development and production purposes, as well as explore the best practices for establishing a clear separation between these environments.

1. Data Integrity Risks
When development and production share the same database, the chances of data corruption and accidental data deletion increase significantly. During the development process, developers may run tests, experiments, or updates that can unintentionally affect critical data in the production environment. A bug or error during development could lead to the loss of valuable production data, leading to financial losses and customer dissatisfaction.

2. Security Vulnerabilities
Combining development and production databases can expose your application to security vulnerabilities. In the development phase, developers often have broader access to the database to facilitate testing and debugging. However, this level of access should not be granted in a production environment, where data privacy and security are of utmost importance. A compromised development database can provide a pathway for attackers to gain unauthorized access to sensitive production data, leading to potential data breaches and compliance violations.

3. Performance Bottlenecks
Sharing a single database between development and production can lead to performance bottlenecks, affecting both environments. In the development phase, developers may conduct extensive testing, run heavy queries, and perform resource-intensive tasks that impact the performance of the production application. Additionally, large-scale testing in the development environment may lead to limited resources for the production environment, causing performance degradation and reduced responsiveness for end-users.

4. Impact on Continuous Integration/Continuous Deployment (CI/CD)
For organizations practicing CI/CD, sharing a database between development and production can hinder the smooth integration and deployment process. Frequent changes in the development environment might not be compatible with the production database schema, leading to deployment failures and application downtime. A separate database for each environment allows developers to test new features and updates without affecting the stability of the production system.

5. Difficulty in Debugging and Troubleshooting
When development and production data coexist in the same database, troubleshooting and debugging become complicated. Issues faced during development may not be reproducible in the production environment, or vice versa, due to data discrepancies. As a result, identifying the root cause of problems becomes more challenging and time-consuming, delaying the resolution process and increasing maintenance costs.

Best Practices for Database Separation

To mitigate the dangers of using the same database for development and production, organizations should adopt best practices for creating clear separation between these environments:

Employ a Staging Environment: Introduce a staging environment that closely resembles the production setup. This environment acts as an intermediary step between development and production, allowing comprehensive testing and validation before deploying changes to the production database.
Implement Access Control: Restrict access rights to the production database, granting only necessary permissions to specific individuals or roles. Developers should have access to the development database, but this should be limited in scope and different from the production environment.
Regular Data Backups: Establish a robust data backup strategy for both development and production databases. Regularly backing up data ensures that in the event of data loss or corruption, recovery can be made without substantial damage.
Automation and Version Control: Embrace automation and version control tools to manage database schema changes. Automated processes reduce the risk of human error and ensure consistency across environments.

Conclusion
The dangers of using the same database for development and production cannot be underestimated. From data integrity risks and security vulnerabilities to performance bottlenecks and deployment challenges, the potential consequences are severe. By implementing best practices for database separation, organizations can safeguard their data, maintain a secure application environment, and optimize the development and production workflows, ultimately leading to improved overall system reliability and performance.

Why You Should Learn Flutter, RIGHT NOW!!

Derrick Zziwa — Thu, 27 Apr 2023 08:02:49 +0000

WHAT IS FLUTTER?

Simply put, Flutter is the future of Mobile App Development. And am not kidding. Flutter is a relatively new mobile application development framework that is quickly gaining popularity in the tech world. Created by Google, Flutter has been designed to help developers build high-performance, visually stunning mobile apps for multiple platforms including Android, iOS, Linux, Windows, and Web, all with a single codebase! Amazing isn't it?

WHY LEARN FLUTTER?

Flutter offers a number of features that make it stand out from other mobile app development frameworks. Here are some of the key benefits of using Flutter for mobile app development:

1. Fast development: Flutter is designed to make the mobile app development process as fast and easy as possible. The framework comes with a rich set of pre-built widgets that can be easily customized to fit the needs of any app. This allows developers to create high-quality mobile apps in significantly less time than it would take with other app development frameworks.

2. Native Performance: One of the key benefits of Flutter is that it delivers native performance on any platform. Flutter uses the Skia Graphics Engine to render UIs directly on the device without any bridges or web views. This means your app can run smoothly and access native features and services without any performance issues or compatibility problems. Flutter also supports hardware acceleration, 60fps animations, and platform-specific gestures, giving your app a natural and responsive feel.

3. Cross-platform development: Because Flutter uses a single codebase for both Android and iOS platforms, developers can create mobile apps that work seamlessly on both platforms. This means that the app will have the same look and feel on both Android and iOS devices, reducing the need to create separate versions of the app for each platform.

4. Beautiful UI: One of the biggest advantages of using Flutter is the ability to create visually stunning UI elements. Flutter comes with a wide range of pre-built widgets that can be easily customized to create beautiful, responsive UI designs that look great on any device.

5. Open-source: Flutter is an open-source framework, which means that developers can easily access the source code and contribute to the development of the framework. This makes it a popular choice among developers who value community involvement and collaboration.

3. Light Weight: Flutter is built using the Dart programming language, which is optimized for high-performance. This means that Flutter apps run smoothly and quickly, even on older or less powerful devices. This makes it a popular choice for developers who want to create mobile apps that offer a top-quality user experience.

Flutter is quickly becoming the future of mobile app development. Its combination of fast development, cross-platform compatibility, high-performance, beautiful UI, and open-source nature make it an attractive choice for developers looking to create top-quality mobile apps. If you're a developer looking to create mobile apps for Android, iOS, Windows, or Linux platforms, consider giving Flutter a try.

You can learn more about Flutter at https://flutter.dev/