Forem: Fatima Alam

All things Kafka

Fatima Alam — Fri, 20 Mar 2026 05:10:32 +0000

Kafka Crash Course: From Concept to a Hands-On Python Project
Have you ever wondered how giant platforms like Uber, Netflix, or LinkedIn handle millions of events per second in real-time? The answer, more often than not, is Apache Kafka.

If you are a microservices developer, data engineer, or just curious about event-driven architecture, understanding Kafka is no longer optional—it's a superpower.

In this post, we will deconstruct what Kafka is, why it exists, and build a real-world "Notify Me" backend using Java.

Let's dive in!

The Problem – "Tight Coupling"
Imagine you are building a simple e-commerce application called "AllThingsStore." In the beginning, you use a synchronous, monolithic architecture where microservices talk directly to each other.

When a customer places an order:

Order Service calls Payment Service.

Payment Service calls Inventory Service.

Inventory Service calls Notification Service.

This works fine...What could go wrong??
Until Black Friday. Suddenly, thousands of orders rush in.

The Problem visualized:
Here is what happens when direct, synchronous communication fails under load.

In traditional microservices, systems use Synchronous Communication. If the "Order Service" calls the "Payment Service" directly, it must wait for a response.

The Domino Effect: If one service lags or crashes (especially during peak traffic), the entire chain freezes.

Single Point of Failure: A 10-minute outage in one service can lead to hours of backlog and business loss.

EG: When Payment Service gets overwhelmed and crashes, the entire chain freezes. The Order Service is stuck waiting for a response that will never come, leading to a catastrophic system failure and lost revenue. This is called tight coupling.

The Kafka Solution – "Event-Driven"
What if we redesigned the system so that services don't need to know about each other directly? This is where Apache Kafka enters the scene as a central message broker.

Instead of Order Service calling Payment Service directly, it simply publishes an event: "An order was placed."

Order Service (now a Producer) hands this event to Kafka and continues its work. It doesn't wait. It trusts Kafka to deliver.

Kafka then organizes this event into a logical category called a Topic (e.g., "orders"). Any other service that needs this information (now a Consumer) simply subscribes to that topic.

Key Kafka Concepts:
Producers: Services that create and send events.

Consumers: Services that read and process events.

Topics: A logical bucket or category for events.

Brokers: The actual servers that store and manage the data. Unlike traditional message brokers, Kafka persists data to disk, allowing it to be re-read (which is not the case with Rabbit MQ, Active MQ, Pub/Sub

Mastering Kafka’s Architecture
To truly harness Kafka, you must understand how it scales. Two main concepts achieve this: Partitions and Consumer Groups.

Partitions A Topic isn't just one big file. To allow for parallel processing, Kafka splits a Topic into multiple Partitions. Think of a partition as a commit log where data is only appended at the end.

This architecture enables multiple producers to write to different partitions simultaneously and multiple consumers to read from different partitions simultaneously.

Consumer Groups To read partitions in parallel, consumers are organized into Consumer Groups. Kafka automatically ensures that each partition in a topic is assigned to exactly one consumer within that group. This provides massive scalability.

Summarizing the Scale: The KRaft Architecture
Modern Kafka (v3.0+) has replaced its dependency on an external orchestration tool (Zookeeper) with a built-in system called KRaft (Kafka Raft).

This simplifies the architecture. A modern Kafka node can act as both a Broker (storing data) and a Controller (managing cluster consensus).

Hands-On Project: A Java Notify me Service for an E-Comm
Now for the best part: bringing these concepts to life! We will build a simple "Notification" simulator:

User clicks "Notify Me"
        ↓
Subscription saved (DB)

Inventory Service
item restocked
        ↓
Kafka Producer publishes event

Topic: item-restocked
        ↓
Kafka Broker
        ↓
Notification Service (Consumer)
        ↓
Fetch subscribed users
Send Email / Push Notification

Producer (Client App): Inventory Service which makes fake products and publishes them to an Products topic.

Consumer (Order Processor): Notification Service that listens to the Products topic in real-time and "processes" the new products.

Prerequisites
Kafka Setup (Local)

Install Apache Kafka

Start services:

zookeeper-server-start.sh config/zookeeper.properties
kafka-server-start.sh config/server.properties

Create topic:

kafka-topics.sh --create \
--topic item-restocked \
--bootstrap-server localhost:9092 \
--partitions 3 \
--replication-factor 1

IntelliJ Project Setup

Create Maven project

Add dependency:

<dependency>
 <groupId>org.apache.kafka</groupId>
 <artifactId>kafka-clients</artifactId>
 <version>3.6.0</version>
</dependency>

Event Model

class RestockEvent {

    private String productId;
    private int quantity;

    public RestockEvent(String productId, int quantity) {
        this.productId = productId;
        this.quantity = quantity;
    }

    public String getProductId() { return productId; }
}

Kafka Producer (Inventory Service)

When item stock becomes available.

import org.apache.kafka.clients.producer.*;

import java.util.Properties;

public class RestockProducer {

    private static KafkaProducer<String,String> producer;

    static {

        Properties props = new Properties();

        props.put("bootstrap.servers","localhost:9092");
        props.put("key.serializer",
                "org.apache.kafka.common.serialization.StringSerializer");
        props.put("value.serializer",
                "org.apache.kafka.common.serialization.StringSerializer");

        producer = new KafkaProducer<>(props);
    }

    public static void publishRestock(String productId) {

        ProducerRecord<String,String> record =
                new ProducerRecord<>("item-restocked", productId);

        producer.send(record);
    }
}

Usage inside Inventory Service

if(product.getStock() > 0){
    RestockProducer.publishRestock(product.getId());
}

Kafka Consumer (Notification Service)

import org.apache.kafka.clients.consumer.*;

import java.time.Duration;
import java.util.*;

public class RestockConsumer {

    public static void main(String[] args) {

        Properties props = new Properties();

        props.put("bootstrap.servers","localhost:9092");
        props.put("group.id","notification-service");
        props.put("key.deserializer",
                "org.apache.kafka.common.serialization.StringDeserializer");
        props.put("value.deserializer",
                "org.apache.kafka.common.serialization.StringDeserializer");

        KafkaConsumer<String,String> consumer =
                new KafkaConsumer<>(props);

        consumer.subscribe(List.of("item-restocked"));

        while(true){

            ConsumerRecords<String,String> records =
                    consumer.poll(Duration.ofMillis(100));

            for(ConsumerRecord<String,String> record : records){

                String productId = record.value();

                notifyUsers(productId);
            }
        }
    }

    static void notifyUsers(String productId){

        List<String> users =
                SubscriptionRepository.getUsers(productId);

        for(String user : users){
            System.out.println("Sending notification to " + user);
        }
    }
}

Subscription Table

When user clicks Notify Me

Subscription
------------
userId
productId
createdAt

Query used:

SELECT userId
FROM subscription
WHERE productId = ?

Example Event Flow

Inventory updated
Stock = 10

Producer sends:

Topic: item-restocked
Message: productId=iphone15

Consumer receives:

Notification service

Fetch subscribers:

user1
user2
user3

Send:

email / push notification

Why Kafka Works Well Here
Benefits:

decouples inventory and notification service
scalable fanout to millions of users
asynchronous processing
fault tolerant

Production Improvements
Real systems add:

Redis → store top subscribers
Kafka partitions → scale notifications
Email service workers
Rate limiting
Batch notifications

Consistent Hashing

Fatima Alam — Mon, 02 Mar 2026 19:31:09 +0000

Problem Statement

We need to distribute keys (data, requests, cache entries) across multiple servers.
A common approach is:
bucket = hash(key) % N
Where N is the number of servers.
The Problem?
When N changes (a server is added or removed), almost all keys get reassigned.

This causes?

Massive data movement
Cache invalidation
Network overhead
We need a way to minimize key movement when servers change.
The Solution: Consistent Hashing
Instead of modulo arithmetic, consistent hashing:
Maps both servers and keys onto a circular hash space (a ring).
Assigns each key to the next server clockwise.

This small change ensures that when servers are added or removed, only a small subset of keys move.

Example
Step 1 — Create the Hash Ring
Assume hash space = 0 to 99 (circular).

Place 3 servers:
A → 10
B → 40
C → 80

0 ----10(A)----40(B)----80(C)----99 -> back to 0

Step 2 — Add Keys
Key Hash
K1 5
K2 25
K3 50
K4 85

Assignment Rule:
Move clockwise → first server you hit owns the key.

Result:
K1 (5) → A (10)
K2 (25) → B (40)
K3 (50) → C (80)
K4 (85) → wrap → A (10)

Distribution:
A → K1, K4
B → K2
C → K3

Removing a Server

Remove Server B (40).
New ring:

0 ----10(A)---------80(C)----99

Which keys move?
Only keys that belonged to B.
That is:
K2
Reassign:
K2 (25) → next clockwise → C (80)

New distribution:
A → K1, K4
C → K2, K3

Only one key moved.
Adding a Server
Add Server D → 30

New ring:

0 ----10(A)----30(D)----80(C)----99

Which keys are affected?
Only keys between 10 and 30.

That is:
K2 (25)
K2 moves from C → D.

New distribution:
A → K1, K4
D → K2
C → K3

Again, only one key moved.

Why Consistent Hashing Is Better Than Traditional Hashing
Traditional Hashing ?
hash(key) % N

If N changes (e.g., 3 → 4 servers):

Almost all keys get reassigned.
Massive reshuffling occurs.

Consistent Hashing
When a server is:

Removed → only its keys move.
Added → only nearby keys move.
Movement is proportional to 1 / number of servers, not the entire dataset.

Takeaway

Consistent hashing minimizes data movement by placing servers and keys on a circular hash space and assigning each key to the next server clockwise.

That design makes distributed systems scalable and stable.

Netflix Stranger Things S5 premiere Outage

Fatima Alam — Mon, 01 Dec 2025 17:42:13 +0000

Stranger Things S5 premiere -> Not a capacity, but a shape problem. How synchronized users, cache coldness and aggressive retries turned a hot release into a short-lived outage.

When the latest season of a blockbuster drops, millions of people behave like one organism: they all hit play at once. This time the surge didn’t look like the usual rounded hill of traffic — it looked like a vertical ladder: a 0→kaboom 🔥 spike in milliseconds.

Well Netflix didn’t fail because it couldn’t scale; it failed because the shape of the traffic broke assumptions baked into caching, retry logic and control planes. Here’s how that happened, what went wrong, and how the incident was recovered in minutes.

Scaled systems can still fail if real users act differently than synthetic load tests: synchronized first-access events + cache cold starts + aggressive client retries = request storm that overwhelms control and data planes in milliseconds.

🤯 Confusing right??
Let's dive in deep then

The story, step-by-step

1. Synchronized demand —> a vertical ladder, not a hill
Millions of users didn’t browse — they searched and hit play. That means first access behavior for each user (auth 🔐, metadata fetches, entitlement checks, CDN validation) executed simultaneously. This is not “more traffic” in the ordinary sense; it’s many cold sessions all at once.

2. The critical difference -> traffic shape vs traffic size
Most autoscaling and load tests assume gradual increases or randomized requests. Here the size could have been handled — but the shape (an instantaneous wall) caused concentrated load on critical coordination points (auth, metadata, caches).

3. Cache cold-starts → cache-miss storm → higher latency & memory pressure
Edge caches / Redis were not warm for this particular object set. Each user triggered the same cache misses, producing many backend DB/metadata reads per user instead of cheap cache hits. Those synchronous misses amplified latency and memory pressure: classic request storm.

4. Smart TVs were the first to go down —> and why that’s important
TV apps aggressively retry on failure and request higher-res manifests (so larger payloads). Retries multiply the original traffic in milliseconds and quickly saturate bandwidth and backend queues. Even extra headroom in bandwidth can’t absorb an exponential retry multiplier.

5. Autoscaling reacted —> but too late
Autoscaling worked: control plane observed high metrics and spun more instances. But scaling happens in seconds/minutes; the spike arrived in milliseconds. By the time new capacity was available, retries had saturated services and increased latencies so much that the system was effectively degraded.

6. It wasn’t a hardware failure —> it was behavioral overload
This was not disks, NICs or CPUs failing. The system became bottlenecked due to request shape, retries and control-plane lag — i.e., how users and clients behaved.

How the system recovered 🔄 (and what the incident response did right)

Detect anomaly —> monitoring caught the abnormal traffic shape and sudden latency/failure spikes.

Stabilize first, diagnose later —> they immediately reduced the blast radius instead of trying to find a perfect root cause under active failure.

Graceful degradation & load-shedding —> rate limit non-critical requests, deprioritize background or non-essential API calls. Some clients were slowed so that others could stream.

Rate limiting 🚦 / request shaping —> tiered throttles and token buckets prevented control-plane saturation and allowed streaming tokens to be issued at a controlled pace.

Partial restore —> restore the core streaming functionality first; investigate detailed root cause after the user-impacting symptoms were minimized.

No client update needed —> changes were server-side (policies at edge / API Gateway) and the service returned within ~5 minutes.

Root causes (summary)

Traffic shape -> traffic size — extreme synchronization induced simultaneous cold-start work for many users.

Cache cold misses —> downstream systems experienced multiplied demand because caches were not warmed.

Aggressive client retries —> smart TVs and other clients retried rapidly, amplifying load.

Autoscaling lag ⚙️ —> control plane and horizontal scaling were too slow compared to the spike velocity.

Control-plane overload —> not only data plane—control systems (auth/token issuers, rate-limiters) were hit hard.

Design lessons & takeaways

Design for traffic volume shape, not just size. Consider worst-case first-access bursts when estimating capacity.

Treat first access as the worst-case scenario. Warm caches proactively, use pre-warming strategies before premieres.

Build recovery-first systems, not perfection. Partial service with graceful degradation is preferable to full failure.

Limit and shape retries client-side. Backoff windows and jitter prevent synchronized retry storms ⚠️.

Split control-plane and data-plane scaling assumptions. Make critical control-plane functions horizontally scalable and fast to spin up.

Plan pre-warm/CDN-population strategies for launches. Pre-heat caches and edge nodes for known release times.

Run synchronized-user load-tests. Simulate large numbers of first-time users hitting the same object set at the same exact millisecond to validate behavior.

Prioritize mitigating retries and bursts over raw throughput. Retry amplification can outstrip any static extra capacity.

This outage was not a sign that the cloud failed — it was a reminder that humans and devices don’t produce uniform loads. They synchronize. They retry. They magnify small edge-cases into systemic stress. The fix is less about infinite servers and more about shaping traffic: smart client design, robust edge policies, pre-warming, and incident playbooks that keep the lights on while the forensic work starts.

Decorator Pattern

Fatima Alam — Sun, 05 Oct 2025 11:55:56 +0000

Who doesn't like customizations!!. Personally I wouldn't mind extra cheese on my pizza 😉.

Just like that when you design systems made for customizations or those who let the user customize their orders you need the Decorator Design Pattern.

Imagine you’re building a vehicle configuration platform (like Tesla’s online configurator 🚘).

A customer can start with a basic car, and then add features dynamically like:

Sunroof 🌞
Sports Package 🏎️
Advanced Sound System 🔊 Each feature adds extra cost and description, without changing the original Car class — perfect use case for the Decorator Pattern.

Component Interface

interface Vehicle {
    String getDescription();
    double getCost();
}

Concrete Component

class BasicCar implements Vehicle {
    @Override
    public String getDescription() {
        return "Basic Car";
    }

    @Override
    public double getCost() {
        return 500000.0; // base price
    }
}

Abstract Decorator

abstract class VehicleDecorator implements Vehicle {
    protected Vehicle vehicle;

    public VehicleDecorator(Vehicle vehicle) {
        this.vehicle = vehicle;
    }

    @Override
    public String getDescription() {
        return vehicle.getDescription();
    }

    @Override
    public double getCost() {
        return vehicle.getCost();
    }
}

Concrete Decorators

Each decorator adds a unique feature and modifies cost/description.

class SunroofDecorator extends VehicleDecorator {
    public SunroofDecorator(Vehicle vehicle) {
        super(vehicle);
    }

    @Override
    public String getDescription() {
        return vehicle.getDescription() + ", Sunroof";
    }

    @Override
    public double getCost() {
        return vehicle.getCost() + 40000.0;
    }
}

class SportsPackageDecorator extends VehicleDecorator {
    public SportsPackageDecorator(Vehicle vehicle) {
        super(vehicle);
    }

    @Override
    public String getDescription() {
        return vehicle.getDescription() + ", Sports Package";
    }

    @Override
    public double getCost() {
        return vehicle.getCost() + 120000.0;
    }
}

class SoundSystemDecorator extends VehicleDecorator {
    public SoundSystemDecorator(Vehicle vehicle) {
        super(vehicle);
    }

    @Override
    public String getDescription() {
        return vehicle.getDescription() + ", Premium Sound System";
    }

    @Override
    public double getCost() {
        return vehicle.getCost() + 35000.0;
    }
}

Main Class — Building the Custom Car

public class VehicleDecoratorDemo {
    public static void main(String[] args) {
        Vehicle car = new BasicCar();
        System.out.println(car.getDescription() + " ₹" + car.getCost());

        // Add sunroof
        car = new SunroofDecorator(car);
        System.out.println(car.getDescription() + " ₹" + car.getCost());

        // Add sports package
        car = new SportsPackageDecorator(car);
        System.out.println(car.getDescription() + " ₹" + car.getCost());

        // Add sound system
        car = new SoundSystemDecorator(car);
        System.out.println(car.getDescription() + " ₹" + car.getCost());
    }
}

🧩 Real-World Analogies

Domain	Example Decorators
E-Commerce	Apply coupon, add gift wrap, add insurance
Vehicles	Add GPS, airbags, turbo engine
Gaming	Add armor, magic boost, invisibility cloak
Media Streaming	Add subtitles, change resolution, enable HDR

Observer Design Pattern

Fatima Alam — Sat, 04 Oct 2025 16:47:13 +0000

How does your favorite websites like Amazon and Flipkart instantly reflect changes like when a product’s price drops and your shopping app flashes a notification, or when someone likes your post and it appears in real time on your feed?

That’s not magic — it’s the Observer Design Pattern in action.

At its core, the Observer Pattern is all about communication and synchronization. It creates a seamless link between objects — so when one object changes, all others that depend on it are automatically updated. Think of it as a digital chain reaction: one event triggers a cascade of updates, all without tightly coupling the system together.

This pattern powers everything from UI frameworks and live dashboards **to **stock market apps, chat systems, and real-time notifications. It’s the silent backbone behind modern reactive systems that need to stay always up to date.

In this post, we’ll break down how the Observer Pattern works, when to use it, and explore a real-world example to help you master one of the most elegant ways to build flexible, event-driven software.

Observable

import java.util.*;

abstract class Observable {
    protected List<Observer> observers = new ArrayList<>();

    public void addObserver(Observer observer) {
        observers.add(observer);
    }

    public void removeObserver(Observer observer) {
        observers.remove(observer);
    }

    public void notifyObservers(Object arg) {
        for (Observer observer : observers) {
            observer.update(this, arg); // push model
        }
    }
}

Observer

abstract class Observer {
    public abstract void update(Observable observable, Object arg);
}

Product

class Product {
    private String id;
    private String name;
    private double price;
    private int stock;

    public Product(String id, String name, double price, int stock) {
        this.id = id;
        this.name = name;
        this.price = price;
        this.stock = stock;
    }

    public String getId() { return id; }
    public String getName() { return name; }
    public double getPrice() { return price; }
    public int getStock() { return stock; }

    public void setPrice(double price) { this.price = price; }
    public void setStock(int stock) { this.stock = stock; }

    @Override
    public String toString() {
        return name + " (₹" + price + ", stock: " + stock + ")";
    }
}

Product Inventory

class ProductInventory extends Observable {
    private Map<String, Product> products = new HashMap<>();

    public void addProduct(Product product) {
        products.put(product.getId(), product);
        System.out.println("Added product: " + product);
        notifyObservers("Added: " + product.getName());
    }

    public void updatePrice(String productId, double newPrice) {
        Product p = products.get(productId);
        if (p != null) {
            p.setPrice(newPrice);
            System.out.println("\nPrice updated for " + p.getName() + " -> ₹" + newPrice);
            notifyObservers(p);
        }
    }

    public void updateStock(String productId, int newStock) {
        Product p = products.get(productId);
        if (p != null) {
            p.setStock(newStock);
            System.out.println("\nStock updated for " + p.getName() + " -> " + newStock + " units");
            notifyObservers(p);
        }
    }

    public Product getProduct(String id) {
        return products.get(id);
    }
}

Website Observer

class WebsitePlatform extends Observer {
    @Override
    public void update(Observable observable, Object arg) {
        if (arg instanceof Product) {
            Product product = (Product) arg;
            System.out.println("🌐 Website Updated: " + product.getName() +
                               " is now ₹" + product.getPrice() +
                               " | Stock: " + product.getStock());
        } else if (arg instanceof String) {
            System.out.println("🌐 Website Notification: " + arg);
        }
    }
}

Mobile App Observer

class MobileAppPlatform extends Observer {
    @Override
    public void update(Observable observable, Object arg) {
        if (arg instanceof Product) {
            Product product = (Product) arg;
            System.out.println("📱 Mobile App Alert: " + product.getName() +
                               " now ₹" + product.getPrice() +
                               " | Stock: " + product.getStock());
        } else if (arg instanceof String) {
            System.out.println("📱 App Notification: " + arg);
        }
    }
}

Kiosk Observer

class KioskPlatform extends Observer {
    @Override
    public void update(Observable observable, Object arg) {
        if (arg instanceof Product) {
            Product product = (Product) arg;
            System.out.println("🏪 Kiosk Screen Updated: " + product.getName() +
                               " — Price: ₹" + product.getPrice() +
                               ", Stock: " + product.getStock());
        }
    }
}

Main Class

public class ECommerceObserverMain {
    public static void main(String[] args) {
        // Create observable inventory
        ProductInventory inventory = new ProductInventory();

        // Create observers (platforms)
        WebsitePlatform website = new WebsitePlatform();
        MobileAppPlatform app = new MobileAppPlatform();
        KioskPlatform kiosk = new KioskPlatform();

        // Register observers
        inventory.addObserver(website);
        inventory.addObserver(app);
        inventory.addObserver(kiosk);

        // Add products
        Product iphone = new Product("P001", "iPhone 15", 79999, 10);
        Product macbook = new Product("P002", "MacBook Air", 119999, 5);

        inventory.addProduct(iphone);
        inventory.addProduct(macbook);

        // Simulate updates
        inventory.updatePrice("P001", 75999);
        inventory.updateStock("P002", 3);
    }
}

Strategy Design Pattern

Fatima Alam — Sat, 04 Oct 2025 16:47:04 +0000

Imagine something that can be changed at runtime and is already been used and has to be replicated again.

Think of an e-comm app like Amazon where you can choose from different Payment methods -> Cash, Card (Credit Card, Debit Card), UPI, etc... Now if you want to buy a Prime membership wouldn't you expect all these Payment methods there too??

Let's look at how such a thing is actually carried and scaled out.

This is your ShoppingCart

class ShoppingCart {
    public void checkout(String paymentType, int amount) {
        if (paymentType.equals("CREDIT")) {
            System.out.println("Paid " + amount + " with Credit Card");
        } else if (paymentType.equals("PAYPAL")) {
            System.out.println("Paid " + amount + " with PayPal");
        } else if (paymentType.equals("UPI")) {
            System.out.println("Paid " + amount + " with UPI");
        }
    }
}

And this is your Subscription

class Subscription {
    public void checkout(String paymentType, int amount) {
        if (paymentType.equals("CREDIT")) {
            System.out.println("Paid " + amount + " with Credit Card");
        } else if (paymentType.equals("PAYPAL")) {
            System.out.println("Paid " + amount + " with PayPal");
        } else if (paymentType.equals("UPI")) {
            System.out.println("Paid " + amount + " with UPI");
        }
    }
}

Don't they look the same!!
Now imagine if you want to add a new method "Cash" for both of them. Yes you are right!! This is a classic example of Code Redundancy
We could have instead did something like this

interface PaymentStrategy {
    void pay(int amount);
}

class CreditCardPayment implements PaymentStrategy {
    @Override
    public void pay(int amount) {
        System.out.println("Paid " + amount + " with Credit Card");
    }
}

class PayPalPayment implements PaymentStrategy {
    @Override
    public void pay(int amount) {
        System.out.println("Paid " + amount + " with PayPal");
    }
}

class UpiPayment implements PaymentStrategy {
    @Override
    public void pay(int amount) {
        System.out.println("Paid " + amount + " with UPI");
    }
}

class ShoppingCart {
    private PaymentStrategy paymentStrategy;

    public void setPaymentStrategy(PaymentStrategy paymentStrategy) {
        this.paymentStrategy = paymentStrategy;
    }

    public void checkout(int amount) {
        paymentStrategy.pay(amount);
    }
}

class Subscription {
    private PaymentStrategy paymentStrategy;

    public void setPaymentStrategy(PaymentStrategy paymentStrategy) {
        this.paymentStrategy = paymentStrategy;
    }

    public void checkout(int amount) {
        paymentStrategy.pay(amount);
    }
}

The code is now

More convenient.
More scalable.
Easily maintainable.
No redundancy.
More reliable

Liskov Substitution Principle

Fatima Alam — Sat, 04 Oct 2025 16:46:48 +0000

If the Parent class has a method which is not required in a new Child class in this case our design becomes unscalable.
There is a need to remove this method from this Parent Class and create a Child class which inherits from this Parent Class and all the present Child Classes now are sub Child of this Child Class now.
For eg ->
🚗 Before (LSP Violation)

Here, Vehicle has hasEngine(). But Bicycle doesn’t really have an engine, so overriding it with null breaks LSP.

// Base class
class Vehicle {
    public int numberOfWheels() {
        return 0;
    }

    public Boolean hasEngine() {
        return true;
    }
}

// Car inherits Vehicle
class Car extends Vehicle {
    @Override
    public int numberOfWheels() {
        return 4;
    }

    @Override
    public Boolean hasEngine() {
        return true;
    }
}

// Motorcycle inherits Vehicle
class Motorcycle extends Vehicle {
    @Override
    public int numberOfWheels() {
        return 2;
    }

    @Override
    public Boolean hasEngine() {
        return true;
    }
}

// 🚲 Bicycle incorrectly overrides hasEngine
class Bicycle extends Vehicle {
    @Override
    public int numberOfWheels() {
        return 2;
    }

    @Override
    public Boolean hasEngine() {
        return null; // ❌ Problem: breaks LSP
    }
}

If a client expects every Vehicle to have a meaningful hasEngine(), Bicycle breaks that expectation.

✅ After (Refactored to Respect LSP)

We separate Engine-related vehicles into another class.
Now, Bicycle does not need to lie about hasEngine().

// Base Vehicle class
class Vehicle {
    public int numberOfWheels() {
        return 0;
    }
}

// Engine-based vehicles
class EngineVehicle extends Vehicle {
    public boolean hasEngine() {
        return true;
    }
}

// Car inherits EngineVehicle
class Car extends EngineVehicle {
    @Override
    public int numberOfWheels() {
        return 4;
    }
}

// Motorcycle inherits EngineVehicle
class Motorcycle extends EngineVehicle {
    @Override
    public int numberOfWheels() {
        return 2;
    }
}

// Bicycle inherits Vehicle (no engine here)
class Bicycle extends Vehicle {
    @Override
    public int numberOfWheels() {
        return 2;
    }
}

🔑 Key Difference

Before: All Vehicles were forced to define hasEngine(), but Bicycle violated LSP.

After: We introduced EngineVehicle, so only engine-based vehicles must define engine-related behavior.

Car and Motorcycle → extend EngineVehicle

Bicycle → extends plain Vehicle

Design Patterns

Fatima Alam — Sat, 04 Oct 2025 16:46:41 +0000

Index

Index of Handling Real-Time Systems

Fatima Alam — Sun, 31 Aug 2025 05:43:24 +0000

Handling Real-Time Systems

Fatima Alam — Sun, 31 Aug 2025 05:32:45 +0000

Imagine you and your friend both try to withdraw money from the same bank account at the exact same time. If the system isn’t careful, it might let both withdrawals go through, even if there isn’t enough money. This is exactly the kind of problem databases face when multiple users access data simultaneously.

If you’ve already studied ACID properties, you know the theory behind reliable transactions. But how do real-world systems actually manage multiple users interacting with the database simultaneously?

Think about it: how do platforms like BookMyShow, Indigo, or IRCTC handle massive traffic, especially during peak times—like booking that coveted Tatkaal ticket—without crashing or mixing up data?

In this guide, we’ll break down concurrency, isolation, and consistency issues in both centralized and distributed database systems. Step by step, you’ll see how real-time systems juggle multiple operations in parallel while keeping data accurate and consistent.

Let’s dive in and paint a clear picture of what really happens behind the scenes when millions of users interact with a database at the same time.

Here's a reference for entire series of these topics.

Distributed Concurrency Control

Fatima Alam — Sun, 31 Aug 2025 05:32:01 +0000

In modern distributed systems, multiple users or applications often access and modify the same data concurrently. Without proper control, this can lead to inconsistent data, race conditions, or system crashes. This is where Distributed Concurrency Control (DCC) comes in—a set of techniques to manage concurrent operations safely across distributed databases.

What is Distributed Concurrency Control?

Managing Bank Transactions Safely Across Systems

Imagine a customer, Alice, transferring money between her savings and checking accounts in a distributed banking system. At the same time, another transaction is trying to deduct a loan payment from her checking account.

Without proper coordination, these concurrent operations could overwrite each other, causing incorrect balances or inconsistent states. Distributed Concurrency Control ensures such conflicts are managed safely across multiple nodes.

Key Goals:

Consistency: Database remains in a valid state after transactions.
Isolation: Transactions behave as if executed one after another.
Atomicity: Each transaction completes fully or not at all.

Bank Transaction Scenario

Initial State:

Account	Balance
Savings	5000
Checking	2000

Transactions:

T1 (Savings → Checking transfer): Transfer 1000 from Savings to Checking.
T2 (Loan Payment): Deduct 1500 from Checking.

Types of Distributed Concurrency Control

There are two main strategies for managing concurrency in distributed systems:

Pessimistic Concurrency Control (PCC)
Optimistic Concurrency Control (OCC)

1. Pessimistic Concurrency Control (PCC)

PCC assumes conflicts between transactions are likely. It prevents conflicts by locking data before accessing it. If another transaction wants the same data, it must wait.
PCC assumes conflicts are likely and prevents them upfront using locks.

Code:

// T1: Transfer Money
lock(savingsAccount, EXCLUSIVE);
lock(checkingAccount, EXCLUSIVE); 
savingsAccount.balance -= 1000;
checkingAccount.balance += 1000;
commit();
unlock(savingsAccount, checkingAccount);

// T2 will wait for locks
lock(checkingAccount, EXCLUSIVE);
checkingAccount.balance -= 1500;
commit();
unlock(checkingAccount);

How it works?

Time	T1 (Transfer)	T2 (Loan Payment)
t1	LOCK Savings (X-lock)
t2	LOCK Checking (X-lock)
t3	Deduct 1000 from Savings
t4	Add 1000 to Checking	BLOCKED (Waiting for X-lock)
t5	COMMIT & Release Locks	LOCK Checking → Deduct 1500
t6		COMMIT

Key Points:

Locks prevent conflicts.
Other transactions wait until locks are released.
Suitable for Repeatable Read and Serializable isolation levels.

Pessimistic Concurrency Control Example

T1: [LOCK Savings+Checking] → [UPDATE] → [COMMIT]
T2: [WAIT] → [LOCK] → [UPDATE] → [COMMIT]

Advantages of PCC:

Prevents conflicts proactively.
Suitable for high-conflict environments.

Disadvantages:

Locks can cause deadlocks.
Reduced concurrency if many transactions wait for locks.

2. Optimistic Concurrency Control (OCC)

OCC assumes conflicts are rare, so it does not lock data. Transactions proceed and validate at commit.
Code ->

# T1: Transfer Money
savings_local = read(savingsAccount)
checking_local = read(checkingAccount)
savings_local -= 1000
checking_local += 1000

if validate(savingsAccount, checkingAccount):
    write(savingsAccount, savings_local)
    write(checkingAccount, checking_local)
else:
    abort_and_retry()

# T2: Loan Payment
checking_local = read(checkingAccount)
checking_local -= 1500
if validate(checkingAccount):
    write(checkingAccount, checking_local)
else:
    abort_and_retry()

How it works?

Time	T1 (Transfer)	T2 (Loan Payment)
t1	Read Savings = 5000, Checking = 2000	Read Checking = 2000
t2	Update local copies	Update local copy
t3	Validate → no conflicts	Validate → conflict detected
t4	Commit: Savings = 4000, Checking = 3000	ABORT → retry

Key Points:

Transactions proceed without locks.
Conflicts detected at commit trigger retries.
Suitable for Read Uncommitted and Read Committed isolation levels.

Optimistic Concurrency Control Example

T1: [READ local copy] → [UPDATE local] → [VALIDATE PASS] → [COMMIT]
T2: [READ local copy] → [UPDATE local] → [VALIDATE FAIL] → [RETRY]

Advantages of OCC:

No locking overhead.
High concurrency in low-conflict environments.

Disadvantages:

Transactions may need to retry frequently in high-conflict scenarios.
Requires careful validation logic.

How PCC and OCC Map to Isolation Levels

Concurrency Control	Isolation Levels it Supports	Notes
Pessimistic (PCC)	Repeatable Read, Serializable	Locks prevent dirty & non-repeatable reads, ensures serializability
Optimistic (OCC)	Read Committed, Read Uncommitted	Validation ensures committed data, allows higher concurrency

Database Isolation Levels

Fatima Alam — Sun, 31 Aug 2025 04:50:43 +0000

💡 Database Isolation Levels

Imagine you’re driving through a busy intersection.
Cars = transactions
Intersection = database
Traffic lights = isolation levels

If there are no rules, everyone just drives whenever they want → chaos.
That’s why databases use Isolation Levels: rules that control how transactions move so data doesn’t crash into each other.

🚦 The Traffic Light Analogy

Think of a busy intersection.
Cars = transactions
Intersection = database
Traffic lights = isolation levels

🚗 Read Uncommitted → No traffic lights. Everyone just rushes in. Fast, but lots of accidents (dirty reads).

🚦 Read Committed → Red/green lights exist, but they only control entry. Cars can still switch lanes suddenly (non-repeatable reads).

🛑 Repeatable Read → Not only red/green, but lanes are locked until you exit the junction. No car can push you out — but surprise, new cars might still show up in an empty lane (phantoms).

👮 Serializable → A strict traffic cop makes cars go one batch at a time. Zero accidents, but slow traffic.

Understanding isolation levels is key to writing correct and concurrent database applications.
This guide explains all levels with clear tables, anomalies, and transaction flows so that learning is fast and practical.

Isolation Level	Key Points	Concurrency	Isolation Strength
Read Uncommitted	Can read uncommitted (dirty) data. Dirty Read, Non-Repeatable Read, Phantom Read possible.	High	Low
Read Committed	Reads only committed data. Prevents Dirty Read. Non-Repeatable & Phantom Read possible.	Medium-High	Medium-Low
Repeatable Read	Reads held via shared locks until commit. Prevents Dirty & Non-Repeatable Read. Phantom Read possible.	Medium	High
Serializable	Strictest. Uses range locks. Prevents Dirty, Non-Repeatable & Phantom Reads.	Low	Highest

2️⃣ What Problems Can Happen? (Accidents on the Road)

- Dirty Read 🚨 → You look at another car’s lane change before it’s even finished. (Reading uncommitted data).
- Non-Repeatable Read 🔄 → You check once, see a car in lane 1. You look again, and it has moved to lane 2. (Same row, different value).
- Phantom Read 👻 → You look at the junction, count 2 cars. A moment later, new cars have appeared out of nowhere. (New rows magically appear).

Isolation Level	Dirty Read	Non-Repeatable Read	Phantom Read
Read Uncommitted	✅ Yes	✅ Yes	✅ Yes
Read Committed	❌ No	✅ Yes	✅ Yes
Repeatable Read	❌ No	❌ No	✅ Yes
Serializable	❌ No	❌ No	❌ No

3️⃣ Transaction Examples (Traffic Flow Timelines)

Let’s watch how two cars (T1 & T2) drive through the intersection.
Time flows downward.
S-lock = car blocking a lane for looking.
X-lock = car blocking a lane for turning/changing.

Read Uncommitted (Dirty Read possible)
One car drives in without waiting, and another peeks too early.


| Time ↓ | T1                    | T2                 |
| ------ | --------------------- | ------------------ |
| t1     | BEGIN                 |                    |
| t2     | UPDATE Alice (X-lock) |                    |
| t3     |                       | READ Alice (dirty) |
| t4     | COMMIT                |                    |

Read Committed (Non-Repeatable Read possible)
Cars wait for the green light, but switching lanes between looks is allowed.


| Time ↓ | T1                                          | T2                    |
| ------ | ------------------------------------------- | --------------------- |
| t1     | BEGIN                                       |                       |
| t2     | READ Alice (S-lock,<br>released after read) |                       |
| t3     |                                             | UPDATE Alice (X-lock) |
| t4     |                                             | COMMIT                |
| t5     | READ Alice again (new value)                |                       |
| t6     | COMMIT                                      |                       |

Repeatable Read (Phantom Read possible)
Your lane is locked until you leave, but new lanes can still open.


| Time ↓ | T1                                   | T2                                |
| ------ | ------------------------------------ | --------------------------------- |
| t1     | BEGIN                                |                                   |
| t2     | READ Alice (S-lock held till commit) |                                   |
| t3     |                                      | UPDATE Alice (X-lock,<br>blocked) |
| t4     | COMMIT                               |                                   |
| t5     |                                      | UPDATE Alice (succeeds)           |
| t6     |                                      | COMMIT                            |

Serializable (No anomalies)
The traffic cop says: “One batch at a time. Wait your turn.”


| Time ↓ | T1                                                                            | T2                            |
| ------ | ----------------------------------------------------------------------------- | ----------------------------- |
| t1     | BEGIN                                                                         |                               |
| t2     | READ Accounts WHERE balance>1000<br>(S-lock + range lock<br>held till commit) |                               |
| t3     |                                                                               | INSERT Bob(2000)<br>(blocked) |
| t4     | COMMIT                                                                        |                               |
| t5     |                                                                               | INSERT succeeds               |
| t6     |                                                                               | COMMIT                        |

4️⃣ Crux / Key Differences

Feature / Isolation	Read Uncommitted	Read Committed	Repeatable Read	Serializable
Dirty Read	Can read uncommitted (dirty) data ✅	Prevents dirty read ❌	Prevents dirty read ❌	Prevents dirty read ❌
Non-Repeatable Read	Non-Repeatable read possible ✅	Non-Repeatable read possible ✅	Prevents Non-Repeatable read ❌	Prevents Non-Repeatable read ❌
Phantom Read	Phantom read possible ✅	Phantom read possible ✅	Phantom read possible ✅	Prevents Phantom read ❌
S-lock on read	None	Short-lived	Held till commit	Held + Range
X-lock on write	Exclusive lock applied ✅	Exclusive lock applied ✅	Exclusive lock applied ✅	Exclusive lock applied ✅
Concurrency	High	Medium-High	Medium	Low
Isolation strength	Lowest	Medium-Low	High	Highest