Forem: mahmoudabbasi

Building a Self-Learning Recommender System (Without Needing Netflix Data!)

mahmoudabbasi — Wed, 24 Sep 2025 10:14:10 +0000

Many people think you need huge datasets like Netflix or Amazon to build a recommender system. The truth is: you can start small — with a simple model — and let it improve itself over time as users interact with your product.

In this post, we'll build a self-learning recommender system that:

Starts with a pre-trained model (solves the cold-start problem)
Learns from user interactions over time (incremental learning)
Updates recommendations to stay relevant
The Problem

When you first launch a product, you usually have no idea what each user likes. If your recommender starts giving random suggestions, users might leave before the system learns anything.

To avoid this, we start with a pre-trained model based on your historical sales or click data. This allows your system to give at least reasonable recommendations on day one.

Building the Initial Model

Let's assume you have a small e-commerce store selling shoes, t-shirts, and accessories. You have some historical purchase data like this:

import pandas as pd


# Historical data
data = pd.DataFrame({
"userId": [1, 1, 2, 2, 3],
"productId": ["T-Shirt", "Shoes", "Shoes", "Jacket", "Hat"],
"category": ["Clothes", "Shoes", "Shoes", "Clothes", "Accessories"]
})


# Step 1: Find most popular categories
category_popularity = data.groupby("category")["productId"].count().sort_values(ascending=False)


def initial_recommendations(user_id):
return list(category_popularity.index) # Recommend popular categories first


print(initial_recommendations(1))
# Output: ['Shoes', 'Clothes', 'Accessories']

With just a few lines of code, you have a model that suggests the most popular categories for new users.

Making It Self-Learning

Now let's say user 1 buys another pair of shoes. Our system should learn from this and boost the weight for "Shoes" in that user's profile.

# A simple way to store user preferences
user_profiles = {
1: {"Shoes": 1, "Clothes": 1},
2: {"Shoes": 2, "Clothes": 1}
}


# Update function when a new purchase happens
def update_profile(user_id, category):
user_profiles.setdefault(user_id, {})
user_profiles[user_id][category] = user_profiles[user_id].get(category, 0) + 1


# User 1 buys new shoes
update_profile(1, "Shoes")
print(user_profiles[1]) # {'Shoes': 2, 'Clothes': 1}

Now we can recommend categories based on this updated profile — focusing on what each user really likes.

The Learning Cycle

Start with pre-trained model (based on historical data)
Serve recommendations to the user
Track interactions (click, purchase, like/dislike)
Update user profile or model weights
Generate improved recommendations

This cycle continues, and your system becomes smarter with every interaction.

Advantages of This Approach

✅ Better cold-start performance – users get meaningful suggestions from day one
✅ Personalization over time – recommendations adapt to user behavior
✅ Scalable – works with small data, can grow into ML models like ALS, bandits, or deep learning later

Next Steps

If you want to take this further, you could:

Use ALS (Alternating Least Squares) in Spark for collaborative filtering
Implement multi-armed bandits for real-time optimization
Combine content-based + collaborative filtering for hybrid recommenders

Would you like me to write a follow-up on building this with Spark or deep learning? Leave a comment below!

This approach lets you build a recommender that doesn't stay static — it learns and adapts with your users.

Database Connection Settings in Java: How to Optimize MongoDB Usage

mahmoudabbasi — Wed, 24 Sep 2025 09:30:08 +0000

When building Java applications that connect to a database like MongoDB, it’s not enough to just provide a connection string. Proper configuration ensures performance, stability, and scalability. In this guide, we’ll explore the key database settings in Java, what each does, and the best practices for connecting to MongoDB — including rate limiting to protect your app during peak load.

Connection Settings

These define how your Java app connects to MongoDB:

URI / Host / Port
Example: mongodb://localhost:27017
If you use a replica set, list all nodes for proper failover.
Authentication
Username, password, and authentication database. Essential for security.
Connect Timeout
Maximum time to wait for a connection to establish.
Default: 10 seconds; can reduce to 3–5 seconds for latency-sensitive apps.

Socket Timeout
Maximum time to wait for a response after a connection is established.
Be careful not to set too low, otherwise long queries may fail.

Connection Pool Settings

MongoDB uses connection pools in Java via MongoClient. Key settings:

Max Pool Size
Maximum open connections. Default: 100.
Choose based on concurrent threads.
Min Pool Size
Minimum connections kept alive for quick allocation.
Max Connection Idle Time
Maximum idle time before a connection is closed.
Wait Queue Timeout
Time a thread waits for a free connection before failing.

Proper connection pooling prevents Connection pool exhausted errors under load.

Read & Write Concerns

MongoDB offers guarantees for data safety:

Write Concern

w=1: write acknowledged by primary only

w=majority: write acknowledged by majority of nodes (safer but slower)

w=0: unacknowledged write (fast, risky)

Read Preference Choose from primary, primaryPreferred, secondary, secondaryPreferred, nearest based on load balancing and consistency needs.

SSL / TLS Use TLS if your database is in the cloud or on an insecure network.

Example

mongodb+srv://user:password@cluster.mongodb.net/test?tls=true

Monitoring & Heartbeat

MongoDB uses heartbeat intervals to check replica set status.
Default: 10 seconds. Shorter intervals allow faster failover.

Recommended Java Settings for MongoDB

Here’s a robust example

MongoClientSettings settings = MongoClientSettings.builder()
    .applyConnectionString(new ConnectionString("mongodb://user:pass@host1,host2,host3/?replicaSet=myRepl"))
    .applyToConnectionPoolSettings(builder -> builder
        .maxSize(100)
        .minSize(10)
        .maxConnectionIdleTime(60, TimeUnit.SECONDS)
        .maxWaitTime(5000, TimeUnit.MILLISECONDS)
    )
    .applyToSocketSettings(builder -> builder
        .connectTimeout(3000, TimeUnit.MILLISECONDS)
        .readTimeout(30000, TimeUnit.MILLISECONDS)
    )
    .readPreference(ReadPreference.primaryPreferred())
    .writeConcern(WriteConcern.MAJORITY)
    .build();

MongoClient mongoClient = MongoClients.create(settings);

Key Tips:

Use a singleton MongoClient to reuse connections.
Use WriteConcern.MAJORITY for data safety.
Choose ReadPreference based on your application's load and latency needs.
Proper timeouts and idle settings prevent threads from blocking indefinitely.

Rate Limiting (Highly Recommended)

Even with a properly tuned connection pool, sudden spikes of traffic can overwhelm MongoDB. Rate limiting is a safety net that ensures your app stays responsive.

How to Implement in Java

*Option 1: Using Guava RateLimiter
*

import com.google.common.util.concurrent.RateLimiter;

RateLimiter limiter = RateLimiter.create(100); // 100 requests per second

public void handleRequest() {
    limiter.acquire(); // blocks until a permit is available
    // Perform MongoDB query
}

Option 2: Bucket4j for Spring Boot APIs

@Bean
public FilterRegistrationBean<Filter> rateLimitingFilter() {
    Bandwidth limit = Bandwidth.simple(100, Duration.ofSeconds(1));
    Bucket bucket = Bucket4j.builder().addLimit(limit).build();

    return new FilterRegistrationBean<>(new RateLimitFilter(bucket));
}

Rate limiting protects both your database and your application from cascading failures during peak load.

Queue & Retry

Combine rate limiting with queueing + exponential backoff retries to avoid dropping user requests immediately when the system is under stress

Conclusion
Correct database configuration is crucial for reliable and scalable Java applications. By tuning:

Connection pool
Timeouts
Read/write concerns
Monitoring & alerting
Rate limiting and retries you can ensure your MongoDB-backed app performs smoothly under load...

architecture diagram :

💡 Pro Tip: Always perform load testing to validate your settings before production.

🧠 Analyzing SOLID Principles in an Epsilon-Greedy Recommender (Java)

mahmoudabbasi — Wed, 24 Sep 2025 04:53:05 +0000

In this post, we’ll take a simple implementation of an Epsilon-Greedy Recommender in Java and check whether it follows the SOLID principles. Then, we’ll see how to refactor it for better maintainability, extensibility, and testability.

*The Example Code
*

public class EpsilonGreedyRecommender {
    private int nItems;
    private double epsilon;
    private int[] counts;
    private double[] values;
    private Random random;

    public EpsilonGreedyRecommender(int nItems, double epsilon) {
        this.nItems = nItems;
        this.epsilon = epsilon;
        this.counts = new int[nItems];
        this.values = new double[nItems];
        this.random = new Random();
    }

    public int recommend() {
        if (random.nextDouble() < epsilon) {
            return random.nextInt(nItems);
        }
        int bestIndex = 0;
        for (int i = 1; i < nItems; i++) {
            if (values[i] > values[bestIndex]) {
                bestIndex = i;
            }
        }
        return bestIndex;
    }

    public void update(int item, double reward) {
        counts[item]++;
        values[item] += (reward - values[item]) / counts[item];
    }

    public double[] getValues() {
        return values;
    }

    public int[] getCounts() {
        return counts;
    }
}

SRP – Single Responsibility Principle

📖 Definition:
A class should have only one reason to change – it should have a single responsibility.

🔍 Analysis:
This class is doing multiple things:

Storing the bandit state (counts, values)
Implementing the selection policy (recommend())
Updating statistics (update())

This means any change to the policy logic, or to how state is stored, requires modifying the same class.

✅ Verdict: SRP is partially violated – we have multiple responsibilities in one place.

OCP – Open/Closed Principle

📖 Definition:
Classes should be open for extension but closed for modification.

🔍 Analysis:
If we want to switch to a different policy (e.g., Softmax, UCB), we would have to edit the recommend() method directly.
Better design: define a SelectionPolicy interface and plug in different implementations.

❌ Verdict: OCP is violated – adding new policies requires modifying the class.

LSP – Liskov Substitution Principle

📖 Definition:
Subtypes must be substitutable for their base types without changing program correctness.

🔍 Analysis:
We don’t have inheritance here, so there is nothing to violate.

✅ Verdict: LSP is respected.

ISP – Interface Segregation Principle

📖 Definition:
Clients should not be forced to depend on interfaces they do not use.

🔍 Analysis:
Since we have no interfaces at all, there’s no problem here.

✅ Verdict: ISP is respected.

DIP – Dependency Inversion Principle

📖 Definition:
Depend on abstractions, not on concrete implementations.

🔍 Analysis:
The class creates its own Random instance. This is a direct dependency on a concrete class, which makes testing harder (no way to inject a predictable RNG).

Better design: inject Random as a dependency via the constructor (or use an interface).

❌ *Verdict: DIP is violated *– we depend on a concrete Random implementation.

Summary Table
Principle Status Notes
SRP ❌ Multiple responsibilities (state + policy + update logic)
OCP ❌ Cannot add new policies without modifying code
LSP ✅ No inheritance, no violation
ISP ✅ No large interfaces, no violation
DIP ❌ Direct dependency on Random, hard to test

Refactored Design
**
Let’s refactor the code to follow **SOLID:

Introduce a SelectionPolicy interface (Strategy Pattern)
Inject Random from outside to improve testability

*Step 1: Define the Policy Interface
*

public interface SelectionPolicy {
    int select(double[] values);
}

*Step 2: Implement Epsilon-Greedy Policy
*

import java.util.Random;

public class EpsilonGreedyPolicy implements SelectionPolicy {
    private final double epsilon;
    private final Random random;

    public EpsilonGreedyPolicy(double epsilon, Random random) {
        this.epsilon = epsilon;
        this.random = random;
    }

    @Override
    public int select(double[] values) {
        int nItems = values.length;
        if (random.nextDouble() < epsilon) {
            return random.nextInt(nItems);
        }
        int bestIndex = 0;
        for (int i = 1; i < nItems; i++) {
            if (values[i] > values[bestIndex]) {
                bestIndex = i;
            }
        }
        return bestIndex;
    }
}

*Step 3: Make the Bandit Class Focus on State
*

public class Bandit {
    private final int[] counts;
    private final double[] values;
    private final SelectionPolicy policy;

    public Bandit(int nItems, SelectionPolicy policy) {
        this.counts = new int[nItems];
        this.values = new double[nItems];
        this.policy = policy;
    }

    public int recommend() {
        return policy.select(values);
    }

    public void update(int item, double reward) {
        counts[item]++;
        values[item] += (reward - values[item]) / counts[item];
    }
}

✅ Now:

SRP is respected → Bandit only manages state, EpsilonGreedyPolicy only handles selection.

OCP is respected → We can add new policies without touching Bandit.

DIP is respected → Random is injected, so we can pass a mock RNG in tests.

Key Takeaways

Applying SOLID makes your code easier to extend and maintain.

Using interfaces and dependency injection helps make your code testable and more robust.

Even small classes can benefit from SOLID – especially if you expect the algorithm to evolve over time.

💡 What do you think? Would you keep the state and policy together for small projects, or always split them like this?

Persian OCR with YOLO + CRNN: Building a Custom Text Recognition Pipeline

mahmoudabbasi — Tue, 23 Sep 2025 14:19:32 +0000

Running OCR for Persian text is tricky. Unlike English, Persian (and Arabic) scripts are right‑to‑left, letters change shape based on position, and there are fewer open‑source datasets available. In this post, we’ll build a custom OCR pipeline using YOLO for text detection and CRNN for character recognition.

Why YOLO + CRNN?

YOLO is great at detecting objects — here, the objects are text regions.

CRNN (Convolutional Recurrent Neural Network) is ideal for sequence recognition like text.

Combined, they form a two‑stage pipeline: detect → crop → recognize.

Preparing the Dataset

For Persian OCR we need two datasets:

Detection dataset (for YOLO): images with bounding boxes around words or lines.
Recognition dataset (for CRNN): cropped images of words/lines with their correct text.

You can:

Use tools like labelImg or Roboflow to annotate bounding boxes.

Generate synthetic data: render Persian text on random backgrounds using different fonts to increase data size.

YOLO expects annotations in this format:

class x_center y_center width height
where values are normalized between 0 and 1.

Training YOLO for Text Detection

Use YOLOv8 for best results:

yolo detect train data=persian_text.yaml model=yolov8s.pt epochs=50 imgsz=640

After training, YOLO will output bounding boxes for text regions.

Training CRNN for Text Recognition

CRNN = CNN + RNN + CTC loss.

Define your Persian character set (32 letters + space) and encode labels as sequences.

Example PyTorch model:

import torch
import torch.nn as nn


class CRNN(nn.Module):
def __init__(self, num_classes):
super(CRNN, self).__init__()
self.cnn = nn.Sequential(
nn.Conv2d(1, 64, 3, 1, 1), nn.ReLU(),
nn.MaxPool2d(2, 2),
nn.Conv2d(64, 128, 3, 1, 1), nn.ReLU(),
nn.MaxPool2d(2, 2)
)
self.rnn = nn.LSTM(128*8, 256, bidirectional=True, num_layers=2)
self.fc = nn.Linear(512, num_classes)


def forward(self, x):
x = self.cnn(x)
b, c, h, w = x.size()
x = x.permute(3, 0, 1, 2).contiguous().view(w, b, c*h)
x, _ = self.rnn(x)
x = self.fc(x)
return x # [T, B, num_classes]

Use CTC Loss to align predictions with ground truth.

Combining YOLO + CRNN
Feed image → YOLO → bounding boxes.
Crop each box and resize to a fixed height.
Pass to CRNN → predicted text.
Concatenate results (right‑to‑left ordering).
Challenges and Tips

Right‑to‑left text: reverse CRNN output sequences before final join.

Fonts and noise: use data augmentation (blur, rotation, brightness) to improve generalization.

Small dataset? Consider transfer learning or fine‑tuning PaddleOCR models for Persian.

Conclusion

By combining YOLO and CRNN, we created a flexible OCR pipeline that works for Persian text. This approach can be extended to other right‑to‑left scripts like Arabic or Urdu.

You can check out the GitHub repo for sample code and try it on your own dataset!

Handling Distributed Transactions with Orchestrator Pattern (Withdrawal & Deposit Example)

mahmoudabbasi — Tue, 23 Sep 2025 12:06:35 +0000

When building microservices, one of the common challenges is dealing with distributed transactions — ensuring data consistency when multiple services need to work together.

Let's consider a simple but very real-world example:

Service A: Withdrawal
Service B: Deposit

We want to withdraw money from one account and deposit into another.
But what if one of them fails?

The Problem

Imagine the following flow:

Withdrawal Service reduces the balance.
Deposit Service adds the amount to the destination account.

If step 1 succeeds but step 2 fails, you have just lost money in the system.
This is where distributed transaction management becomes critical

Common Approaches

Two-Phase Commit (2PC)

A transaction coordinator asks each service to prepare and commit.
If all services agree, the transaction is committed.
If any fail, everything is rolled back.

✅ Strong consistency

❌ High complexity, risk of blocking, not always a good fit for microservices

Saga Pattern with Orchestrator

The Saga pattern breaks the big transaction into a sequence of smaller, local transactions.
Each step has a compensating transaction (rollback action) if something goes wrong.

Example:

Step Action Compensation
1 Withdraw from Account A Deposit back to Account A
2 Deposit to Account B Withdraw from Account B

The orchestrator is a service that manages this workflow:

Starts with withdrawal
If successful, triggers deposit
If deposit fails, runs compensation (refund) for withdrawal This is much more scalable and microservice-friendly.

Event-Driven & Eventually Consistent

Another approach is using message queues (Kafka, RabbitMQ):

Send a WithdrawalCompleted event
Deposit service consumes and processes it
Retry on failure until success
Make services idempotent (safe to retry without double processing)

This ensures eventual consistency even if failures occur.

Orchestrator Workflow Example

Example: Orchestrator Implementation (Pseudo-Code)

public class TransferOrchestrator {

    public void transfer(String fromAccount, String toAccount, BigDecimal amount) {
        boolean withdrawSuccess = withdrawalService.withdraw(fromAccount, amount);

        if (!withdrawSuccess) {
            log.error("Withdrawal failed");
            return;
        }

        boolean depositSuccess = depositService.deposit(toAccount, amount);

        if (!depositSuccess) {
            log.error("Deposit failed, triggering compensation...");
            withdrawalService.compensate(fromAccount, amount); // rollback
        } else {
            log.info("Transfer completed successfully");
        }
    }
}

Key Best Practices

✅ Idempotent APIs – handle retries safely
✅ Proper Logging – so you can trace what happened
✅ Dead Letter Queues – for failed events that need manual review
✅ Monitoring & Alerts – you don’t want silent failures

Final Thoughts
**
Distributed transactions are challenging, but with the **Saga pattern and orchestration, you can build resilient and scalable systems.

The orchestrator gives you full control over the transaction flow and lets you recover gracefully from failures — which is critical for financial and mission-critical systems

Real-Time Fraud Detection Using Apache Flink

mahmoudabbasi — Tue, 23 Sep 2025 08:14:26 +0000

Introduction
Every second, millions of financial transactions happen worldwide. How can banks instantly detect fraudulent activity and prevent losses in real-time?

This is where Apache Flink comes in — a powerful stream processing engine that can analyze millions of events as they happen.

Why Real-Time Fraud Detection?

Fraud in financial systems can be extremely costly.
Examples: Credit cards, online payments, bank transfers.
Batch vs. Stream processing:
- Batch: Data is analyzed after being collected → too late
- Stream: Every transaction is analyzed instantly → fast and effective

Why Flink?

Event-driven: Processes data streams with low latency
Stateful computation: Maintains account-level or user-level state for detecting anomalies
Scalable: Handles millions of transactions per second

Proposed Architecture

Kafka as the transaction source
Flink Job for real-time analysis and fraud scoring
Rules & ML model to detect suspicious patterns
Alerts → sent to dashboards or monitoring systems

architecture Diagram :

StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();

DataStream<Transaction> transactions = env
    .addSource(new KafkaTransactionSource());

DataStream<Transaction> flagged = transactions
    .keyBy(Transaction::getAccountId)
    .process(new FraudDetectionFunction());

flagged.addSink(new AlertSink());

env.execute("Real-Time Fraud Detection");

keyBy: Groups transactions by account
process: Runs fraud detection logic
AlertSink: Sends alerts when suspicious activity is detected

Conclusion

Flink enables real-time fraud detection
Helps reduce financial losses and increase customer trust
Offers high scalability and flexibility

CTA (Call to Action)
Curious how real-time fraud detection can save millions? Let’s connect and discuss streaming analytics with Flink!

How Self-Learning Recommender Systems Adapt to User Behavior

mahmoudabbasi — Tue, 23 Sep 2025 07:42:23 +0000

Introduction
Recommender systems are key components in many applications—from online stores to content platforms. Traditional models often require manual updates and may struggle to adapt to users’ changing preferences.

In this article, we introduce a self-learning recommender system in Java that dynamically learns from user interactions and provides more accurate, personalized suggestions.

🔹 Architecture

The system has three main components:

Data Collection – Track user interactions, e.g., views, ratings, clicks.
Self-Learning Model – Updates recommendations automatically as user behavior changes.
Recommendation Engine – Provides personalized suggestions to each user.

import java.util.*;

public class SelfLearningRecommender {
    private Map<String, Map<String, Integer>> userRatings = new HashMap<>();

    public SelfLearningRecommender() {
        userRatings.put("Alice", new HashMap<>(Map.of("Book A", 5, "Book B", 3)));
        userRatings.put("Bob", new HashMap<>(Map.of("Book A", 2, "Book B", 4)));
    }

    // Add a new rating (self-learning updates automatically)
    public void addRating(String user, String item, int rating) {
        userRatings.computeIfAbsent(user, k -> new HashMap<>()).put(item, rating);
    }

    // Recommend an item user hasn't interacted with yet
    public String recommend(String user) {
        Map<String, Integer> ratings = userRatings.getOrDefault(user, new HashMap<>());
        return ratings.entrySet().stream()
                .max(Map.Entry.comparingByValue())
                .map(Map.Entry::getKey)
                .orElse("No recommendation available");
    }

    public static void main(String[] args) {
        SelfLearningRecommender recommender = new SelfLearningRecommender();
        System.out.println("Recommendation for Alice: " + recommender.recommend("Alice"));

        // Simulate new behavior
        recommender.addRating("Alice", "Book C", 6);
        System.out.println("Updated Recommendation for Alice: " + recommender.recommend("Alice"));
    }
}

✅ Features of this self-learning model:

Updates recommendations automatically with new data
Adapts to changing user behavior
Simple, scalable, and easy to extend

🔹 Results and Benefits
Users get more personalized suggestions
No need for manual model retraining
Can scale from small apps to large e-commerce platforms

🔹 Conclusion
Self-learning recommender systems represent the future of personalized user experiences. By combining data collection, dynamic learning, and personalized recommendations, apps can better serve their users and improve engagement