Forem: Namit Jain

What is the N+1 Query Problem and How to Solve it?

Namit Jain — Sun, 03 Aug 2025 12:05:12 +0000

You’ve written the code. It works. But the page loads slowly, the database is doing way too much work, and you're staring at the query logs thinking, “Why are there so many SELECTs here?”

That moment when everything should be working but feels sluggish is when you might be meeting the N+1 query problem for the first time.

It’s not a bug. It’s not an error. It’s a pattern. And it’s quietly eating your performance.

What Is the N+1 Query Problem?

Let’s say you want to list all blog posts along with their comments.

So you do this:

Fetch the posts.
For each post, fetch its comments.

Seems fine, right?

But under the hood, this happens:

-- Step 1
SELECT * FROM posts;

-- Step 2
SELECT * FROM comments WHERE post_id = 1;
SELECT * FROM comments WHERE post_id = 2;
SELECT * FROM comments WHERE post_id = 3;
-- and so on...

If you have 100 posts, that’s 1 query for the posts and 100 more for the comments: N+1 queries in total.

That’s the problem.

It starts small. Feels innocent. But once your data grows, this turns into a flood of unnecessary database traffic and your app crawls.

Why Does It Happen?

Because it’s easy.

It happens when you’re thinking about data in terms of objects or loops instead of query efficiency. It happens when you reach for a simple solution without noticing what’s happening behind the scenes.

And yes, it happens with raw SQL too not just with ORMs. The problem isn’t the tools. It’s the pattern.

How to Tell You’re Doing It

Here’s what to watch for:

You’re running a loop in your code that includes a query.
Your query log is filled with the same SELECT statement, repeated with different IDs.
Your app is fast with 5 records but starts dragging with 50.
A profiler or database monitor shows a growing number of similar queries.

If your backend touches the database inside a loop, stop and look again. You might be doing more work than you think.

How to Solve It

The fix is about shifting your thinking: batch the data access, then organize it in memory.

Use a Single Query with a JOIN

This works when the related data is lightweight and belongs in the same result set.

SELECT posts.id, posts.title, comments.body
FROM posts
LEFT JOIN comments ON comments.post_id = posts.id;

Now you’ve done in one query what used to take 101.

Use an IN Clause to Fetch Related Data

Sometimes you don’t want a big JOIN. Maybe the data is complex or you need to process it differently.

In that case:

-- First: get the posts
SELECT * FROM posts WHERE ...;

-- Then: get all related comments at once
SELECT * FROM comments WHERE post_id IN (1, 2, 3, 4, 5);

Now you’ve done it in two queries, not 100.

Build the Relationship in Your Code

Once you’ve got both sets of data, use code to group the related records by foreign key:

# Pseudocode
grouped = group_by(comments, key=lambda c: c.post_id)
for post in posts:
    post.comments = grouped.get(post.id, [])

Should You Care?

Yes, especially if your app is growing or handles lots of data.

The N+1 problem often hides until it becomes painful. You’ll think your logic is fine, your queries are clean, and yet performance keeps slipping. That’s why this issue is worth learning early.

Fixing it is often the difference between a system that “feels fine for now” and one that scales without surprise slowdowns.

Best Habits to Avoid N+1

Don’t query the database inside a loop. Ever.
Think in sets, not rows. How can you get everything you need at once?
Use profiling tools and query logs often.
If you’re using a framework or ORM, learn how it handles data fetching under the hood.
Watch for slowdowns even when the code is “technically correct.”

The N+1 query problem isn’t a bug in your code. It’s a lesson in how computers think about data and how to outsmart that inefficiency before it slows you down.

Devlog: Why My API Was Called Twice (And It Was Totally My Fault)

Namit Jain — Wed, 30 Jul 2025 17:17:24 +0000

It started like any other day. I was poking around the app - nothing serious. Just clicking through a few screens, checking the UI, casually living my best DevTools life.

And then I saw it.

In the Network tab: Two identical API calls. Back to back.

Same endpoint. Same payload. Same timestamp, almost.

At first, I thought I was seeing things. Maybe some retry logic? Maybe the backend was returning something weird and the frontend panicked?

Nope. The backend was just responding. Twice. Calmly. As if I’d asked it to.

So obviously, I did what any reasonable developer would do.

I hit refresh. Again. And again. Still two calls. Identical.

Something was up. And I was pretty sure I had written it.

The Investigation

I started digging through the component that made the request - a pretty straightforward React component that used a custom hook to fetch data from the backend.

The hook looked fine.

No useEffect shenanigans. No dependencies causing it to run twice.

But then I noticed the same hook was being used in the parent component and in the child component.

The child used it to display a widget.

The parent used it to handle state.

Both were independently calling the same API - without realizing they were stepping on each other.

Two components. Same hook.

Two renders. Two calls.

What Went Wrong

Technically, nothing was broken. The app wasn’t crashing. The UI looked fine.

But under the hood? It was inefficient, messy, and one step away from becoming a problem.

This is one of those subtle bugs that doesn't break anything, but just quietly burns your backend and bloats your network usage until someone notices.

What I Did

I moved the hook to the parent component - the one that logically owned the data - and passed the result down to the child as props.

No duplication. One fetch. One call.

Problem gone.

What You Can Learn From This

If you're working with React and using custom hooks to fetch data, here's the takeaway:

Don't blindly drop hooks in every component that needs data.

Hooks are just functions. If you call them twice, they’ll run twice. That includes API calls, effects, subscriptions - all of it.

Here are a few tips I wish I had followed earlier:

1. Centralize your data fetching

If multiple components need the same data, let one component fetch it and pass it down.

Props still work. It’s not uncool to use them.

2. Use caching if you must duplicate

If you're using libraries like SWR or React Query, at least you're not hitting the network twice.

But be intentional about it.

3. Watch your tree

It's easy to forget that a child component might also be calling the same logic as its parent - especially if you're reusing hooks across the app.

4. Trust DevTools

If something feels off, open the Network tab. It doesn't lie.

(It’s also the fastest way to catch dumb mistakes.)

TL;DR

I found an API being called twice.
It was my fault: I was calling the same hook in both parent and child.
I moved the fetch to the parent and passed data down.
Always check for duplicated logic when you see repeated API calls.
One hook, one call. Don’t let your frontend flood your own backend.

This bug wasn’t big. It didn’t crash anything.

But it taught me a valuable lesson:

Sometimes performance issues aren’t about bad code.

They’re about code that looks fine - but doesn’t respect boundaries.

Hooks are powerful. But with great power comes...

Well, extra network calls - if you're not paying attention.

WTF is git cherry-pick?

Namit Jain — Wed, 30 Jul 2025 17:15:48 +0000

Ever wished you could steal just one commit without dragging the entire messy branch with it?

Welcome to Git’s commit heist tool: git cherry-pick.

What actually happens?

Run:

git checkout main
git cherry-pick <commit-hash>

Git grabs the diff from <commit-hash> and replays it on main as a brand-new commit.
Same content, fresh identity (new hash).

Why you need this

✅ Hotfix emergencies: pluck that one bugfix without the noise.
✅ Selective feature moves: steal just the feature you want.
✅ History hacks: clean up messy branches by cherry-picking verified commits.

💡 Pro tip: If you need more than 3 commits, merging might be smarter. Don’t over-cherry-pick like a code raccoon.

Anatomy of the output

bash [main 5f8d3e2] Fix typo in README Author: teammate <teammate@example.com> Date: Fri Jul 4 17:22:46 2025 +0530

5f8d3e2 → brand new hash, no shady reuse.
Message & author? Still carried over for context clarity.

Conflict mode: activate

Conflicts are inevitable. When they hit:

Fix markers (<<<<, ====, >>>>) in your favorite editor.
git add <resolved-files>
Continue the drama:

bash git cherry-pick --continue

⚠️ WTF moment: Cherry-pick conflicts are like that one friend who says “I’ll be chill” and then flips the table.

Spooked? Reset with:

bash git cherry-pick --abort

Back to pre-cherry-pick bliss.

Pro tips

✅ Cherry-pick small, self-contained commits. Big ones? Big drama.
✅ Avoid merge commits unless you enjoy pain.
✅ Push right after you pick. Don’t ghost your teammates.

Quick scenario

Team fixes a critical bug on dev.
Stakeholders demand it on main right now.
Other commits? Unstable, unreviewed. Nope.
You pull the heist:

bash git checkout main git cherry-pick abc1234

Customers: happy.
You: legend.

Interactive demo

Visit https://www.namitjain.com/blog for interactive demo

TL;DR

What: Steal a commit like Ocean’s Eleven.
How:

bash git cherry-pick <hash>

Why: Hotfixes, targeted changes, branch surgery.

🔥 Next WTF:

“WTF is git rebase (and Why It’s Both Magic and Chaos)”

WTF is Git Rebase (and Why It’s Both Magic and Chaos)

Namit Jain — Wed, 30 Jul 2025 17:12:08 +0000

So you’ve been happily doing:

git pull
git merge

And then someone in your team says:

“Just rebase it.”
…and you start questioning your life choices.

WTF Is Git Rebase, Really?

Definition (boring version):
git rebase is a command that lets you rewrite the commit history by moving or combining commits onto a new base commit.

Definition (WTF version):
Rebase is basically telling Git:

Yo, take my messy branch, pretend I started working from this new place, and make it look like I always had my sh*t together.

Why Even Bother?

Without rebase:
Your history looks like:

main ---A---B---C ...........\ ...........D---E (feature)

Then you merge:

main ---A---B---C------merge commit ............\........../ .............D---E-----

Looks like spaghetti 🍝.

With rebase:

main ---A---B---C ................\ ..................D'---E'

(Notice the '? Those are rebased commits. Same code, different timeline.)

The Actual WTF Command

Say you’re on feature branch and want it updated with main:

bash git checkout feature git fetch origin git rebase origin/main

Boom. Feature branch now looks like it was based on the latest main all along.
Congratulations, you’re now a Git historian.

Common Rebase Moves

Interactive Rebase:

bash git rebase -i HEAD~3

Lets you:
✅ squash commits
✅ rename commits
✅ feel like a time-traveling wizard

Abort if you panic:

bash git rebase --abort

(You WILL need this.)

Continue after fixing conflicts:

bash git add . git rebase --continue

When to Use Rebase vs Merge

✅ Use rebase when:

You want a clean history
You’re working on your own feature branch
You hate merge commits

❌ Don’t rebase shared branches (like main), or you’ll rewrite history and summon demons in your team Slack.

Pros and Cons of Rebasing

✔ Pros:

Cleaner history
No merge commits
Makes you feel pro

✖ Cons:

Can cause conflicts
Rewrite history → danger zone
Makes newbies(me) cry

Interactive demo

Visit https://www.namitjain.com/blog/git-rebase for interactive demo

TL;DR WTF

git merge = keep the mess
git rebase = rewrite history to look clean
Do it locally, don’t mess with shared branches
Always have --abort and coffee ready ☕

🔥 Next WTF Episode:

WTF is git cherry-pick (and why it ruins friendships)

Debugging a Segmentation Fault in Python?! Yes, It's Possible (And Here's What I Learned)

Namit Jain — Tue, 13 May 2025 15:46:24 +0000

Let me set the scene: I’m working on a Python app — pretty standard stack, nothing wild. Then out of nowhere...

Segmentation fault.

Wait. What? In Python?

Yeah. That was my reaction too.
Isn't Python safe from stuff like this?

Usually, yes. One of the reasons I love Python is because it protects you from low-level memory issues. But here's the catch: if you're using C extensions, or database drivers with native bindings under the hood... the safety net can have holes.

And that’s exactly what happened to me.

What Went Wrong

In my app, I was using mysql-connector-python to talk to a MySQL database. Everything looked fine on the surface.

But over time, the app would randomly crash with a segfault. No Python stack trace, just a hard exit.

I was stumped.

Enter: faulthandler + gdb

To investigate, I enabled Python’s built-in faulthandler module to try and catch something useful. I also ran the app with gdb to get a trace of what was going on at the C level.

And eventually, I found it.

The culprit?

Too many open MySQL connections. 😅

Turns out I was opening a new connection to the database on every single request — but I wasn’t closing them properly. No pooling, no reuse.

Eventually MySQL hit its connection limit and just bailed — crashing everything along with it.

The Fix: Connection Pooling FTW

The solution was (thankfully) pretty straightforward: I implemented a connection pool using mysql.connector.pooling.

Now the connections are managed properly, reused across requests, and I don’t run out of resources.

No segfaults. No database freakouts. No angry sysadmins. 🎉

Takeaways

Yes, Python can segfault. Especially when you're using C extensions or native libraries.
Always manage your database connections carefully.
Use tools like faulthandler and gdb to dig into the deeper layers when the usual logs don't help.
Don’t assume you're immune to low-level bugs just because you're writing high-level code.

TL;DR

If you're seeing weird crashes in Python, especially when working with databases or C extensions — don’t rule out things like memory errors or connection overloads.

Take the time to investigate. Your future self (and your uptime dashboard) will thank you. 🙌

Building Lip Reading Model. Part-1

Namit Jain — Mon, 03 Apr 2023 14:09:52 +0000

Lip reading, also known as speech reading, is the ability to understand speech by observing a speaker's lip movements and facial expressions. It is a crucial skill for individuals who are deaf or hard of hearing, as well as for those who need to communicate in noisy environments. However, lip reading is a challenging task that requires extensive training and experience, and even then, it can be unreliable due to variations in lip movements and speech patterns.

Recent advances in machine learning and computer vision have made it possible to develop automatic lip reading systems that can recognize speech from visual cues alone. These systems have numerous applications, from enhancing speech recognition in noisy environments to improving the accessibility of multimedia content for the deaf and hard of hearing.

In this blog series, we will explore how to build a lip reading model using deep learning techniques. In Part 1, we will cover the basics of lip reading and the dataset we will be using for our model.

Understanding Lip Reading

Lip reading involves analyzing the movements of the mouth, tongue, and jaw to determine the sounds being produced by a speaker. There are several challenges associated with lip reading, including variations in speech patterns, differences in accent and dialect, and the fact that many sounds are produced inside the mouth and are not visible on the lips.

Despite these challenges, lip reading is a useful skill that can be learned with practice. It is typically taught using a combination of visual cues, such as the shape of the lips and the movement of the jaw, and contextual cues, such as the topic of conversation and the speaker's body language.

Dataset

The Grid Corpus dataset is a widely used dataset in the field of lip reading, which contains videos of people speaking a wide range of words. The dataset was created by recording videos of speakers reading out sentences while their faces were tracked using a facial landmark detection algorithm. The dataset contains over 30,000 video clips, each clip corresponding to a spoken sentence.

The videos in the dataset are captured at a resolution of 640x480 and a frame rate of 25 frames per second. The dataset also includes the corresponding audio for each video clip, as well as the transcriptions of the spoken sentences.

The Grid Corpus dataset has been used in numerous studies to develop and evaluate lip reading models. Some of the popular models developed using this dataset include LipNet and Watch, Listen, and Spell (WLAS).

Conclusion

In Part 1 of this blog series, we have introduced the concept of lip reading and the dataset we will be using to train our lip reading model. In the next part of the series, we will explore the preprocessing steps required to prepare the Lip Reading dataset for training, including data cleaning and feature extraction. Stay tuned for Part 2!