Forem: Bharathvaj

The Dunning-Kruger Effect - Why Beginners Feel Like Experts

Bharathvaj — Mon, 11 Aug 2025 01:06:42 +0000

Ever noticed how someone who just learned PHP for a week suddenly feels ready to build the next Facebook? Or how new drivers think they're Formula 1 material after passing their test? Welcome to the fascinating world of the Dunning-Kruger effect.

🧠 What Is the Dunning-Kruger Effect?

The Dunning-Kruger effect is a cognitive bias where people with limited knowledge or competence in a domain overestimate their own knowledge or competence in that domain. Named after psychologists David Dunning and Justin Kruger, who first described it in 1999.

Here's the paradox: The less you know, the less you realize how much you don't know.

📈 The Confidence Curve

Imagine a graph where X-axis is competence and Y-axis is confidence:

Confidence
    ^
    |   Peak of Mt. Stupid
    |        /\
    |       /  \___Valley of Despair
    |      /       \___
    |     /            \___Slope of Enlightenment___Plateau of Sustainability
    |____/________________________________________________> Competence

Peak of Mt. Stupid: Know just enough to be dangerous
Valley of Despair: Reality hits - you realize how much you don't know
Slope of Enlightenment: Steady learning and realistic confidence
Plateau of Sustainability: True expertise with appropriate confidence

💻 Example 1: The Junior Developer Syndrome

Week 1 of learning React:
"React is easy! It's just components and props. I could build Netflix with this!"

Month 3 of React:
"Wait, what's the difference between useEffect and useLayoutEffect? Why is my app re-rendering 47 times? What's a render prop? Help!"

Year 2 of React:
"I know enough to build solid apps, but there's always more to learn about performance optimization, patterns, and edge cases."

🚗 Example 2: The New Driver Phenomenon

Fresh license holders often exhibit peak Dunning-Kruger:

Day 1: "I'm a natural! Driving is intuitive!"
Month 6: First close call in rain/snow makes them realize driving involves complex decision-making
Year 5: Understands defensive driving, anticipates other drivers' mistakes, respects road conditions

💰 Example 3: The Crypto Trading Expert

The 2021 crypto boom was a Dunning-Kruger festival:

# The Dunning-Kruger Trader's Journey
def trading_confidence(experience_days):
    if experience_days < 7:
        return "I'm the next Warren Buffett! 🚀"
    elif experience_days < 30:
        return "Why did my $DOGE investment drop 80%?"
    elif experience_days < 365:
        return "Maybe I should learn about market cycles..."
    else:
        return "Diversification and risk management are key"

🎯 Example 4: The Git "Expert"

After learning git add, commit, and push:
"I know Git!"

After first merge conflict:
"What have I done? How do I undo this? What's a rebase?"

After years of experience:
"I'm comfortable with Git, but I still Google how to undo specific operations."

🛡️ How to Combat It

1. Embrace the Beginner's Mind

Always assume there's more to learn. The moment you think you've mastered something, you've stopped growing.

2. Seek Feedback Actively

// Don't be this developer
function myCode() {
    // "My code is perfect, no review needed"
}

// Be this developer
function myCode() {
    // "Please review - what am I missing?"
}

3. Learn in Public

Share your work early. The internet will quickly (sometimes brutally) show you what you don't know.

4. Find Your Valley of Despair

It's actually a good sign! It means you're learning enough to recognize complexity. Every expert has been through it.

🎭 The Flip Side: Imposter Syndrome

Ironically, as people become more competent, they often experience imposter syndrome - underestimating their abilities. Experts know enough to see the vast ocean of knowledge they haven't explored.

This is why:

Beginners apply for jobs requiring "5 years experience" with 6 months under their belt
Experts hesitate to apply even when they exceed requirements

🔍 Real-World Implications

In Tech Teams

Code reviews become battles when junior devs can't recognize senior expertise
Architecture decisions get derailed by overconfident opinions
"How hard can it be?" leads to massive technical debt

In Startups

"We'll build our own payment system! How complex can it be?"
Six months later: drowning in PCI compliance, edge cases, and security issues

💡 The Sweet Spot

The goal isn't to eliminate confidence - it's to calibrate it. The best professionals maintain:

Confidence to take on challenges
Humility to recognize limitations
Curiosity to keep learning

Final Thoughts

The Dunning-Kruger effect isn't about intelligence - smart people fall for it too. It's about experience and exposure. The more you learn, the more you realize the depth of what you don't know.

Next time you feel like an expert after a YouTube tutorial, remember: you might be standing on Mt. Stupid. And that's okay - we've all been there. The key is to keep climbing down into the valley and up the other side.

Remember: True expertise whispers while ignorance shouts.

Happy Learning! 🚀

Python's mro

Bharathvaj — Mon, 11 Aug 2025 00:57:13 +0000

Python's __mro__ (Method Resolution Order) is a crucial concept that determines how Python looks up methods in inheritance hierarchies. Let's dive into what it is, why we need it, and see it in action.

🔍 What is mro?

__mro__ is a tuple that defines the order in which Python searches for methods and attributes in a class hierarchy. Every class has this attribute, showing its linearized inheritance path.

class Animal:
    pass

class Dog(Animal):
    pass

print(Dog.__mro__)
# (<class '__main__.Dog'>, <class '__main__.Animal'>, <class 'object'>)

Python searches from left to right: first in Dog, then Animal, finally object.

🎯 Why Do We Need It?

The MRO solves the diamond problem in multiple inheritance - when a class inherits from multiple classes that share a common ancestor.

class A:
    def method(self):
        return "A"

class B(A):
    def method(self):
        return "B"

class C(A):
    def method(self):
        return "C"

class D(B, C):  # Diamond inheritance
    pass

d = D()
print(d.method())  # "B" - follows MRO
print(D.__mro__)
# (<class '__main__.D'>, <class '__main__.B'>, <class '__main__.C'>, <class '__main__.A'>, <class 'object'>)

Without MRO, Python wouldn't know whether to call B.method() or C.method(). The MRO provides a consistent, predictable order using the C3 linearization algorithm.

💡 Practical Use Cases

1. Super() in Cooperative Inheritance

super() follows the MRO, not just the parent class. This enables cooperative multiple inheritance:

class Logger:
    def save(self):
        print("Logging...")
        super().save()  # Continues MRO chain

class Database:
    def save(self):
        print("Saving to database...")

class Model(Logger, Database):
    def save(self):
        print("Validating...")
        super().save()  # Calls Logger.save()

model = Model()
model.save()
# Output:
# Validating...
# Logging...
# Saving to database...

2. Mixin Classes

MRO makes mixins powerful by ensuring methods are called in the right order:

class TimestampMixin:
    def save(self):
        self.updated_at = "2025-08-11"
        super().save()

class ValidationMixin:
    def save(self):
        if not hasattr(self, 'name'):
            raise ValueError("Name required")
        super().save()

class BaseModel:
    def save(self):
        print(f"Saving {self.name} at {self.updated_at}")

class User(ValidationMixin, TimestampMixin, BaseModel):
    def __init__(self, name):
        self.name = name

user = User("Alice")
user.save()  # Validates, adds timestamp, then saves
# Output: Saving Alice at 2025-08-11

3. Framework Method Overriding

Web frameworks like Django use MRO to let you override specific behaviors while keeping others:

class BaseView:
    def get(self):
        return self.render()

    def render(self):
        return "Base rendering"

class AuthMixin:
    def get(self):
        if not self.is_authenticated():
            return "Login required"
        return super().get()

    def is_authenticated(self):
        return False

class MyView(AuthMixin, BaseView):
    def render(self):
        return "Custom rendering"

view = MyView()
print(view.get())  # "Login required" - AuthMixin.get() runs first
print(MyView.__mro__)
# Shows: MyView -> AuthMixin -> BaseView -> object

🚀 Key Takeaways

__mro__ defines the method lookup order in inheritance hierarchies
It solves the diamond problem using C3 linearization
Order matters in multiple inheritance: class Child(Parent1, Parent2) searches Parent1 before Parent2
super() follows the MRO, not just the immediate parent
Understanding MRO is essential for designing effective mixin classes and using frameworks

Remember: when in doubt, check YourClass.__mro__ to see the exact lookup order!

How to patch vulnerable sub-dependencies with overrides in package.json

Bharathvaj — Mon, 28 Jul 2025 05:26:00 +0000

Suppose you’re building a front end app and your package.json includes react-scripts (used by Create React App). Suddenly, your security scanner flags a high-severity vulnerability in lodash—but you’re not using lodash directly.

Instead, lodash is a dependency of several libraries, including react-scripts, and those libraries haven’t updated yet.

You want to fix the vulnerability now, not weeks later.

Solution

Use overrides in package.json. Add an override to force every dependency to use a safe version of lodash:

{
  "overrides": {
    "lodash": "4.17.21"
  }
}

After running npm install, all nested dependencies will use the patched version of lodash, even if their maintainers haven’t updated yet.

Target and patch specific dependency

You can also target and override a dependency for a specific package, rather than applying it globally.

Suppose you use express, which relies on lodash@3.16, and you need to upgrade just lodash used by express:

{
  "overrides": {
    "express": {
      "lodash": "4.17.21"
    }
  }
}

Nested Patch

You can also do nested patch, Let’s say express → body-parser → qs, and only qs inside body-parser needs an update:

{
  "overrides": {
    "express": {
      "body-parser": {
        "qs": "6.11.0"
      }
    }
  }
}

Other Common Use Cases

Resolve version conflicts: Make all dependencies use a consistent version, avoiding duplication and bugs.
Test unreleased or experimental versions: Try a fix or feature from a specific sub-dependency before it’s officially released.
Quickly unblock development: Patch issues in your dependency tree without forking or waiting on upstream changes.

With overrides, you control your dependency tree. It’s a lifesaver when security and stability can’t wait.

Happy Maintenance!!

Understanding the punycode Deprecation Warning in Node.js

Bharathvaj — Sun, 18 May 2025 06:18:10 +0000

If you’ve seen this warning in your Node.js application recently you're not alone. In this post, we’ll break down what this warning means, why it happens, and how to fix it.

(node:xxxx) [DEP0040] DeprecationWarning: `punycode` module is deprecated. Please use a third-party alternative instead.

🧩 What Is `punycode`?

The punycode module implements the Punycode algorithm, which converts Unicode characters (like emojis or characters with accents) into ASCII-safe strings that can be used in domain names.

This is especially important for internationalized domain names (IDNs) like:

mañana.com → xn--maana-pta.com

✨ How It Works (Simplified)

The Punycode algorithm:

Starts with all basic ASCII characters.
Encodes non-ASCII characters (like ñ or ü) using a base-36 compression scheme.
Adds a xn-- prefix to indicate it's a Punycode-encoded domain.

So mañana.com becomes xn--maana-pta.com, a valid DNS-compliant domain name.

🛑 Why Was `punycode` Deprecated in Node.js?

Node.js removed the core punycode module from the default API starting in 21, for a few reasons:

✅ It’s rarely needed in modern applications.
✅ Modern web APIs handle punycode automatically (more on that below).
✅ Node core was being slimmed down to encourage modularity.
✅ A maintained version still exists on npm if needed.

✅ What’s the Alternative?

1. Use the npm package (still maintained)

If you really need to work with punycode directly:

npm install punycode

const punycode = require('punycode/');

const unicode = 'mañana.com';
const ascii = punycode.toASCII(unicode);

console.log(ascii); // 'xn--maana-pta.com'

🔍 Note: Use require('punycode/') to avoid unexpected behavior.

2. Use the WHATWG `URL` API (Preferred for most use cases)

Modern Node.js versions (v10+) support the WHATWG URL API, which automatically handles punycode encoding:

const url = new URL('https://mañana.com');
console.log(url.hostname); // 'xn--maana-pta.com'

It’s browser-compatible, standards-based, and requires no extra packages.

🧪 Example: Replacing `punycode` with `URL`

Old code using punycode:

const punycode = require('punycode/');
const domain = 'mañana.com';
console.log(punycode.toASCII(domain)); // 'xn--maana-pta.com'

Modern alternative:

const domain = new URL('http://mañana.com').hostname;
console.log(domain); // 'xn--maana-pta.com'

🧼 Wrap-Up

The punycode deprecation warning is harmless — but it’s a signal to update your codebase for modern Node.js compatibility. In most cases, you don’t need to install anything. Just switch to the native URL API, and you’re good to go.

Need help upgrading legacy code or migrating URL logic? Feel free to reach out!

Avoiding the God Object - A Guide to Better Software Design

Bharathvaj — Sun, 18 May 2025 05:55:04 +0000

In software design, a God Object (or God Class) is a class that knows too much or does too much. It's a common code smell that tends to emerge as a system evolves without proper boundaries.

What Is a God Object?

A God Object violates the Single Responsibility Principle by taking on multiple responsibilities. It typically centralizes control and knowledge of many parts of a system—handling tasks that should be delegated to other specialized components.

Why Do We Create Them?

Even experienced developers can fall into this trap, usually due to:

Moving fast: Prioritizing speed over modularity in early stages.
Avoiding perceived complexity: Putting logic in one place “for convenience.”
Fear of breaking things: As the class grows, refactoring becomes scarier.
Lack of clear ownership boundaries: Developers aren't sure where logic belongs.

Example of a God Object

Let’s say you're building a food delivery service and start with a class like this:

class DeliveryManager:
    def assign_driver(self, order): ...
    def calculate_fee(self, order): ...
    def notify_customer(self, order): ...
    def refund_order(self, order): ...
    def log_event(self, event): ...

Over time, DeliveryManager becomes a dumping ground for unrelated concerns: logistics, billing, notifications, logging, and more.

Refactoring the God Object

To fix this, you’d separate concerns into smaller, focused classes:

class DriverAssigner:
    def assign(self, order): ...

class FeeCalculator:
    def calculate(self, order): ...

class CustomerNotifier:
    def notify(self, order): ...

class PaymentProcessor:
    def refund(self, order): ...

class EventLogger:
    def log(self, event): ...

Now, a high-level class can orchestrate these responsibilities without owning the details:

class DeliveryService:
    def __init__(self, assigner, calculator, notifier, processor, logger):
        self.assigner = assigner
        self.calculator = calculator
        self.notifier = notifier
        self.processor = processor
        self.logger = logger

    def process_order(self, order):
        self.assigner.assign(order)
        fee = self.calculator.calculate(order)
        self.notifier.notify(order)
        self.logger.log(f"Order processed with fee {fee}")

When a God Object Might Be Acceptable

There are rare scenarios where a centralized object is acceptable:

Prototypes or scripts: When you’re experimenting or working solo.
Game engines: A “world” object sometimes needs to hold global state.
Tiny microservices: If there’s only one job to do and no team dependency.

But even then, use them with clear documentation and a plan to refactor if the scope grows.

Why You Should Avoid God Objects

Hard to test: Everything depends on everything.
Poor readability: New developers won’t know where to look or change.
Risky changes: One bug fix might introduce five new ones.
Team bottlenecks: Everyone needs to modify the same big class.

Final Thoughts

The God Object is an easy pitfall that feels efficient—until it isn’t. Build smaller, focused classes and let them work together. Clear boundaries make your codebase easier to maintain, test, and scale.

Think of it this way: A chef doesn’t build the kitchen, grow the vegetables, and drive the delivery van. Your code shouldn’t either.

Happy Designing!.

Quartz CRON Explained

Bharathvaj — Sat, 03 May 2025 12:21:45 +0000

At work, I recently encountered Quartz CRON — a more powerful and flexible version of the classic UNIX CRON. It's mainly used in Java applications with the Quartz Scheduler, and I found it really useful for managing complex job schedules.

If you're looking to understand the basics of CRON, check out my previous blog for a detailed explanation.

🧩 Quartz CRON Syntax

Unlike UNIX CRON, which uses 5 fields, Quartz CRON has 7 fields, including seconds and an optional year field. Here’s what it looks like:

┌───────────── second (0 - 59)
│ ┌─────────── minute (0 - 59)
│ │ ┌───────── hour (0 - 23)
│ │ │ ┌─────── day of month (1 - 31)
│ │ │ │ ┌───── month (1 - 12 or JAN-DEC)
│ │ │ │ │ ┌─── day of week (1 - 7 or SUN-SAT)
│ │ │ │ │ │ ┌─ year (optional)
│ │ │ │ │ │ │
* * * * * * *

✅ Why It’s Better

Second-level precision: You can schedule jobs as frequently as every second.
Optional year field: This makes it easy to add or omit year-based scheduling.
Flexible day/week options: It supports advanced patterns like “first Monday of the month” or “last day of the month.”

⚙️ Common Examples

0 0 12 * * ? → Every day at 12 PM
0 15 10 ? * MON-FRI → 10:15 AM, Monday to Friday
0 0/5 14 * * ? → Every 5 minutes starting at 2 PM

⚠️ Quartz vs UNIX CRON

Quartz CRON extends the traditional format with:

7 fields instead of 5.
Second-level precision.
Special characters like ?, L, W, and #.

Happy Scheduling!

Mastering CRON Expressions - A Developer's Quick Guide

Bharathvaj — Sat, 03 May 2025 11:48:53 +0000

CRON expressions are powerful tools for scheduling tasks — from backups to emails — across servers, cloud platforms, and CI/CD pipelines. While they may look cryptic at first, once you decode the pattern, you unlock a whole new level of automation.

🔹 What is a CRON Expression?

A CRON expression is a string with 5 space-separated fields (sometimes 6 or 7 depending on the system) that define a recurring schedule.

┌──────── minute (0 - 59)
│ ┌────── hour (0 - 23)
│ │ ┌──── day of month (1 - 31)
│ │ │ ┌── month (1 - 12 or JAN-DEC)
│ │ │ │ ┌─ day of week (0 - 6 or SUN-SAT)
│ │ │ │ │
* * * * *

🔹 Common Examples

Expression	Meaning
`0 0 * * *`	Every day at midnight
`/5 * * *`	Every 5 minutes
`0 9 * * 1-5`	9 AM on weekdays
`30 14 1 * *`	2:30 PM on the 1st of each month
`@daily`	Shortcut for `0 0 * * *`

🔹 Special Characters

Symbol	Meaning
`*`	Every value
`,`	List (e.g., `MON,WED,FRI`)
`-`	Range (e.g., `1-5`)
`/`	Step (e.g., `*/10` → every 10 units)

🔹 Special Strings

String	Equivalent CRON	Description
`@reboot`	—	Run at system startup
`@yearly`	`0 0 1 1 *`	Once a year
`@monthly`	`0 0 1 * *`	Once a month
`@weekly`	`0 0 * * 0`	Every Sunday
`@daily`	`0 0 * * *`	Every day at midnight
`@hourly`	`0 * * * *`	Every hour

🔹 Tips & Tools

Always test your expressions using tools like crontab.guru or Cronhub.
This format changes based on the system. For example, the Quartz CRON scheduler has a slightly different format to support extra capabilities, such as seconds and nth recursive patterns, like the 3rd Friday of the month. Similarly AWS Cloudwatch has different one.

How Generators improves performance ?

Bharathvaj — Thu, 01 May 2025 16:27:16 +0000

In modern software systems, we often work with large datasets — think server logs, customer records, telemetry streams, or product catalogs. Loading everything into memory just to loop over it can easily bring your system to its knees.

The smarter way? Use iterators or generators that process data one piece at a time.

🌍 Real-World Use Case: Processing a Large Log File

Imagine you're analyzing a log file with 1 million entries — stored line-by-line. You want to extract specific events or count errors. Let’s see how to do this efficiently in Python and JavaScript.

❌ Memory-Heavy: Read Entire File in Memory

const fs = require("fs");

const lines = fs.readFileSync("server.log", "utf-8").split("\n");

for (const line of lines) {
  if (line.includes("ERROR")) {
    console.log("Found an error:", line);
  }
}

This loads the entire file into memory, which may be fine for small logs — but dangerous for huge files.

✅ Memory-Efficient: Async Generator

const fs = require("fs");
const readline = require("readline");

async function* readLogLines(filePath) {
  const fileStream = fs.createReadStream(filePath);
  const rl = readline.createInterface({ input: fileStream });

  for await (const line of rl) {
    yield line; // yield one line at a time
  }
}

// Process the log
(async () => {
  for await (const line of readLogLines("server.log")) {
    if (line.includes("ERROR")) {
      console.log("Found an error:", line);
    }
  }
})();

While this reads line one after another, making it efficient.

In Python,

def read_log_lines(file_path):
    with open(file_path, 'r') as file:
        for line in file:
            yield line.strip()  # Yield one line at a time

# Process the log
for line in read_log_lines("server.log"):
    if "ERROR" in line:
        print("Found an error:", line)

✅ Key Takeaways

Technique	Memory Usage	Suitable For
Full Load	High	Small datasets
Generator/Stream	Low	Large files, real-time streams, infinite data

🧠 Summary

Iterators and generators let you process large data without blowing up memory.
Use them when working with:
- Big files (e.g., logs, exports)
- API pagination
- Sensor data
- Infinite sequences (e.g., timers, streams)
And best of all — you still get to use the familiar for loop syntax.

📌 Pro Tip: Always ask: Do I really need the entire dataset in memory? If not, generators might be your best friend.

Iterator vs Generator — What’s the Difference?

Bharathvaj — Thu, 01 May 2025 16:06:06 +0000

Iterators and generators are closely related and often used together, they are not the same. Let’s break it down clearly.

🔁 Iterator

An iterator is an object that defines a sequence and potentially a return value upon completion. It must implement a next() method that returns an object with value and done properties.

Use iterators when:

You want fine-grained control over iteration.
You are implementing custom data structures (e.g., trees, graphs) that need to conform to the iterable protocol.
You want to manually manage the state of iteration.

Example

function createIterator(arr) {
  let index = 0;
  return {
    next: function () {
      if (index < arr.length) {
        return { value: arr[index++], done: false };
      } else {
        return { done: true };
      }
    }
  };
}

const it = createIterator([10, 20, 30]);
console.log(it.next()); // { value: 10, done: false }
console.log(it.next()); // { value: 20, done: false }
console.log(it.next()); // { value: 30, done: false }
console.log(it.next()); // { done: true }

⚙️ Generator

A generator is a special function using function* syntax that can pause and resume execution using the yield keyword.

Use generators when:

You need lazy evaluation or infinite sequences.
You want to create iterators more simply.
You need to pause and resume logic (like co-routines).
You're working with asynchronous flows (using async generators with for await).

Example

function* numberGenerator() {
  yield 10;
  yield 20;
  yield 30;
}

const gen = numberGenerator();
console.log(gen.next()); // { value: 10, done: false }
console.log(gen.next()); // { value: 20, done: false }
console.log(gen.next()); // { value: 30, done: false }
console.log(gen.next()); // { value: undefined, done: true }

🆚 Key Differences

Feature	Iterator	Generator
Syntax	Manual object with `next()`	Uses `function*` and `yield`
Code Complexity	More verbose	Concise and readable
State Management	Manual	Automatically managed
Reusability	Custom setup needed	Can be reused easily
Use in Loops	Harder, manual loop	Works with `for...of` out of the box
Lazy Evaluation	Yes, but manually	Yes, naturally supported
Pause/Resume logic	No	Yes, with `yield`
Async Support	No (without Promises)	Yes, via `async function*`

Examples

🐍 Python Example

Iterator

class MyIterator:
    def __init__(self, limit):
        self.limit = limit
        self.current = 0

    def __iter__(self):
        return self

    def __next__(self):
        if self.current >= self.limit:
            raise StopIteration
        val = self.current
        self.current += 1
        return val

it = MyIterator(3)
for val in it:
    print(val)  # 0 1 2

Generator

def my_generator(limit):
    for i in range(limit):
        yield i

for val in my_generator(3):
    print(val)  # 0 1 2

🌐 JavaScript Example

Iterator (Manual)

const myIterator = {
  current: 0,
  limit: 3,
  [Symbol.iterator]() {
    return {
      current: this.current,
      limit: this.limit,
      next() {
        if (this.current < this.limit) {
          return { value: this.current++, done: false };
        }
        return { done: true };
      }
    };
  }
};

for (const item of myIterator) {
  console.log(item);
}

Generator

function* myGenerator(limit) {
  for (let i = 0; i < limit; i++) {
    yield i;
  }
}

for (const val of myGenerator(3)) {
  console.log(val); // 0 1 2
}

You can also have async generator but you need to await the iterator.

async function* myGenerator(limit) {
  for (let i = 0; i < limit; i++) {
    yield i;
  }
}

for await (const val of myGenerator(3)) {
  console.log(val); // 0 1 2
}

✅ Summary

Use generators when you want a simpler, more powerful way to create iterators or need to pause/resume execution. When performance optimization is needed.
Use iterators when you want custom control over how iteration works or need to implement the iterator protocol manually.

What kind of database should you choose for your startup app?

Bharathvaj — Thu, 01 May 2025 12:26:14 +0000

When you’re starting to build an app for your startup, one common question is: what kind of database should we use?

My answer would be surprisingly simple:

👉 Just go with a database you’re most comfortable with.

Whether it's SQL or NoSQL, your productivity matters the most in the early stages.

That said, SQL databases often have a slight edge:

✅ More mature
✅ Better for data analysis
✅ Higher fidelity
✅ Easier to use for many developers
✅ Still cost-effective

NoSQL can shine in cost-sensitive, unstructured scenarios — but unless you have a strong reason, SQL is a great starting point.

Start simple. You can always optimize later.

This meets the system design evaluation metrics that I've mentioned in my previous blog.

Happy Building!.

How Async Generators Work with SSE in JavaScript

Bharathvaj — Tue, 29 Apr 2025 16:57:31 +0000

Async generators in JavaScript provide an elegant way to handle Server-Sent Events (SSE)—especially when dealing with token-by-token or chunked data streams from LLMs.

Let’s explore how they help you handle real-time streaming with ease.

🎥 What is SSE?

SSE (Server-Sent Events) lets the server push text messages to the browser over a single HTTP connection. Each message looks like:

data: Hello

data: world!

SSE is great for:

Real-time logs
Chat apps
Streaming LLM responses

🔄 Enter Async Generators

An async function* lets you:

Read from a stream incrementally
Use await while iterating
Yield chunks as soon as they arrive

✨ Example: Consuming SSE with Async Generator

async function* parseSSE(stream) {
  const reader = stream.getReader();
  const decoder = new TextDecoder();
  let buffer = '';

  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    buffer += decoder.decode(value, { stream: true });

    const lines = buffer.split('\n');
    buffer = lines.pop();

    for (const line of lines) {
      if (line.startsWith('data: ')) {
        yield line.slice(6);
      }
    }
  }
}

Usage:

const response = await fetch('/api/stream', {
  method: 'POST',
  headers: { Accept: 'text/event-stream' }
});

for await (const chunk of parseSSE(response.body)) {
  console.log('Chunk:', chunk);
}

🚀 Benefits

✅ Clean syntax with for await...of
✅ Supports POST & headers (good for LLMs)
✅ Works in Node.js (with fetch polyfill)
✅ Custom parsing logic

🔚 Summary

Async generators give you streaming superpowers with SSE:

Handle LLM output in real-time
Decode and react to each message chunk
Maintain control over request method, headers, and stream parsing

Perfect for modern AI-powered apps using LLM APIs that stream!

EventSource vs Async Generator - What to Use for LLM Streaming?

Bharathvaj — Tue, 29 Apr 2025 16:46:32 +0000

When it comes to consuming streaming responses in the browser, especially from Server-Sent Events (SSE), two main approaches emerge:

Using the native EventSource API
Using fetch with an async generator to manually parse the stream

Let’s break down the differences and when to use each.

🔁 Quick Comparison

Feature	`EventSource`	`fetch` + Async Generator
HTTP Method	GET only	✅ Supports POST, PUT
Custom Headers	❌ Limited	✅ Full control
Streaming Support	✅ Yes	✅ Yes
LLM API Friendly	❌ No	✅ Yes
Reconnection	✅ Auto	❌ Manual
Binary Support	❌ No	✅ Yes
Browser Support	✅	✅ (modern browsers)
Node.js Use	❌	✅ (with polyfills)

🧠 When to Use What?

✅ Use `EventSource` if:

Your server supports only GET-based SSE
You need auto-reconnect
You’re building a simple real-time dashboard

const es = new EventSource('/events');
es.onmessage = (e) => console.log(e.data);

✅ Use `fetch` + async generator if:

You're working with LLM APIs that require POST or auth headers
You want fine-grained control over stream parsing

async function* streamSSE(response) {
  const reader = response.body.getReader();
  const decoder = new TextDecoder();
  let buffer = '';

  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    buffer += decoder.decode(value, { stream: true });

    const lines = buffer.split('\n');
    buffer = lines.pop(); // Incomplete line

    for (const line of lines) {
      if (line.startsWith('data: ')) yield line.slice(6);
    }
  }
}

🔚 Summary

Use Case	Best Approach
Authenticated LLM streaming	`fetch` + async generator
Lightweight real-time updates	`EventSource`

Forem: Bharathvaj

The Dunning-Kruger Effect - Why Beginners Feel Like Experts

🧠 What Is the Dunning-Kruger Effect?

📈 The Confidence Curve

💻 Example 1: The Junior Developer Syndrome

🚗 Example 2: The New Driver Phenomenon

💰 Example 3: The Crypto Trading Expert

🎯 Example 4: The Git "Expert"

🛡️ How to Combat It

1. Embrace the Beginner's Mind

2. Seek Feedback Actively

3. Learn in Public

4. Find Your Valley of Despair

🎭 The Flip Side: Imposter Syndrome

🔍 Real-World Implications

In Tech Teams

In Startups

💡 The Sweet Spot

Final Thoughts

Python's __mro__

🔍 What is mro?

🎯 Why Do We Need It?

💡 Practical Use Cases

1. Super() in Cooperative Inheritance

2. Mixin Classes

3. Framework Method Overriding

🚀 Key Takeaways

How to patch vulnerable sub-dependencies with overrides in package.json

Solution

Target and patch specific dependency

Nested Patch

Other Common Use Cases

Understanding the punycode Deprecation Warning in Node.js

🧩 What Is punycode?

✨ How It Works (Simplified)

🛑 Why Was punycode Deprecated in Node.js?

✅ What’s the Alternative?

1. Use the npm package (still maintained)

2. Use the WHATWG URL API (Preferred for most use cases)

🧪 Example: Replacing punycode with URL

🧼 Wrap-Up

Avoiding the God Object - A Guide to Better Software Design

What Is a God Object?

Why Do We Create Them?

Example of a God Object

Refactoring the God Object

When a God Object Might Be Acceptable

Why You Should Avoid God Objects

Final Thoughts

Quartz CRON Explained

🧩 Quartz CRON Syntax

✅ Why It’s Better

⚙️ Common Examples

⚠️ Quartz vs UNIX CRON

Mastering CRON Expressions - A Developer's Quick Guide

🔹 What is a CRON Expression?

🔹 Common Examples

🔹 Special Characters

🔹 Special Strings

🔹 Tips & Tools

How Generators improves performance ?

🌍 Real-World Use Case: Processing a Large Log File

❌ Memory-Heavy: Read Entire File in Memory

✅ Memory-Efficient: Async Generator

In Python,

✅ Key Takeaways

🧠 Summary

Iterator vs Generator — What’s the Difference?

🔁 Iterator

Example

⚙️ Generator

Example

🆚 Key Differences

Examples

🐍 Python Example

Iterator

Generator

🌐 JavaScript Example

Iterator (Manual)

Generator

Python's mro

🧩 What Is `punycode`?

🛑 Why Was `punycode` Deprecated in Node.js?

2. Use the WHATWG `URL` API (Preferred for most use cases)

🧪 Example: Replacing `punycode` with `URL`

✅ Use `EventSource` if:

✅ Use `fetch` + async generator if: