Forem: Prasad Rane

Starting Enterprise-wide Kafka Governance

Prasad Rane — Wed, 06 May 2026 17:35:52 +0000

Technical leadership isn't always about having the right title; it’s about having the right influence. When you are operating without direct authority, setting standards across autonomous teams requires a shift from "enforcing mandates" to "solving shared pain."

The Mess: Visualizing the Chaos

It began on a Tuesday in a conference room, facing a whiteboard filled with chaotic arrows representing event flows. The AWS MSK dashboard displayed a patchwork of inconsistent topic names.

loan-processing.loancreated
LoanService.Events.Created
mortgage.v2.loan.originated

These topics often performed the same functions but were impossible to correlate during system failures. Same intent. Same cluster. Completely different conventions.

When something broke, tracing ownership or flow was guesswork.

But naming inconsistencies were just the symptom.

IAM policies ranged from overly permissive to painfully restrictive
Schema evolution had no guardrails
Ownership was unclear
Every team used Kafka differently

The real issue was simple:

Everyone depended on Kafka. Nobody owned its governance.

That’s how technical debt quietly compounds; until it becomes operational risk.

The Realization: Governance is a Missing System, Not a Missing Rule

Sitting down, one thought became clear:

We’re building technical debt faster than we can document it.

Without intervention, cross-service debugging would become unsustainable within a year. But there was a constraint:

No central authority
No mandate power
Fully autonomous teams This wasn’t a tooling problem. It was a coordination problem.

Meeting Invite: Start with Pain, Not Policy

Instead of proposing a solution, I invited collaboration by framing it as a way to fix shared frustrations.

That framing mattered. No one wants another rulebook. But everyone wants their pain heard.
Five engineers showed up:
2 from platform engineering
3 from application teams

That was enough to start.

The Meeting: Shifting from Skepticism to Engagement

In the meeting, I led with pain points rather than solutions, showing real examples of redundant and confusing topics on the MSK dashboard.
Then someone said:

“We got paged last week because a producer couldn’t write to a topic. Nobody knew how to fix it.”

That changed the room. We moved from:

“Why are we here?”

“We need to fix this.”

That shift from abstract discussion to shared pain is where influence begins.

Drafting Standards: The Technical Blueprint

Over the next two weeks, I drafted the first version of a governance framework. It focused on three pillars:

1.Topic Naming Convention

Convention: {domain}.{service}.{event-type}.{version}
Example: loan.loanservice.loan-created.v1

Goal: Make the topic names - discoverable, searchable, self-descriptive, and future-proof with versioning.

2. IAM Policies (Least Privilege)

Creating mandatory least-privilege templates tied to service roles to eliminate wildcards.

Goal: Make the secure path the default path.

3. Schema Evolution (Avro + Compatibility)

Enforced Avro with BACKWARD compatibility which is integrated with schema registry and gets checked before deployment.

Goal: Prevent breaking downstream consumers silently

Automation: If It’s Not Enforced, It’s Optional

Documentation alone doesn’t scale. So we embedded the standards into tooling.

Terraform Modules: Topic provisioning templates now included regex validation for naming and pre-configured IAM defaults.
CI/CD Linters: We built a linter to flag violations in service configuration files before deployment.
Example failure: FAILED: topic name 'test_topic' does not match pattern
Example success: PASS: topic 'loan.loanservice.loan-created.v1'
Gate: Automation turned guidance into a "gate," ensuring that the right way was also the only way to deploy.
That’s when governance becomes real:

Not a suggestion. A system constraint.

The Tipping Point: Adoption Without Intervention

Three months later, something subtle happened. I was reviewing a PR from a team I’d never worked with.

Everything passed:

Naming ✔
IAM ✔
Schema ✔

No feedback needed. That’s when I knew:

The system didn’t need me anymore.

And that’s the goal of good governance.

Key Takeaway

This experience reshaped how I think about technical leadership:

Standards don’t succeed because they’re correct.
They succeed because people believe in them.

What Actually Worked

Start with pain, not policy
Involve stakeholders early
Facilitate, don’t dictate
Bake standards into tooling

Final Thought

If you're working on platform or architecture problems in a multi-team environment, remember:

You don’t need authority to create alignment.

You need:

clarity
empathy
and systems that reinforce good decisions

Standards stick when they make people’s lives better — not when they add gates.

The Feynman Algorithm: A Developer’s Guide to "Thinking Very Hard"

Prasad Rane — Mon, 27 Apr 2026 16:06:50 +0000

If you search for Richard Feynman’s methods, you’ll likely stumble upon the famous Feynman Technique, a brilliant framework for learning new concepts by explaining them to a child.

But there’s another, lesser-known Feynman method. It was jokingly coined by his colleague, physicist Murray Gell-Mann, who said that Feynman’s process for solving problems looked like this:

Write down the problem.

Think very hard.

Write down the solution.

At first glance, this sounds like a meme. "Draw the rest of the owl," right? But if we look closer at this "Feynman Algorithm," it actually perfectly outlines the lifecycle of debugging and writing software.
Let’s break down how this tongue-in-cheek algorithm can actually help you become a better developer.

Step 1: Write down the problem ✍️

Albert Einstein supposedly said, "If I had an hour to solve a problem, I'd spend 55 minutes thinking about the problem and 5 minutes thinking about solutions."

In programming, we often skip this. We see an error log and immediately start changing variables and refreshing the browser. But "writing down the problem" forces you to define exactly what is going wrong.

Bad Problem Definition:

"The login page is broken."

Good Problem Definition:

"When a returning user tries to log in with Google OAuth, the API returns a 500 server error, but only on mobile Safari."

When you write the problem down clearly, the scope narrows. Half the time, simply defining the problem accurately reveals the solution.

Step 2: "Think very hard" 🧠

This is the punchline of the joke. To outsiders, Feynman just stared at a chalkboard and magically found the answer. But "thinking very hard" doesn't mean having a massive IQ; it means engaging in structured, focused analysis.

How do developers "think very hard"?

We Rubber Duck: We explain the code line-by-line to an inanimate object (or a patient coworker).

We Isolate: We comment out chunks of code to see if the bug still happens. We narrow down the variables.

We Step Away: Sometimes, "thinking very hard" means taking a shower or going for a walk. Your brain's "diffuse mode" keeps working on the problem in the background.

"Thinking very hard" means resisting the urge to copy-paste the first Stack Overflow answer you see without understanding why it works.

Step 3: Write down the solution 💡

You figured it out! You fixed the bug! Time to push to production and go home, right? Not quite.

In the context of coding, "writing down the solution" means three things:

Writing clean code: Making sure your solution isn't just a hacky workaround, but a readable, maintainable piece of logic.

Writing tests: Proving that your solution actually works and ensuring it won't break in the future.

Writing documentation: Leaving comments explaining why you did what you did.

Your future self (and your team) will thank you when they read a comment like:

// Using a Set here instead of an Array because checking for duplicates was causing a massive performance bottleneck.

Final Thoughts

The Feynman Algorithm might have started as a joke among Nobel Prize-winning physicists, but it’s remarkably practical for software engineers.

Next time you are stuck on a nasty bug, don't just start mashing the keyboard. Stop. Write the problem down. Take a walk to think very hard. And when you solve it, document it for the next developer.

Let me know in the comments how you practice "thinking very hard" when you are stuck on a tricky coding problem!