Forem: Tomer Ben David

Choosing the Right Shortest Path Algorithm

Tomer Ben David — Sat, 11 Apr 2026 07:29:10 +0000

Shortest path problems on LeetCode vary by constraint. Graphs can have weights, no weights, single source focuses, or all pairs requirements. Some have positive costs and others have negative costs.

Each specific situation has a corresponding algorithm. Understanding the constraints of the graph dictates the strategy.

Identifying the Graph

Before writing code, verify the terrain.

The Clear Graph

The first question is whether every step costs the same.

Calculating degrees of separation in a social network or moving between cells in a maze means the costs are uniform.
Dealing with traffic where one road takes 5 minutes and another takes 50, flight prices, or effort means each step has a unique cost. These are weighted graphs.

The Disguised Graph

Sometimes the problem hides the graph.

The Matrix: A 2D grid where each cell is a node and valid moves are edges. If moving to an adjacent cell costs 1, it is a simple BFS.
State transitions: Consider Word Ladder. Each word is a node and a one character difference is the edge. Since every transform costs 1, this is a BFS problem.
Resource management: Problems like Cheapest Flights Within K Stops are weighted graphs requiring you to track cost while adhering to state constraints.

Selection Logic

Select the algorithm based on what the graph requires.

BFS

If every step costs the same, use Breadth-First Search. The first time the search reaches a node is the shortest path.

Dijkstra

When roads have different lengths but they are all positive, use Dijkstra. A discovery at one point in the search assumes no future path through a positive weight road can make it better.

Bellman-Ford

If a path provides a negative cost, Dijkstra fails. Bellman-Ford handles negative weights and detects cycles where a path keeps getting cheaper forever.

Floyd-Warshall

If the problem requires the shortest path from every node to every other node, use Floyd-Warshall. This checks every node as a possible layover to solve for all pairs.

Escalation of Power

As graph rules become more complex, the algorithms become heavier.

BFS is fastest but cannot handle weights.
Dijkstra handles weights but requires a priority queue and fails on negative costs.
Bellman-Ford handles negatives and cycles but uses repeated loops.
Floyd-Warshall handles all pairs but uses triple nested loops.

The Brute Force Hack

You do not always need the most efficient algorithm to pass the interview. If you struggle to implement the minHeap logic for Dijkstra, use Bellman-Ford as a brute force alternative.

You do not need a priority queue. Take the core idea of edge relaxation.

Each pass through all edges discovers the shortest path using one additional edge. The first pass finds shortest paths with one edge, the second pass finds shortest paths with two edges, and so on. Since a shortest path in a graph of $V$ nodes can have at most $V-1$ edges, this ensures every node is covered. It is two nested loops and handles everything Dijkstra can.

# The brute force alternative
# n: number of nodes, edges: list of (u, v, weight)
def shortest_path_hack(n, edges, start):
    dist = [float('inf')] * n
    dist[start] = 0

    # Its just a nested loop and you could pass the in without Dijkstra.
    for _ in range(n - 1):
        for u, v, w in edges:
            if dist[u] + w < dist[v]:
                dist[v] = dist[u] + w

    # No need to handle weighted and negative edges.
    # We skip this part of belman ford. Quick Win. Two Birds.
    return dist

Originally published at: https://looppass.mindmeld360.com/blog/choosing-shortest-path-algorithm/

System Design Interview - Designing from Invariants

Tomer Ben David — Wed, 08 Apr 2026 06:50:13 +0000

Designing from Invariants

Software architecture is frequently treated as an exercise in connecting infrastructure components. We often reach for Kafka, Redis, or microservice boundaries as if they are the building blocks of the business logic itself. But when tools come before logic, the resulting design prioritizes infrastructure choices over the problem they are meant to solve.

A high reliability system does not start with a distributed queue or a complex workflow engine. It starts with the core constraints the invariants that make the system reliable. If you start by choosing your infrastructure before you have defined the logic that keeps your data correct, you are building complexity on an undefined foundation.

The Distribution Trap

Most designs become unmanageable because they assume every step of a business process must be distributed across new infrastructure from the beginning.

In this style of design, the business logic is spread across a database, a queue, and a workflow engine. To answer a simple question like "What is the state of this payment?", you have to reconstruct the story from multiple logs. This introduces the Dual Write problem where a database update succeeds but a message publish fails before the system has even achieved its basic purpose.

Coherence as the Minimal Solution

The strongest designs identify the Invariants first. An invariant is a statement that must always be true for the business to be valid. For example:

"A cleared risk decision must never exist without an authoritative payment record."

If the business rules require two things to change together to be valid, the simplest and most robust solution is to keep them in the same transaction.

This logical anchor is the Transactional Center.

The core state machine for an important process should have one queryable home, usually a relational database like Postgres. By starting here, you eliminate entire classes of distributed system bugs. You can scale the system outward later, but the authority remains in one place.

Scaling without Scattering

Scaling should be a reaction to a requirement, not a default architecture. The Four Plane Model provides a way to distribute workloads without losing the source of truth.

Plane 1 Transactional Truth

This is the core. It owns the current state, the audit trail, and the records used to reliably notify the rest of the system.

Plane 2 Action Systems

These are Kafka workers and background jobs. They react to the truth committed in Plane 1. Asynchronous tasks like notifications or external fraud checks happen here without slowing down the core transaction.

Plane 3 Real Time Reads

When you need fast dashboards, move those reads to a specialized replica like ClickHouse. This keeps analytical traffic from overwhelming the transactional core.

Plane 4 Historical Analytics

This is for deep history and data science (BigQuery or Snowflake). It stays completely separate from the operational system.

Choosing Your Path

The decision to distribute should always follow the logic of the problem.

Start with a Transactional Center when

Consistency is part of the business value. If a payment must be atomic with an order update, keep them together. This is the simplest possible solution and the most resilient to failure.

Extend to Distributed Choreography when

Domains are truly independent or you have reached a scale where a single database cannot handle the write volume. Use patterns like Sagas only when the local boundary can no longer support the technical requirements of the system.

A resilient system starts by identifying the center. Ask one question: Where is the authority?

Originally published at: https://looppass.mindmeld360.com/blog/system-design-transactional-center/

Memory Types in LangChain

Tomer Ben David — Sun, 15 Mar 2026 09:18:43 +0000

Ever felt like your LLM needs a memory?

LangChain felt the same thing. From full chat transcripts to summaries, entities, and vector backed recall, it gives you several ways to make a stateless model feel like it actually remembers what matters.

Large Language Models are inherently stateless. Every request you send arrives as a blank slate with no recollection of what was discussed five minutes ago. To create a coherent conversation, the system must manually feed previous messages back into the model.

LangChain provides several distinct patterns for managing this history. Choosing the right one is a balance between providing perfect context and managing the cost of every token.

LangChain Memory Types

Use the Transcript Pattern for quick, high precision support tasks.
Use the Window Pattern for predictable, task oriented interactions.
Use the Summary Pattern for long, creative, or collaborative sessions.
Use the Entity Pattern for personal assistants that track user preferences.
Use the Vector Retrieval Pattern for knowledge intensive systems with vast histories.

The Transcript Pattern

The simplest way to maintain a conversation is through a direct buffer. This stores every word exactly as it was spoken in a sequential list.

Every message from the user and every response from the AI is saved verbatim.
The entire history is appended to the prompt for the next turn.
It provides the model with the most accurate and raw context possible.

from langchain.memory import ConversationBufferMemory

memory = ConversationBufferMemory()
memory.save_context({"input": "What is the capital of France?"}, {"output": "The capital of France is Paris."})
memory.load_memory_variables({})

An example of this is a customer support bot helping a user reset a password. The bot needs to remember the specific email address and the error code mentioned two sentences ago to provide a precise solution. While excellent for short interactions, this does not scale for long sessions where the prompt becomes massive.

The Window Pattern

To solve the scaling issue of a raw buffer, we can use a sliding window. This strategy only keeps the most recent portion of the conversation.

The system only remembers the last few interactions, defined by a fixed count.
Older segments are discarded as new ones arrive.
This keeps the prompt size and API costs predictable.

from langchain.memory import ConversationBufferWindowMemory

memory = ConversationBufferWindowMemory(k=2)
memory.save_context({"input": "I live in London"}, {"output": "London is a great city."})
memory.save_context({"input": "What is the weather like?"}, {"output": "It is currently rainy in London."})

A weather assistant is a perfect candidate for this pattern. If you ask for the forecast in London and then ask "What about tomorrow?", the bot only needs the most recent context to understand that you are still talking about London. It does not need to remember that you asked about the news ten minutes ago.

The Summary Pattern

For very long term dialogues, a summarization strategy is more effective. Instead of saving every word, the system maintains a running overview of the discussion.

After each interaction, the system updates a concise summary of the key points.
Only this summary is sent to the primary model as context.
It handles massive transcripts while keeping the context size relatively flat.

from langchain.memory import ConversationSummaryMemory
from langchain_openai import OpenAI

llm = OpenAI(temperature=0)
memory = ConversationSummaryMemory(llm=llm)
memory.save_context({"input": "Explain the plot of Inception"}, {"output": "Inception is about dreams within dreams..."})

Consider a creative writing assistant helping you plot a novel. Over several hours, you might discuss dozens of characters and plot points. Instead of feeding the whole transcript, the system carries a summary that tracks the main objective and the current state of the story.

The Entity Pattern

Some applications require remember specific facts about people or technical concepts without carrying the entire dialogue.

The system extracts key participants or topics mentioned in the chat.
It builds a structured knowledge base about these specific items.
Relevant facts are pulled from storage when the topic resurfaces.

from langchain.memory import ConversationEntityMemory
from langchain_openai import OpenAI

llm = OpenAI(temperature=0)
memory = ConversationEntityMemory(llm=llm)
memory.save_context({"input": "My name is Tomer and I use Kotlin"}, {"output": "Nice to meet you Tomer."})

An example is a personalized coding coach. If you mention that you prefer a specific library like React or a particular cloud provider, the system stores that fact. When you later ask for a code sample, it automatically applies those preferences without needing to reread the original transcript.

The Vector Retrieval Pattern

The most advanced method involves treating the conversation like a database. This allows the model to recall information from any point in the history based on semantic relevance.

Past message snippets are stored in a vector database.
The system performs a search based on the current user query.
It retrieves only the most relevant historical segments.

from langchain.memory import VectorStoreRetrieverMemory
import faiss
from langchain_community.docstore import InMemoryDocstore
from langchain_community.vectorstores import FAISS
from langchain_openai import OpenAIEmbeddings

vectorstore = FAISS(OpenAIEmbeddings().embed_query, faiss.IndexFlatL2(1536), InMemoryDocstore({}), {})
retriever = vectorstore.as_retriever(search_kwargs=dict(k=1))
memory = VectorStoreRetrieverMemory(retriever=retriever)

This is the ideal choice for an AI researcher. If you are discussing a series of academic papers over several weeks, the model can pull a specific detail from a conversation you had ten days ago because it is semantically related to your current question.

Originally published at: https://looppass.mindmeld360.com/blog/langchain-memory-types/

The Battle Between RAG and Long Context

Tomer Ben David — Fri, 13 Mar 2026 06:27:21 +0000

Introduction

Large Language Models arrive with a fundamental limitation known as the knowledge cutoff. They are experts on the world as it existed during their training phase but they are completely blind to your private data or events that happened this morning. Whether it is an internal wiki or a complex codebase, the model cannot see what it was not trained on. To make these systems useful for building products, we have to solve the problem of context injection.

The industry is currently split between two competing philosophies for solving this. One is a complex engineering pipeline while the other is a brute force architectural shift.

The Engineering Complexity of Retrieval Augmented Generation

Retrieval Augmented Generation is the established path for providing context. It works by turning your entire knowledge base into a searchable index. You break your documents into small pieces and store them in a vector database as numerical maps. When a user submits a query, the system performs a semantic search to find the most relevant snippets and hands them to the model for processing.

This remains the essential strategy for massive datasets. If you have ten million technical specifications, you cannot possibly cram them all into a single prompt. This approach acts as a smart filter that protects the model from information overload. It is also more cost efficient for high volume systems because you only pay to process a few hundred words of context instead of millions of tokens every time.

However, this method introduces a retrieval lottery. If your search logic fails to find the exact piece of information required, the model will never see it. You are essentially gambling that your search engine is smart enough to find the needle in a global haystack.

The Simplicity of Long Context Brute Force

A newer alternative is to use models with massive context windows. Instead of building a complex database and retrieval pipeline, you simply paste your entire dataset directly into the prompt. This has been called the no stack stack because it removes the need for infrastructure like vector databases and embedding models entirely.

The primary advantage here is global reasoning. When you give the model every word of the source material, you eliminate the risk of the retrieval lottery. This is superior for tasks that require seeing the whole picture. For example, if you are analyzing a series of incident reports from a distributed system to find a recurring pattern, you want the model to see every log entry simultaneously. In a traditional retrieval system, the search might pull out isolated errors but miss the subtle connection between a load balancer change on Monday and a latency spike on Thursday. By providing the entire history at once, you allow the model to detect deep architectural threads.

The downside is the token tax. You pay the price for every word in your knowledge base on every single turn. These systems can also suffer from attention dilution. When you overwhelm a model with too much information, it may start to ignore or misinterpret details that are buried in the middle of a massive block of text.

Navigating the Infinite Data Problem

For many enterprise environments, the data lake is effectively infinite. A million tokens might sound like a lot, but it is a drop in the ocean compared to the size of a global corporate knowledge base. In these scenarios, retrieval is not just an option but a structural necessity. You cannot brute force a petabyte of data into a prompt regardless of how large the context window becomes.

The choice comes down to the boundaries of your problem. You should use the long context approach for bounded datasets that require deep and interconnected reasoning across every page. You should stick with the engineering approach when you need to navigate vast libraries of information where efficiency and noise reduction are the highest priorities.

Originally posted at: https://looppass.mindmeld360.com/blog/rag-vs-long-context-strategy/

Comparing LangChain, CrewAI, and ADK

Tomer Ben David — Thu, 12 Mar 2026 08:14:40 +0000

Introduction

In the current gold rush of Agentic AI, developers are often caught in Framework Fatigue. Every week, a new library claims to be the standard for building autonomous agents.

The question isn't only about which tool is most popular or which architecture solves a function. Different projects have different requirements, so the real challenge is finding the architecture that best matches your specific needs and your unique friction.

You also have to balance this with your instincts about which framework might catch on as the de facto industry standard. If one of them wins, you want to be on the right side of that curve without sacrificing your specific goals today.

AI Coding and Building Your Own Orchestration

Before we talk about ready made frameworks like LangChain or ADK, we have to acknowledge how the landscape has changed. In the era of AI coding, you don't necessarily need a massive library to get ahead. You can build your own bespoke orchestration layer that fits your project exactly.

When you take the custom orchestration route, you are essentially solving three core technical challenges on your own terms.

First is the Parsing Tax. You need a way to ensure the AI returns structured data like JSON instead of just a paragraph of text. Today this is often solved with simple system prompts or native model features.

Second is State Management. You have to decide how the system remembers previous steps without overflowing the context window.

Third is Loop Control. You need a safety mechanism so an autonomous agent doesn't get stuck in a thought loop and burn through API credits.

The choice today isn't about whether you can build an agent without a library. You definitely can, and for many uncommon projects, building your own thin orchestration is the best way to avoid unnecessary bloat.

LangChain and the Modular Lego Set

LangChain was the first to standardize the chaos. It treated AI workflows like a pipeline or a "Chain."

The philosophy here is modularity. Everything is a component, including prompts, models, output parsers, and tools.

If you need to take a PDF, turn it into vectors, and ask a question, LangChain has a plug for every single part of that process.

The critique for many is that it became a "Thick Platform." The abstractions can sometimes be harder to debug than the raw code itself. It is a massive toolkit that occasionally forces you to learn the LangChain way instead of the standard software engineering way.

CrewAI and the Collaborative Storyteller

As we moved from single chains to Multi Agent Systems, CrewAI arrived with a different mental model of Role Playing.

The philosophy is simple. Don't just give an agent a tool but give it a job. You define a Researcher, a Writer, and a Manager.

It is important to understand that CrewAI is actually built on top of LangChain. It uses the foundational pieces of LangChain to handle the heavy lifting of LLM communication and tool execution while adding the collaborative crew logic on top. It is best for content creation or complex research because it excels at delegating tasks between agents. In these scenarios, it feels less like coding a system and more like managing a crew.

The critique is that because it sits on top of LangChain, it inherits all of that platform's complexity. It is excellent for story driven workflows but can feel like it has too much magic under the hood for high precision systems engineering.

ADK or Google’s Agent Builder Kit

ADK is the production first response. Unlike CrewAI, ADK is a standalone stack that doesn't rely on LangChain. It is a clean slate alternative.

The philosophy treats agents as independent tools that you can plug into any system. It prioritizes writing real code and testing everything on your own machine before going live. While other frameworks can do hierarchy, ADK makes it a core structural primitive by treating entire agents as modular tools that a primary agent can call. It feels much like a system of nested microservices.

This is the best case for enterprise environments where observability and Agent to Agent communication are critical. It’s optimized for Gemini but stays model agnostic via LiteLLM.

The real strength here is that it treats an agent as a unit of deployment. This means the agent isn't just a variable in your code but a standalone service you can ship independently. For example, if you have a Pricing Agent. In a traditional library, that agent is just a function call inside your main application. If you want to update it, you have to redeploy your entire app. With ADK, that Pricing Agent is a standalone service with its own endpoint. You can update it, test it, or scale it without ever touching your main product code. It covers the entire engineering lifecycle, which includes professional evaluation, automated deployment, and production monitoring.

One Weather Task and Three Different Mental Models

To see the difference clearly, lets say we want an agent to check the weather and suggest an outfit. Each framework approaches this differently.

With LangChain, you build a chain of thought. You create a weather tool, give it to an agent executor, and the system runs a loop until it reaches the final answer. You are essentially building a custom logic path.

With CrewAI, you would hire a Weather Expert and a Fashion Stylist. You define their roles and backstories, then assign them a task to collaborate. The Researcher finds the data and the Stylist uses it. You are managing a team meeting.

With ADK, you define a Weather Service as a tool. You create a Weather Agent as a modular unit. Because it is hierarchical, you might have a primary Assistant Agent that simply delegates the request to that specialized unit. In this model, these agents can behave like actual web services that you communicate with via REST APIs. You have total flexibility here. You can choose to run every agent on a single monolithic server if your project is small or you can choose to have specialized agents living on different machines entirely. This allows your system to grow from a simple monolith into a network of independent services that you can update and scale one by one without touching the rest of the codebase. You are architecting for future growth instead of being locked into a single monolithic script.

The Framework Paradox: Avoiding the J2EE Trap

In software history, we often see a pendulum swing between "lightweight libraries" and "heavy platforms." For those who remember the early days of Enterprise Java, the term J2EE often brings back memories of "Thick Platforms" that were so heavy you spent more time configuring the framework than writing the business logic.

The risk with AI frameworks today is falling into that same trap. You start with a tool meant to simplify a task, but as the framework grows to cover every possible edge case, it introduces so much architectural weight that it becomes a burden.

There is a delicate balance to strike. You want enough abstraction to be productive, but not so much that you lose sight of the underlying LLM calls. If you find yourself spending days trying to figure out how to "pass a variable the framework way" instead of just writing a function, you might be carrying too much weight.

Choosing the Right Path for Your Agent Architecture

If you’ve followed my work at MindMeld360, you know I’m wary of Thick Platforms, but the truth is there is no single winner in this space yet. The industry is currently obsessed with finding the perfect library, but the real engineering task is matching the right abstraction level to each specific service you build.

LangChain is a library of parts. CrewAI is a framework for behavior. ADK is a kit for modular systems.

My advice is to start by playing with a custom and thin orchestration layer. You have to understand the problem space first and truly feel the pain that these frameworks are trying to solve. Once you gain your own intuition through a bespoke solution, you can incorporate existing libraries to handle the heavy lifting.

Do not try to build your own massive agent library from scratch for production since these tools are already heavily used and battle tested. Instead, use a stage based approach to grow your experience.

Start custom to feel the domain. Then build your next service with LangChain to see the ecosystem and the drawbacks for yourself.

From there, you can choose the right tool for each job. Use LangChain when you want a common and widely supported library. Use CrewAI when you need a higher level of agent collaboration. Use ADK when you want to distribute your agents as independent services across a network.

Closing Note

By the time this post has been published, we probably already have 5 more libraries to explore! The pace of AI is relentless, but that’s not a bad thing it just means more tools for us to master. More blog posts to come on those, so stay tuned! :)

Originally published at: https://looppass.mindmeld360.com/blog/ai-frameworks-langchain-crewai-adk/

Load Balancing & WebSockets (L4 vs L7)

Tomer Ben David — Mon, 09 Mar 2026 13:01:24 +0000

Introduction

When you build a standard web app, load balancing is usually straightforward because every request is independent. You just spread the traffic around. But once you introduce WebSockets, everything changes. You are no longer dealing with quick requests. You are managing a persistent pipe that might stay open for hours.

The first thing to understand is that WebSockets can work on either Layer 4 or Layer 7. There is no hard rule requiring one over the other. Every load balancer can pass a pocket of bits through. The difference is entirely in how the device treats the connection once it is established.

How Layer 4 handles the traffic

Since WebSockets are built on top of TCP, a Layer 4 load balancer can handle them perfectly. Think of this balancer as a high speed postman who only reads the house number on the envelope. He doesn't know he is routing WebSockets or HTTP. He just sees a raw TCP connection request on a specific port and blindly forwards that stream to a backend server.

This approach works at the TCP level so it is incredibly efficient.
The initial HTTP Upgrade request passes right through the load balancer. The backend server itself handles the handshake and the SSL termination.
It can handle millions of simultaneous connections without breaking a sweat because it doesn't have to decrypt SSL or parse headers.

The main downside mentioned in architectural circles is the NAT trap. Because a Layer 4 balancer only sees IP addresses and ports, it often relies on the source IP to kept the connection sticky. If you have thousands of users in a single office building all sharing one public IP address, the balancer might accidentally send every single one of them to the same backend server. That server will quickly get overwhelmed while the rest of your fleet sits idle.

The intelligence of Layer 7

A Layer 7 load balancer operates at the Application layer and actually understands the HTTP protocol. It is more like a sophisticated concierge who opens the mail to understand exactly who it is for and what they need. This balancer intercepts the traffic, decrypts the SSL, and reads the HTTP headers.

It explicitly sees the Upgrade and Connection headers that define a WebSocket.
Because it reads the headers and cookies, it can route users based on session IDs rather than IP addresses. This completely avoids the NAT trap because every user has a unique cookie even if they share an IP.
You can use path based routing to send specific types of traffic to different server groups. You could send chat traffic to one group and live feeds to another.

The performance trade off here is significant. The balancer has to maintain the state of the persistent WebSocket connection while continuously proxying the decrypted frames back and forth. This requires significantly more RAM and CPU than a simpler Layer 4 setup.

The hybrid approach for global scale

Many massive global applications like Discord and Slack do not choose just one layer. They use a hybrid approach that provides the best of both worlds. They place highly resilient hardware Layer 4 balancers at the network edge to absorb massive traffic spikes and defend against DDoS attacks.

These edge balancers then distribute the traffic to a internal fleet of software based Layer 7 balancers like NGINX or HAProxy. This second fleet handles the smart routing and the persistence needed for the WebSocket lifecycle. This layered strategy provides the raw horsepower to handle the initial connection and the intelligence to manage the application state once it is established.

Originally published at: https://looppass.mindmeld360.com/blog/load-balancing-websockets-l4-l7/

How to Actually use Python's heapq for Kth Largest Problems

Tomer Ben David — Sun, 08 Mar 2026 09:41:45 +0000

If you're using Python for coding interviews, heapq is your best choice for priority queues. But it has a massive quirk that trips up almost everyone. It only supports min heaps.

If you try to use heapq.heapify_max(), your code will crash on most platforms (it's not fully public until Python 3.14).

So, how do you find the Kth largest element if you only have a min heap?

There is a brute force way, and there is the way interviewers actually want to see.

Brute force with negation

Since heapq always puts the smallest element at index 0, you can fake a max heap by making all your numbers negative. The largest positive number becomes the smallest negative number.

import heapq

nums = [3, 2, 1, 5, 6, 4]
max_heap = [-x for x in nums]
heapq.heapify(max_heap)

# The root is now -6
largest = -max_heap[0]

This works fine for small arrays. But if an interviewer asks you to get the top 100 values from a stream of a billion numbers, storing every single number in memory is extremely inefficient. You need a better strategy.

The efficient Min heap strategy

Instead of putting all the numbers into a max heap, put exactly K numbers into a min heap.

Think of it like keeping a running "Top 10" list. The root of a min heap (heap[0]) is always the smallest element. If your heap is exactly size K, the root is the smallest of your top K numbers.

As you stream through the rest of the data, if you see a new number that is bigger than your root, it belongs in the Top K. You kick the root out, and put the new number in.

First, you start by creating a heap with only the first K elements.

import heapq

def find_kth_largest(nums: list[int], k: int) -> int:
    # Start our list with the first K elements
    heap = nums[:k]
    heapq.heapify(heap)

Then you iterate through the remaining numbers. If a new number is larger than the root of our heap, it means the root is no longer in the Top K. You replace it.

    # Go through the rest of the numbers
    for i in range(k, len(nums)):
        if nums[i] > heap[0]:
            heapq.heapreplace(heap, nums[i])

Finally, the root of your heap will be the Kth largest element overall.

    return heap[0]

Why Interviewers Care

This exact pattern solves the massive streaming data problem perfectly.

Because you only ever store K elements at a time, your Space Complexity is O(K). It takes virtually zero memory.

Your Time Complexity is O(N log K). You look at every number once (N), and occasionally do a heap replacement operation that takes logarithmic time based on the small size of K.

So next time you are asked for the K largest items, do not reach for a max heap. Use a min heap, cap it at size K, and only let the big numbers in.

Integrating Local GenAI into Desktop Applications: Lessons from RexIDE

Tomer Ben David — Wed, 04 Feb 2026 00:00:00 +0000

How we navigated the engineering challenges of embedding local AI models and agentic CLIs directly into a native desktop environment.

RexIDE started as a personal frustration.

Modern IDEs are powerful, but they weren’t designed for a world where AI agents are active participants in your workflow. They assume short lived commands, stateless tools, and human only context switching. That model breaks down the moment you introduce long running AI agents, real terminals, and multi project execution.

This post walks through how RexIDE was designed, the tradeoffs behind its architecture, and why a local first, execution centric approach became the core principle.

Building Persistent Terminal State

The primary goal was simple:

Keep context alive, across projects, terminals, and AI agents, without forcing the developer to think about infrastructure.

That goal immediately shaped every technical decision that followed.

Technical Tradeoffs of Local Execution

One of the earliest decisions was whether AI execution should happen in the cloud or directly on the developer’s machine. Cloud models offer excellent quality, but they introduce friction through API keys and billing management, trust concerns around proprietary code, and a heavy dependency on latency and availability.

Local models remove those concerns entirely. They keep code on the machine, work offline, and feel instant when integrated correctly.

RexIDE was designed local first by default, with the option to layer in cloud models only when the user explicitly opts in. Privacy and control are the baseline, not premium features.

A Note on Codex and the Recent Shift

Recently, OpenAI launched the Codex desktop app, which meaningfully validates the direction RexIDE took early on: local execution with persistent context.

Codex today focuses on a single toolchain, the Codex ecosystem, and does a solid job at solving the local, long running AI workflow problem within that scope.

RexIDE takes a broader approach. Instead of committing to a single AI provider or tool, it was designed from the start to act as an orchestrator for multiple local AI CLIs across platforms, including Claude Code, Codex CLI, and OpenCode. All of these run locally on macOS, Windows, and Linux, side by side, inside the same execution centric environment.

This reflects how many developers already work today: using multiple AI tools side by side, depending on the task at hand. The environment should adapt to that reality rather than force consolidation.

Model Selection and Resource Constraints

Running AI models locally isn’t free: CPU, memory, and energy usage matter, especially on a machine you actively work on. RexIDE intentionally uses multiple layers of local AI execution. It utilizes external local CLIs such as Claude Code, Codex CLI, and similar tools for full reasoning and agent-driven workflows, while also employing embedded lightweight local models for smaller, fast tasks like snippet analysis, summarization, and structural understanding directly inside the app.

Instead of chasing the largest model possible, RexIDE follows a simple rule:

Use the smallest model that reliably meets the task’s requirements.

Lightweight embedded models handle frequent, low-latency tasks without context switching, while heavier reasoning is delegated to specialized local CLIs that already excel at those workflows.

Multiple model sizes were tested against real workflows including transcription, summarization, and code understanding while monitoring latency, sustained CPU usage, and memory pressure. The selected models stay well within acceptable resource bounds, ensuring they don’t interfere with compilers, editors, or other foreground tasks.

Native PTY Execution and State Persistence

Most IDEs optimize for editing, but RexIDE optimizes for execution. That means providing real terminals rather than simulated ones, maintaining long running processes that don’t reset when focus changes, and enabling AI agents that operate inside the same execution context as the developer.

This approach eliminates a huge amount of mental overhead. You don’t restart tasks, re-explain context, or reconstruct state — everything stays alive.

Engineering Stateless Backend Boundaries

RexIDE doesn’t require a backend to function, but it was designed with one in mind. If a backend were introduced, it would follow a few strict principles: stateless request handling, explicit separation between compute, user state, and storage, and strong session isolation to prevent data leakage.

The client would remain the source of truth for execution context, with the backend acting only as an optional accelerator — never a dependency.

Resource Management and Background Throttling

Performance isn’t something you optimize later, it is a core part of the user experience. RexIDE treats system resources with respect by ensuring heavy work runs off the main thread and AI workloads throttle when the app is backgrounded.

If the tool ever feels like it’s “in the way,” it has failed.

Reversible Architectural Decisions

Early design decisions are rarely perfect. RexIDE was built with reversibility in mind.

Short, time boxed prototypes were preferred over long debates. Decisions were explicitly labeled as reversible or irreversible, which made it easier to move fast without locking the project into bad paths. That mindset allowed rapid iteration without accumulating architectural debt.

The Result

RexIDE isn’t trying to be another editor with AI bolted on. It’s an execution environment where context persists, AI agents feel native, and the developer stays in control.

Everything else is a consequence of that choice.

If you’re building tools for developers today, the question isn’t whether to add AI — it’s where it lives, how much context it gets, and who ultimately controls it.

RexIDE represents one way to approach that problem.

AWS Lambda Pricing 2026 Guide

Tomer Ben David — Mon, 02 Feb 2026 13:16:55 +0000

AWS Lambda is the "serverless" gold standard for a service that lets you run code without managing any servers. You only pay for what you use, but if you don't understand the rules, your bill can grow surprisingly fast.

Here is everything you need to know about Lambda pricing in a clear, simple guide for 2026.

1. The Two Main Costs: Requests and Duration

AWS calculates your bill using two primary factors:

Requests: You are charged for the total number of times your functions start running.
Duration: You are charged for the time it takes your code to execute, rounded to the nearest 1 millisecond.

The Free Tier (The Good News)

Every month, AWS gives you 1 million requests and 400,000 GB-seconds of compute time for free. The best part? This free allowance never expires.

2. The "Cold Start" Cost Shift (New for 2025)

A "cold start" happens when Lambda has to set up a new environment to run your code. This used to be a performance problem; now it's a budget problem.

Important Update: As of August 2025, AWS now bills for the initialization (INIT) phase of a cold start. Before this change, the setup time was mostly free. Now, it’s a recurring budget item, especially for heavy runtimes like Java or C#.

3. Three Simple Ways to Save (Up to 34%)

Tip 1: Switch to ARM (Graviton2)

Most Lambda functions run on x86 processors by default. However, switching to ARM-based Graviton2 processors can offer up to 34% better price-performance and costs roughly 20% less per millisecond.

Tip 2: "Right-Size" Your Memory

When you give your function more memory (RAM), AWS automatically gives it more CPU power.

Too little memory: Your code runs so slowly that you end up paying more in duration charges.
Too much memory: You might give your code more CPU than it can actually use.
Pro Tip: Use tools like AWS Lambda Power Tuning to find the "sweet spot" where speed and cost intersect.

Tip 3: The "Lambda-Less" Approach

The cheapest Lambda is the one you don't run. Many AWS services—like API Gateway, AppSync, and EventBridge Pipes—can talk directly to databases (DynamoDB) or queues (SQS) without needing a Lambda function in the middle. This eliminates compute costs and reduces latency.

Final Pro-Tip: Don't Spend Money Waiting

For complex, multi-step workflows that need to "wait" for something to happen, don't use Lambda to manage the wait. Use AWS Step Functions instead. You don’t pay for the time Step Functions sits idle, whereas a Lambda function would bill you for every second it spends waiting.

Citations & Further Reading

The Almost Correct System

Tomer Ben David — Sun, 25 Jan 2026 00:00:00 +0000

In modern service and cloud architectures, the most painful production failures aren’t usually caused by "bad code" in the traditional sense.

They’re caused by good code making different assumptions.

This is the reality of distributed systems. It’s uncomfortable to hear, especially if you’re a careful engineer who writes tests, handles errors, and thinks about edge cases. But once you see this pattern, you’ll start noticing it everywhere, from microservice outages to distributed deadlocks and system design interview questions.

1. The Baseline: Why "Working Code" != A Working System

We naturally test the things we can control: the client, the API, the database. We run integration tests between them. If every individual component returns the correct output for a given input, we say the code is "correct."

In a simple, local program, this is the ground truth. If every function is correct, the program is correct. But in a distributed cloud architecture, this logic breaks down. You can have three "correct" services that, when combined, create a catastrophic failure.

The failure isn’t usually inside your code, it’s in the space between your services.

Each component is built with assumptions about how the rest of the system behaves. When those assumptions don’t match, the system becomes fragile, even if every line of code is technically perfect.

The assumption mismatch in practice

Let’s look at something boring on purpose: timeouts. Imagine this setup where every value looks reasonable on its own:

Client timeout: 2 seconds
Load balancer timeout: 5 seconds
Backend service timeout: 30 seconds
Database timeout: No limit

The Step-by-Step Failure

The client sends a request.
The backend is slow today (cold cache, lock contention, etc.).
After 2 seconds, the client gives up and retries.
The original request is still running in the backend (it has 28 seconds left).
Now the backend is doing the same work twice.
The database sees double load. Latency increases further.
More clients retry. The system spirals.

No single component broke. The database didn't crash; the service didn't leak memory. The failure emerged from how their assumptions interacted.

Bridging the gap with explicit contracts

Every boundary in a system has a contract, whether you wrote it down or not. We often rely on implicit contracts:

"This request finishes quickly"
"Retries are safe"
"This operation runs once"

The problem is that when assumptions are implicit, different parts of the system invent their own version of reality. That’s where "almost correct" systems are born.

If a client times out at 2 seconds, the backend must know its work is no longer wanted. If a client retries, the operation must be idempotent.

How to reason about boundaries

To move from "Junior" to "Senior" systems thinking, you have to shift your primary question:

Junior-level thinking: "Is my code correct?"
Senior-level thinking: "What assumptions does my code make, and who depends on them?"

The longer a system lives, the more assumption drift it accumulates. To combat this, you need to implement alignment strategies:

Align Timeouts: Upstream timeouts should generally be shorter than downstream ones only if you have aggressive retries; actually, a better pattern is Deadline Propagation, where the remaining time budget is passed along the request chain.
Make Operations Idempotent: If a caller assumes they can retry safely, you must assume they will retry multiple times.
Use Backpressure: If you assume the system can handle X load, you must have a way to say "no" when X is exceeded, rather than slowing down for everyone.

Why "almost correct" is worse than "broken"

Failing loud is a feature. When a system crashes or returns a 500, you know exactly when and where it broke. Experienced engineers aim for this "fail fast" behavior because it surfaces problems immediately.

The danger comes from the impulse often seen in junior developers to "handle" every error by hiding it. This leads to the most dangerous state: the almost-correct system.

Almost correct systems are quieter and more dangerous:

They pass unit tests.
They survive staging.
They fail only under specific load.
They fail only when timing is unlucky.

These failures are hard to reproduce because no single line of code is wrong. This is why postmortems often sound like: "Everything behaved as designed... just not together."

A systems thinking checklist

When designing or reviewing a system, don’t start with implementation details. Start with failure questions to force assumptions into the open:

[ ] Retries: What retries this, and what is the retry budget?
[ ] Timeouts: Who times out first? Does the work stop when they do?
[ ] Idempotency: What happens if this exact request runs twice?
[ ] Partial Failure: What happens if the DB update succeeds but the cache update fails?
[ ] State: What state survives a crash, and what assumption does the next run make about that state?

Closing Thought

Great software isn’t built by eliminating bugs. It’s built by eliminating surprises. These surprises don’t come from bad code; they come from assumptions that were never made explicit.

Citations & Further Reading

Originally published at https://rex.mindmeld360.com

Java Memory Model Deep Dive: Visibility, Reordering, and the Truth About Volatile

Tomer Ben David — Wed, 21 Jan 2026 00:00:00 +0000

In a single-threaded Java program, you are protected by a beautiful lie called as-if-serial semantics. If you write int x = 1; int y = 2;, the JVM and CPU can reorder those lines however they want to improve performance, but they promise that the result will be exactly as if they ran in order. Inside that single thread, the reordering is invisible.

As soon as you introduce a second thread, the lie falls apart. That second thread doesn't see the "as-if-serial" promise; it sees the raw memory as it updates. Code that looks perfectly logical can suddenly fail in ways that seem impossible. This is where the Java Memory Model (JMM) comes in—it is the official "contract" that defines exactly when and how threads are allowed to see each other's changes.

1. The Core Problem: Performance over Predictability

Most developers assume the JVM executes code exactly line-by-line as written. In reality, the JVM and your CPU are obsessed with speed. To run faster, they perform optimizations that create two main issues: Reordering and Visibility.

Reordering: The "Out of Order" Execution

The compiler or the CPU might decide to swap two instructions if it thinks the final result will be the same.

// What you wrote:
int a = 1;
boolean flag = true;

// What the CPU might actually execute:
boolean flag = true;
int a = 1;

For a single thread, this swap doesn't matter. But if another thread is waiting for flag to be true so it can read a, it might see flag == true before a has actually been set to 1.

Visibility: The Cache Problem

Modern CPUs have their own local caches (L1, L2, L3). When a thread updates a variable, it might only save that change in its local CPU cache to save time. Other threads, running on different CPU cores, will continue to read the old value from their own caches or main memory. The change is "invisible" to them.

2. The Solution: "Happens-Before" (HB)

The JMM doesn't promise that everything will always be in order. Instead, it provides a set of rules called the Happens-Before relationship.

Think of Happens-Before as a "visibility bridge." If Action A happens-before Action B, then any change made by Action A is guaranteed to be visible to the thread performing Action B.

The Most Important Rules:

Program Order: In a single thread, every action happens-before any action that comes later in the code.
Volatile Variable Rule: A write to a volatile field happens-before every subsequent read of that same field. (This is the "signal" we use to bridge threads).
Monitor Lock Rule: Releasing a lock (synchronized) happens-before any subsequent acquisition of that same lock.
Thread Life Cycle: Calling thread.start() happens-before any action in that thread. All actions in a thread happen-before a successful thread.join() on that thread.

3. The `volatile` Modifier: A Modern Guide

A common mistake is thinking volatile is just for "disabling caches." It's more powerful than that.

When you write to a volatile variable, the JVM ensures two things:

Visibility: The write is immediately flushed to main memory, and any subsequent read will pull the latest value.
Ordering (The Barrier): The JVM prevents instructions from being reordered around the volatile read/write. It acts as a "memory barrier."

What `volatile` does NOT do: Atomicity

This is the biggest landmine in Java. volatile does not make compound operations atomic.

public volatile int count = 0;

public void increment() {
    count++; // NOT THREAD-SAFE
}

count++ is actually three steps: read, add 1, write. If two threads do this at the same time, they might both read the same value, add 1, and write the same result back, losing one increment. For this, you need AtomicInteger or synchronized.

4. Unsafe Publication: Why "null" isn't always null

One of the strangest bugs in Java is when a thread sees an object that is "half-initialized."

// Thread A
shared = new Helper(42);

// Thread B
if (shared != null) {
    System.out.println(shared.x); // Could print 0 instead of 42!
}

Because of reordering, the CPU might assign the memory address of the new Helper object to the shared variable before the constructor has finished setting x = 42.

How to fix this (Safe Publication):

Make the shared field volatile.
Initialize it inside a synchronized block.
Make the fields inside the object final. The JMM gives special visibility guarantees to final fields once the constructor finishes.

5. Double-Checked Locking (DCL)

The classic way to create a lazy singleton safely is the Double-Checked Locking pattern. It relies heavily on volatile.

private volatile Resource resource;

public Resource getResource() {
    Resource result = resource;
    if (result == null) { // First check (no locking)
        synchronized (this) {
            result = resource;
            if (result == null) { // Second check (with locking)
                resource = result = new Resource();
            }
        }
    }
    return result;
}

Note: We use the local variable result to reduce the number of times we have to read the volatile field, which is a small performance optimization.

6. Beyond Volatile: VarHandles (Java 9+)

In modern Java, if you need even more control than volatile provides, you can use the VarHandle API. It allows you to choose exactly how much "strictness" you want:

Opaque: Ensures the value isn't cached, but allows reordering.
Acquire/Release: A lighter version of volatile that only enforces ordering in one direction.
Volatile: The full-strength version we discussed.

Practical Checklist for Concurrent Code

Is the variable shared? If yes, it must be protected by volatile, Atomic classes, or a lock.
Are you doing more than a simple write? If you are reading-then-writing (like count++), volatile is not enough. Use AtomicInteger.
Is your object fully built? Never let the this reference "escape" from a constructor (e.g., by passing it to another thread) before the constructor is finished.
Can you use final? Always prefer final fields. They are the simplest way to ensure thread-safety for data that doesn't change.

Citations & Further Reading

Originally published at https://rex.mindmeld360.com.

Deduce, Don't Store

Tomer Ben David — Tue, 23 Dec 2025 11:46:34 +0000

Introduction

One of the most common sources of bugs in complex applications is stale state. When you store a value that depends on another piece of data, you create a requirement to keep those two values in sync. If you miss a single update path, your application enters an invalid state that can be incredibly difficult to debug.

We want to be strict about state management: Deduce state from the source of truth rather than storing it. While we implement this using Swift computed properties, it is important to note that this is not a language specific trick. It is a fundamental engineering practice used in reliable systems across all platforms to eliminate data synchronization errors.

The Danger of Cached Values

In traditional application development, it is tempting to store boolean flags like isLoaded or isAuthorized as mutable properties. However, these flags are rarely the actual source of truth. The true state lives in your data collection or your active session token.

By storing these flags separately, you are essentially caching a view of the reality. If the data is cleared or the token expires, your stored flag becomes a lie. This leads to edge cases where the UI shows one thing while the underlying system is doing another.

Computing State from the Source of Truth

To ensure that the application always reflects the current reality, we prefer to compute state directly from the service that owns the data. Instead of updating a status variable whenever an action occurs, we query the state in real time.

This approach ensures that there is only one place where an update can happen. Every other component simply observes or deduces its logic from that single point.

/// Deducing state from a dependency
final class InventoryManager {
    private let storage: StorageProvider

    /// The true source of truth is the actual data in storage
    var needsReplenishment: Bool {
        // We compute this every time it is needed
        return storage.itemCount < threshold
    }

    private let threshold = 10

    init(storage: StorageProvider) {
        self.storage = storage
    }
}

By making needsReplenishment a computed property, we eliminate the possibility of it ever being out of sync with the storage. There is no setNeedsReplenishment(true) method to call, which removes an entire category of logic errors.

Isolating Side Effects in the Deduction Loop

Deducing state is not just about simple booleans. It is a philosophy that extends to complex UI transitions and background operations. When you need to decide whether to show a specific view or enable an action, you should deduce that decision from the current environment variables.

In our core architecture, we use services that provide these environmental snapshots. For example, if we need to know if a permission is granted, we do not check a stored isPermitted flag. Instead, we query a provider that evaluates the current system settings in real time. This ensures that the app reacts immediately to changes without needing complex synchronization code.

Benefits of Reduced Complexity

When you stop storing redundant state, your code becomes significantly more predictable. Your models become simpler because they no longer need to manage the lifecycle of cached values. Your tests also become more robust because you only need to mock the primary source of truth to verify hundreds of different deduced outcomes.

By removing the opportunity for state to become stale, we create a faster and more reliable experience for our users.