Forem: Ma Uttaram

Part 3

Ma Uttaram — Fri, 17 Apr 2026 21:11:04 +0000

Facebook News Feed

Tinder

LeetCode

yelp

Rate Limiter

Part 2

Ma Uttaram — Fri, 17 Apr 2026 19:04:45 +0000

Uber

Robinhood
-- Key is the Number of requests from various IPs is not allowed so limit src Ips.

Google Docs

Distributed Cache

Youtube

Parts 1

Ma Uttaram — Thu, 16 Apr 2026 19:49:43 +0000

News Feed Service

Drop Box

Tiny URL

Local Delivery Service

Ticket Master

DAG vs Langraph Nodes

Ma Uttaram — Sun, 29 Mar 2026 00:39:21 +0000

When we have DAG that represents our tasks and its dependencies do we still need Langraph nodes?

LangGraph nodes are most valuable when your flow has cycles — loops, retries, HITL resume, conditional branching back to earlier steps. A pure DAG has none of that, so LangGraph's core value proposition doesn't apply.

So are they mutually exclusive? No! They can complement each other. See the beautiful project to understand why.

LangGraph nodes for the orchestration layer: intent_parser → entity_extractor → plan_builder → hitl_plan → cot_builder → hitl_confirm. This is the stateful, cyclic part.

Your own DAG executor for task execution: takes the confirmed TaskGraph, runs a topological sort, fans out independent tasks with asyncio, collects results. Twenty to thirty lines of plain Python. No framework.

| Send API for parallel dispatch | LangGraph-native fan-out; no manual thread/asyncio management; results merged automatically |

The DAG (networkx) and LangGraph Send API solve different problems and are complementary — not alternatives. See the 'Why both networkx DAG and LangGraph Send API?' section added to the Overview. Short answer: networkx tells us WHAT to run (which steps are ready, via topological ordering and predecessor checks); LangGraph Send API handles HOW to run them concurrently (fan-out to parallel node invocations, automatic state merge on fan-in). You could replace Send with asyncio.gather inside a single node, but you'd lose per-step checkpointing (partial progress survives failures), automatic state merging, and consistency with the rest of the LangGraph pipeline.

Your mental model was almost right — let me clarify the full split:\n\n| Concern | Tool | Why |\n|---|---|---|\n| Dependency graph structure | networkx DAG | Planner builds it, CoT validator walks it, executor queries it for readiness |\n| Parallel task execution | LangGraph Send API | Fan-out ready steps as concurrent node invocations; LangGraph auto-merges results |\n| HITL + session resume | LangGraph interrupt() + PostgresSaver | Only LangGraph provides checkpointed pause/resume semantics |\n\nThe DAG is used across all pipeline stages — not just the executor — as a data structure for graph operations (cycle detection, topological sort, predecessor queries). LangGraph is the execution engine throughout. The executor uses both: DAG to decide which steps are ready, Send API to run them in parallel. Added a dedicated section in the executor design doc explaining this.

The DAG (networkx) is a data structure for representing and querying step dependencies — it lives in dag.py and is used by three different pipeline stages (planner for cycle detection, CoT validator for topological walk, executor for readiness checks). LangGraph graph nodes (validate_cot, execute_step, etc.) are the execution units in the agent runtime. They use the DAG as a utility library. There's no conflict — the DAG tells the LangGraph nodes what order/parallelism is required; LangGraph handles actually running them, checkpointing state, and managing HITL interrupts.

Langraph vs Langchain

Ma Uttaram — Sat, 28 Mar 2026 22:32:39 +0000

As I was working on HITL, I was checking to see if we can start with existing Agentic Frameworks like Langraph and Langchain. The below was very interesting

Evaluation Techniques

Ma Uttaram — Sat, 28 Mar 2026 20:07:01 +0000

There are six main evaluation techniques, falling into two broad families: those that compare against a known answer, and those that use judgment. Here's a visual overview, then the explanation of each.Here's each technique explained:

Exact match — the simplest form. You know the correct answer, and you check whether the output equals it exactly. Works well for structured tasks: intent classification ("is this a booking request?"), entity extraction where the expected output is a fixed JSON, or tool selection ("should the agent call the calendar API or the email API here?"). Brittle for open-ended text because two correct answers can be worded differently.

Schema / constraint validation — instead of checking exact values, you check the shape of the output. Does entity extraction return a valid Task schema with all required fields? Did the plan builder produce a properly ordered list? This is what Pydantic and Zod do, and it's directly relevant to BuddingBuilder's FR #7. It catches malformed outputs even when the content is hard to verify.

Code execution / unit test — the gold standard for any agent that produces code or structured plans. You run the output and check whether tests pass. For BuddingBuilder this applies to any task whose result is deterministically verifiable — a calculation, a formatted document, a database query result.

Reference-based LLM judge — you have a golden answer, and you ask a judge model to compare the agent's output against it and score the match. Returns a score and a reason. More flexible than exact match because it can handle paraphrasing, but requires you to maintain a library of golden examples, which takes effort to build and keep current.

Rubric-based LLM judge — no golden answer needed. You give the judge a scoring rubric ("rate this response 1–5 on correctness, task completion, and safety") and it evaluates the output on its own. This is the most practical technique for staging, because you can write rubrics faster than you can curate golden answers. The key is writing rubrics that are specific enough that the judge can't wriggle around them.

Pairwise preference — the judge sees two outputs side by side and picks the better one. You're not asking "is this good?" but "which is better — the old prompt or the new one?" This is the right technique for promotion gates: before moving from dev to staging, run pairwise eval between the new version and the current prod version. If the new version wins consistently, promote. This is also how RLHF preference data is collected.

Human eval — a human reads and rates the output. Highest signal, but too slow and expensive to run on everything. Its real job is to calibrate your automated judges — you periodically sample flagged traces, have a human rate them, and check whether your judge model's scores agree. If they don't, your rubric needs refining.

Online monitoring — the only technique running continuously in prod. The guard model scores inputs before the agent acts; the output validator scores responses after. Neither produces a detailed critique — they produce a fast pass/fail signal with enough metadata to route flagged interactions to the human review queue. This is what closes BuddingBuilder's prod → dev feedback loop.

claude.ai

Reasoning Techniques

Ma Uttaram — Sat, 28 Mar 2026 18:49:11 +0000

Reasoning is the process of working through a problem in steps rather than jumping straight to an answer. For an LLM, it means producing intermediate thoughts — a chain of logic — before committing to a response.
Think of it this way: when you ask someone a hard question, a thoughtful person doesn't blurt the first thing that comes to mind. They pause, break the problem down, consider what they know, check their logic, and then answer. Reasoning gives LLMs that same pause.