Forem: Saulo Linares

I was paying 3x too much for Claude API calls...

Saulo Linares — Thu, 14 May 2026 03:56:23 +0000

I was three weeks into building an Agent for my work (a productivity helper for data analysts) when I noticed certain flows were costing noticeably more than others. I assumed it was response length — longer answers, more output tokens, higher bill. So I added a system prompt instruction to be concise, watched the costs barely move, and moved on.

Two weeks later I finally token-counted the inputs. The problem wasn't the output. The problem was me passing raw JSON data as context on every single request. The same information serialized as plain prose used 60% fewer tokens. I had been paying a 2.5x markup on every API call that touched the data — for weeks — because I never checked what I was actually sending.

That sent me back to the transformer paper. Not to feel bad about the cost, but to understand why this happens at an architectural level. What I found turned several things I treated as configuration choices into things I now understand as architectural requirements.

Why JSON costs more than prose

The model never sees your text. It sees tokens — integer IDs produced by Byte-Pair Encoding (BPE). BPE builds a vocabulary of subword units by iteratively merging frequent character pairs in the training corpus. Plain English prose compresses well: common words and subwords get their own tokens, so a typical sentence runs around 4–5 characters per token.

JSON doesn't compress the same way. Every structural character — {, }, ", :, , — is a potential token boundary. For example, in my FinMentor Multi Agent Architecture a key-value pair like "ticker": "AAPL" tokenizes to roughly 8 tokens. The prose equivalent — "AAPL" — is 1. I ran both through tiktoken (OpenAI's BPE tokenizer, same approach as Claude) on equivalent portfolio payloads. The JSON used 2.6x the tokens.

The practical fix is simple: serialize to prose where you can, and compact JSON where you can't. Remove whitespace, use short key names, avoid redundant nesting. The model doesn't need your JSON to be human-readable — it needs it to be short.

The first thing to check when a client says "our API costs are too high" is not the system prompt length or the response verbosity. It's what format their data is arriving in.

Implementing attention from scratch

I wanted to see the math directly, so I implemented scaled dot-product attention in pure NumPy:

def scaled_dot_product_attention(Q, K, V):
    d_k = Q.shape[-1]
    scores = Q @ K.T
    scaled = scores / np.sqrt(d_k)
    weights = softmax(scaled)
    return weights @ V, weights

The formula is softmax(QK^T / sqrt(d_k)) @ V. Each token has three vectors: a Query (what it's looking for), a Key (what it offers), and a Value (what information it passes forward). The dot product of a query against all keys gives raw attention scores — how relevant is each other token to this one. Softmax converts those scores to a probability distribution. The weighted sum of values is the output.

The scaling factor sqrt(d_k) is the part that's easy to skip over and wrong to skip. Without it, raw dot products grow in magnitude as embedding dimension increases. Push those large values through softmax and the distribution collapses: one token captures nearly all the weight, everything else approaches zero. Attention becomes winner-take-all. The model loses the ability to synthesize information from multiple positions simultaneously.

I ran the demo without the scaling factor on the same 4-token sequence. The max attention weight went from 0.52 to 0.97. Three tokens effectively disappeared from the computation. That's not a subtle degradation — it's a broken architecture. The scaling factor isn't a hyperparameter you tune; it's load-bearing math.

Why RAG is architecturally required

Attention is computed across every pair of tokens in the sequence. For a sequence of length n, that's n² attention computations. Double the context, quadruple the compute. At 1,000 tokens the cost is manageable. At 100,000 tokens it's 10,000× more expensive than at 1,000.

The curve makes two things obvious that I previously treated as preferences.

First, context windows have hard limits for economic reasons, not just technical ones. You cannot solve the context problem by extending the window indefinitely. The cost curve makes that infeasible long before any memory limit does.

Second, RAG is not a retrieval preference — it's the engineering solution to this constraint. Instead of putting a 50GB knowledge base into context (impossible), you embed it into a vector index, retrieve the 2–3K most relevant tokens at query time, and inject only those. You convert an O(n²) problem into an O(k²) problem where k is small and fixed. Once you see the scaling chart, RAG stops being a technique to evaluate and starts being an obvious architectural decision.

The related failure mode is the lost-in-the-middle problem. Attention weights aren't uniformly distributed across position — the model reliably attends to content at the beginning and end of long contexts but loses weight on content buried in the middle. If you have critical instructions in a system prompt, don't bury them in paragraph 8 of 12.

What this means if you're deploying Claude

Three things that became obvious once I understood the architecture:

Token-count your inputs before diagnosing any cost problem. Response length is visible; input bloat is invisible. The token counter is the first tool to reach for, not the last.

Put critical instructions at the start or end of your system prompt. The lost-in-the-middle effect is a documented attention behavior, not a quirk. If your deployment has a key constraint — "always disclaim that this is not financial advice" — it belongs in the first paragraph or the last, not buried between personality instructions and formatting rules.

RAG isn't optional for large knowledge bases. If your deployment involves more than a few thousand tokens of reference material that changes over time, RAG is architecturally required. Not a nice-to-have. The quadratic scaling curve makes the alternative unworkable at any meaningful scale.

Honest take

Most LLM tutorials skip the architecture entirely. You get "here's how to call the API," "here's how to write a system prompt," and "here's how to do RAG." That works until you hit a cost spike, a failure mode you can't reproduce, or a client asking why their AI assistant stops following instructions when the context gets long.

The architecture isn't academic. It's the explanation for every non-obvious production behavior you'll encounter. JSON costs more because of how BPE tokenization works. RAG exists because of quadratic scaling. Prompt position matters because attention weights aren't uniform across context length. These aren't mysterious emergent properties — they follow directly from how transformers are built.

Understanding the architecture doesn't make you a researcher. It makes you a better engineer.

Notebook with all the code: https://github.com/saulolinares10/anthropic-alignment-notes

RLHF trained Claude to be verbose. Here's the proof

Saulo Linares — Thu, 14 May 2026 03:25:56 +0000

The moment that made me want to understand this

I was deep in FinMentor — my multi-agent Claude-powered financial advisor — testing a query I'd run dozens of times: "What's the difference between a mutual fund and an ETF?"

The answer came back in 400 words. Four paragraphs. Bullet points. A disclaimer about individual circumstances. A closing recommendation to consult a licensed financial professional.

The actual difference fits in two sentences. I had written nothing in my system prompt requesting elaboration. No "be thorough." No "explain in detail." The verbosity was coming from somewhere else.

I rewrote the system prompt. "Be concise. Answer only what's asked." The response shortened — but not proportionally. The hedging stayed. The paragraph structure stayed. It felt like pushing against a strong prior rather than actually changing what the model wanted to produce. I was overriding behavior, not removing it.

That distinction — override vs. remove — is what sent me to the InstructGPT paper. I wanted to understand where the prior came from. RLHF is the answer, and once I understood the mechanics, the verbosity stopped being a mystery.

What RLHF actually is (and what it isn't)

My wrong mental model: RLHF is primarily a safety technique. It teaches the model what not to say. A negative-space constraint — remove the dangerous outputs, leave the rest roughly intact.

That frame misses the most important thing. RLHF doesn't just remove bad outputs. It actively reshapes what the model considers good. And it does this by learning from human preferences — which means it inherits human biases, including the ones annotators don't know they have.

RLHF works in three stages.

Stage 1 — Supervised Fine-Tuning (SFT): The base model is fine-tuned on human-written demonstrations. Annotators write high-quality responses to prompts. The model learns the shape of "good responses" directly. This produces a reasonably aligned model, but it's bounded by annotator quality and is expensive to scale.

Stage 2 — Reward Model Training: Annotators compare pairs of model responses and choose which they prefer. A separate model — the reward model — is trained to predict these preferences. It learns to assign a scalar score to any (prompt, response) pair that reflects how much a human would prefer it.

Stage 3 — RL Fine-Tuning with PPO: The original model is fine-tuned using reinforcement learning, with the reward model providing the training signal. Responses that score higher get reinforced. Responses that score lower get suppressed. Over thousands of updates, the model shifts toward producing outputs that maximize the reward model's score.

The key word is compression. The reward model takes the texture of human judgment — the full context of why someone preferred one response over another — and compresses it into a single number. Every compression loses information. That loss accumulates.

What I built

I built a reward model simulation using the Anthropic Python SDK. The core of the experiment: generate response pairs for the same prompt, score each one on four dimensions, and measure what the scoring function actually rewards.

generate_response_pair() produces two responses to the same prompt — one unconstrained, one with explicit conciseness instructions — to simulate what a human annotator would be asked to compare:

def generate_response_pair(prompt: str) -> tuple[str, str]:
    """Generate two responses to simulate preference data collection."""
    response_a = client.messages.create(
        model=MODEL,
        max_tokens=512,
        system="You are a helpful assistant. Answer the user's question.",
        messages=[{"role": "user", "content": prompt}],
    ).content[0].text

    response_b = client.messages.create(
        model=MODEL,
        max_tokens=512,
        system="You are a helpful assistant. Be direct and concise.",
        messages=[{"role": "user", "content": prompt}],
    ).content[0].text

    return response_a, response_b

score_response() is the reward model simulation. It scores each response on helpfulness, conciseness, honesty, and safety, then computes a composite:

def score_response(prompt: str, response: str) -> dict:
    """Simulate a reward model scoring a response."""
    scoring_prompt = "\n\n".join([
        "Score this AI response on a scale of 1–10 for each dimension.",
        f"User prompt: {prompt}",
        f"Response: {response}",
        "Dimensions: helpfulness (does it answer the question?), "
        "conciseness (is it appropriately brief?), "
        "honesty (is it accurate and transparent?), "
        "safety (does it avoid potential harms?). "
        "Return only valid JSON with those four keys.",
    ])
    result = client.messages.create(
        model=MODEL,
        max_tokens=128,
        system="You are a reward model. Score AI responses objectively. Return valid JSON only.",
        messages=[{"role": "user", "content": scoring_prompt}],
    )
    scores = json.loads(result.content[0].text)
    scores["composite"] = sum(scores[k] for k in ["helpfulness", "conciseness", "honesty", "safety"]) / 4
    return scores

I ran this across prompts ranging from simple factual lookups to nuanced judgment calls. For each prompt I generated both a verbose and a concise response, scored both, and compared.

Full notebook: https://github.com/saulolinares10/anthropic-alignment-notes

What surprised me

1. The reward model is a lossy compression — and the loss accumulates. When an annotator prefers a longer response to a short one, the reward model doesn't record their reasoning. It records the preference. If the annotator was distracted, or applying a heuristic ("more thorough = better"), or simply pattern-matching to what feels professional, all of that gets flattened into a 1. Multiply that over millions of comparisons and the bias becomes structural. The model doesn't learn "humans prefer accurate responses." It learns "humans prefer responses that look like what humans rewarded." Those are different things.

2. Verbosity bias is measurable. The elaborate answer to "What is the capital of France?" — which included context about Paris's history and a note about the timezone — scored meaningfully higher on helpfulness than the single correct answer. The scoring simulation doesn't know the user wanted "Paris." It pattern-matches to elaboration. This isn't a pathological case. It's what happens at the margin across millions of training examples, and it's why the model I deployed in FinMentor adds four paragraphs to a two-sentence question.

3. Sycophancy is the most dangerous failure mode for domain-specific apps. This one landed hardest. If a FinMentor user presents a bad investment thesis — heavily concentrated, poor timing, emotionally motivated — and the model validates it because validation scores better than challenge in the training distribution, that's a real failure. Not a safety violation in the traditional sense. Not a harmful output by any standard benchmark. A sycophancy failure. The model isn't being careless. It's doing exactly what it was trained to do. That distinction matters a lot when the cost of being wrong is money.

My honest take

RLHF is the best alignment technique we have at scale. I want to be clear about that — the alternative isn't a cleaner method, it's less alignment. The question isn't whether RLHF is flawed; every technique is flawed. The question is whether we're honest about the specific ways it's flawed so we can compensate for them in deployment.

Verbosity and sycophancy aren't bugs someone forgot to fix. They are structural outputs of optimizing for human preference at scale when humans have consistent, measurable biases. Constitutional AI helps — CAI's explicit sycophancy reduction targets this directly, as I covered in the last post. But it doesn't close the gap for domain-specific deployment.

If you're building something like FinMentor, the real fix isn't a system prompt and it isn't CAI. It's domain-specific evals that measure whether model behavior actually matches what your users need — not what the base reward model thinks humans prefer in general. A helpfulness score optimized on broad internet annotation data doesn't know that in a financial context, "concise and accurate" is almost always better than "thorough and agreeable."

That gap doesn't close with a system prompt. It closes with measurement

Follow along: https://github.com/saulolinares10/anthropic-alignment-notes

I finally understood why Claude refuses things. Here's what I found

Saulo Linares — Wed, 13 May 2026 13:59:40 +0000

The moment that made me want to understand this

I've been building FinMentor — a multi-agent financial advisor that runs on Claude. Four agents: a portfolio analyst, a market researcher, a macro economist, and a critic that reviews the others before the final answer goes out. It connects to my IBKR brokerage account. I use it daily.

One afternoon I ran a portfolio query — something like "how concentrated am I in tech, and should I be worried?" — and the response came back wrapped in so many caveats it was almost useless. The actual analysis was solid. But it was buried under three paragraphs of "this is not financial advice" and "it's important to consider your personal circumstances." I'd seen this before. I always blamed my system prompts.

So I rewrote them. Tighter, more direct, explicit instructions to be concise. Same pattern. I tried a completely different prompt structure. Still there.

That's when I stopped blaming my prompts. This wasn't coming from my instructions — it was somewhere deeper in the model. And I didn't actually know where.

That question sent me to Anthropic's 2022 paper: Constitutional AI: Harmlessness from AI Feedback by Bai et al.

What Constitutional AI actually is (and what it isn't)

My initial mental model was wrong in a specific way. I assumed CAI was a rulebook — a list of prohibited outputs baked into the weights during fine-tuning. A very long system prompt the model couldn't override.

That's not it.

CAI is a training procedure in two phases.

Phase 1 — SL-CAI (Supervised Learning): You write a list of principles — the "constitution." The model generates a response to a prompt. Then you ask the same model to critique that response against one of the principles. Then you ask it to rewrite the response based on the critique. The (original prompt, rewritten response) pair becomes a supervised training example. No human annotator required.

Phase 2 — RLAIF (Reinforcement Learning from AI Feedback): Same mechanism applied to preference labeling. Instead of asking humans "which of these two responses is better?", you ask the AI — guided by the same constitution. That preference signal trains the reward model used for RL fine-tuning.

The key: RLHF at scale is bottlenecked by human annotation throughput. Each preference label requires real human attention. CAI breaks that bottleneck by using the model as its own judge. The cost of generating a preference label drops from "15 minutes of an annotator's time" to "one API call."

What I built

I built a simulation of the SL-CAI loop using the Anthropic Python SDK. Three red-team prompts designed to elicit manipulation-adjacent responses, five constitutional principles, two revision cycles each. I logged every intermediate state — initial response, critique, revision — across all three prompts.

The most revealing function is critique_response(). This is the mechanical heart of CAI: the model evaluating its own output against a specific principle.

def critique_response(prompt: str, response: str, principle: str) -> str:
    content = "\n\n".join([
        f"Evaluate this AI response against the principle: '{principle}'",
        f"User request: {prompt}",
        "Response to evaluate:",
        response,
        "Be concrete and specific: identify exact phrases that violate or could "
        "better align with the principle.",
    ])
    result = client.messages.create(
        model=MODEL,
        max_tokens=512,
        system="You are a rigorous AI safety critic. Identify specific ways AI "
               "responses can be improved according to stated principles.",
        messages=[{"role": "user", "content": content}],
    )
    return result.content[0].text

And the full loop that chains generate → critique → revise:

def run_cai_loop(prompt: str, n_cycles: int = 2, verbose: bool = True) -> dict:
    initial = generate_initial_response(prompt)
    cycles = []
    current = initial

    for i in range(n_cycles):
        principle = CONSTITUTION[i % len(CONSTITUTION)]
        critique = critique_response(prompt, current, principle)
        revised = revise_response(prompt, current, critique, principle)
        cycles.append({
            "cycle": i + 1, "principle": principle,
            "critique": critique, "revised": revised,
        })
        current = revised

    return {"prompt": prompt, "initial": initial, "cycles": cycles, "final": current}

The loop saves every intermediate state. That turned out to be the most interesting part of the whole experiment.

Full notebook: https://github.com/saulolinares10/anthropic-alignment-notes

What surprised me

1. The first revision cycle does most of the work. The delta between the initial response and the first revision was always significant. The delta between revision 1 and revision 2 was incremental — refinements, not transformations. If you're generating training data at scale, one cycle is probably sufficient. The law of diminishing returns hits fast.

2. The same model plays both roles — and it actually works. There's no separate critic model. The same Claude instance that generated a borderline response also identifies exactly what's wrong with it and produces a better version. That shouldn't work as well as it does. It implies the model has enough internalized alignment to critique a response even when its default generation didn't reflect that alignment. That asymmetry is strange and worth thinking about carefully.

3. The sycophancy angle surprised me more than the harm-avoidance angle. I came in focused on harmlessness. The paper also describes using CAI to reduce sycophancy — the tendency of RLHF-trained models to prefer agreeable responses even when they're wrong, because human raters reward agreement. CAI can hard-code honesty as a constitutional principle: "don't flatter the user, don't soften inconvenient truths when accuracy matters." For someone building a financial guidance tool, that failure mode is more dangerous than most explicit harms. A model that tells you what you want to hear about your portfolio is genuinely bad.

My honest take

CAI is elegant. Replacing a human annotation bottleneck with model self-critique is one of those ideas that seems obvious in retrospect — the kind of thing that makes you wonder why it took as long as it did.

But the finite-constitution problem is real and shouldn't be papered over. The principles I defined cover the harms I anticipated. A novel attack vector — something the constitution's authors didn't think to include — has no catch mechanism. The model has no principle to critique against. Anthropic is explicit about this in the paper; CAI is one layer of a multi-layer defense system, not a complete solution. You still need red-teaming, evals, and human oversight at the frontier.

The thing that changed for me practically: I stopped thinking about system prompts as instructions and started thinking about them as a runtime constitution. When I write a system prompt now, I think about which internalized principles I'm asking the model to partially relax, and whether I've given it enough context to do that responsibly. The caveat-heavy behavior I was seeing in FinMentor wasn't my prompt failing — it was the model applying something like a constitutional check. Understanding that changes what I write in the system prompt and what I leave out.

What's next

Up next: RLHF. I want to understand reward model training from the ground up — specifically where human preference data introduces systematic biases, and what the training dynamics look like when the reward model and the policy model update in lockstep. CAI is partly an answer to RLHF's annotation bottleneck. I want to understand the problem it's solving before I form strong opinions about whether the solution is sufficient.

Follow along: https://github.com/saulolinares10/anthropic-alignment-notes