Forem: AwxGlobal

Preventing CrewAI Budget Overruns: Hard Limits Per Agent Role

AwxGlobal — Mon, 04 May 2026 07:00:43 +0000

Originally published at awx-shredder.fly.dev/blog

Preventing CrewAI Budget Overruns: Hard Limits Per Agent Role

A multi-agent CrewAI workflow spun up in production last month and burned through $340 in API costs before anyone noticed. The culprit? A research agent stuck in a loop, making hundreds of GPT-4 calls to refine a single report. The agent eventually completed its task, but the invoice was brutal.

CrewAI's built-in max_rpm and max_iter parameters help, but they're blunt instruments. RPM limits don't account for token usage variance—a single long-context call can cost 50x more than a short one. Iteration limits stop runaway loops but won't catch an agent that makes expensive calls within reasonable iteration counts.

What you actually need is hard budget enforcement at the agent level, ideally denominated in dollars rather than tokens or requests.

Why Agent-Level Budgets Matter

In a typical CrewAI setup, different agents have wildly different cost profiles:

Research agents make many calls with large contexts (expensive)
Routing agents make quick classification calls (cheap)
Writing agents generate long-form content (moderate, predictable)

A flat budget across all agents means your routing agent's allocation goes unused while your research agent blows past acceptable spend. You need granular control that maps to how each agent actually behaves in production.

The Naive Approach: Token Counting

Your first instinct might be to wrap the LLM client and count tokens:

from crewai import Agent, LLM
from collections import defaultdict
import tiktoken

class BudgetEnforcedLLM:
    def __init__(self, model_name, budget_dollars):
        self.llm = LLM(model=model_name)
        self.model_name = model_name
        self.budget_dollars = budget_dollars
        self.spent_dollars = 0.0
        self.encoding = tiktoken.encoding_for_model(model_name)

        # Simplified pricing (actual pricing is more complex)
        self.price_per_1k_input = 0.03 if "gpt-4" in model_name else 0.0015
        self.price_per_1k_output = 0.06 if "gpt-4" in model_name else 0.002

    def call(self, messages, **kwargs):
        if self.spent_dollars >= self.budget_dollars:
            raise RuntimeError(f"Budget exceeded: ${self.spent_dollars:.4f} / ${self.budget_dollars}")

        # Estimate input cost
        input_tokens = sum(len(self.encoding.encode(str(m))) for m in messages)

        response = self.llm.call(messages, **kwargs)

        # Count actual output tokens
        output_tokens = len(self.encoding.encode(response.content))

        cost = (input_tokens / 1000 * self.price_per_1k_input + 
                output_tokens / 1000 * self.price_per_1k_output)
        self.spent_dollars += cost

        return response

# Usage with CrewAI
research_llm = BudgetEnforcedLLM("gpt-4-turbo-preview", budget_dollars=5.0)
researcher = Agent(
    role="Research Analyst",
    goal="Gather comprehensive market data",
    llm=research_llm,
    verbose=True
)

This works in theory, but falls apart in practice:

Pricing complexity: OpenAI's pricing varies by model version, has different rates for cached tokens, and changes frequently
Token counting edge cases: Function calls, vision inputs, and prompt caching make accurate token estimation nearly impossible
No persistence: Restarting your process resets budgets, making daily limits impossible
No observability: You're flying blind until something breaks

Production-Grade Budget Enforcement

The real solution is intercepting calls at the API level, not in your application code. This is where a proxy layer makes sense.

AWX Shredder is built specifically for this use case—it sits between your CrewAI agents and OpenAI's API, enforcing hard budget limits per agent role without requiring code changes beyond swapping the base URL:

import os
from crewai import Agent, Crew, Task, LLM

# Configure different budget headers for each agent role
research_llm = LLM(
    model="gpt-4-turbo-preview",
    base_url="https://awx-shredder.fly.dev/proxy/v1",
    api_key=os.getenv("OPENAI_API_KEY"),
    default_headers={"X-AWX-Agent-ID": "researcher", "X-AWX-Daily-Budget": "5.00"}
)

writer_llm = LLM(
    model="gpt-4-turbo-preview",
    base_url="https://awx-shredder.fly.dev/proxy/v1",
    api_key=os.getenv("OPENAI_API_KEY"),
    default_headers={"X-AWX-Agent-ID": "writer", "X-AWX-Daily-Budget": "3.00"}
)

researcher = Agent(
    role="Research Analyst",
    goal="Gather comprehensive market data",
    llm=research_llm,
    verbose=True
)

writer = Agent(
    role="Content Writer",
    goal="Create engaging reports from research",
    llm=writer_llm,
    verbose=True
)

The proxy tracks actual API costs in real-time and returns HTTP 429 (rate limit) responses the moment an agent exceeds its daily budget. CrewAI handles this gracefully—the agent fails fast rather than racking up surprise costs.

Rolling Your Own Proxy

If you need full control, building a simple budget-enforcing proxy isn't difficult. The minimal viable version needs:

A Redis instance for persisting spend by agent ID and date
A FastAPI or Express server that proxies to api.openai.com
Middleware that checks budget before forwarding requests
A parser for OpenAI's response to extract actual costs from the usage field

The tricky parts are handling streaming responses (costs aren't known until the stream completes) and dealing with OpenAI's eventual consistency in reporting costs. You'll also need to build alerting, dashboards, and handle key rotation.

For most teams, the build-vs-buy calculation favors a managed solution unless budget enforcement is core to your product's IP.

Key Takeaways

Set agent-level budgets based on each role's expected behavior, not flat limits across your crew
Enforce budgets at the API proxy level, not in application code
Use hard blocks (HTTP 429) rather than soft warnings—agents can't learn from budget alerts
Start conservative: set daily budgets at 2x your observed costs, then tighten after a week of production data

Today's action: audit your last month of OpenAI bills, break down costs by the logical "agent roles" you're running, and set per-role daily budgets at 150% of the highest daily spend you saw. Hard limits prevent one-off disasters while giving your agents room to operate.

Why Your LLM Agent Costs 10x More Than Your Estimate

AwxGlobal — Sat, 02 May 2026 07:00:43 +0000

Originally published at awx-shredder.fly.dev/blog

Why Your LLM Agent Costs 10x More Than Your Estimate

Your product manager approved the $500/month LLM budget. Two weeks later, you're staring at a $4,200 bill from OpenAI. The agent works perfectly in testing, but production is eating tokens like a memory leak eats RAM.

I've debugged this exact scenario four times in the past year. The culprit is never a single smoking gun—it's the multiplication of hidden costs that developers systematically underestimate during planning.

The Token Math Nobody Does

Most developers estimate LLM costs like this:

1000 requests/day × 500 tokens/request × $0.002/1k tokens = $1/day

This calculation assumes every request is a pristine, single-shot API call. Real agents don't work that way.

Here's what actually happens:

System prompts are charged on every call. That 800-token system prompt explaining your agent's role, output format, and business rules? It's not free. It's billed on every single request.
Tool call overhead compounds exponentially. Each function call requires the model to output JSON, you to execute the function, then send results back with the full conversation history. A single user request often triggers 3-5 tool calls.
Conversation history grows linearly. If you're maintaining context across turns (and you probably are), each subsequent message in a conversation includes all previous messages.

Let's recalculate with reality:

# What you estimated
simple_cost = 1000 * 500 * 0.002 / 1000
print(f"Estimate: ${simple_cost}/day")  # $1.00/day

# What actually happens
system_prompt_tokens = 800
avg_user_input = 150
avg_assistant_response = 300
tool_calls_per_request = 3.5  # Average across all requests
tool_call_overhead = 250  # JSON formatting + function results
conversation_turns = 4  # Average conversation length

# First turn
turn_1 = system_prompt_tokens + avg_user_input + avg_assistant_response

# Subsequent turns include history
turn_2 = turn_1 + avg_user_input + avg_assistant_response
turn_3 = turn_2 + avg_user_input + avg_assistant_response  
turn_4 = turn_3 + avg_user_input + avg_assistant_response

# Add tool call overhead
total_tokens = (turn_1 + turn_2 + turn_3 + turn_4) + (tool_calls_per_request * tool_call_overhead * conversation_turns)

real_cost = 1000 * total_tokens * 0.002 / 1000
print(f"Reality: ${real_cost}/day")  # $12.40/day

multiplier = real_cost / simple_cost
print(f"Hidden multiplier: {multiplier}x")  # 12.4x

This 12x multiplier is before we account for the really expensive mistakes.

Retry Loops: The Silent Budget Killer

Retry logic is essential for production reliability. It's also where costs spiral out of control.

Consider this common pattern:

from openai import OpenAI
import time

client = OpenAI()

def call_agent_with_retry(prompt, max_retries=3):
    for attempt in range(max_retries):
        try:
            response = client.chat.completions.create(
                model="gpt-4",
                messages=[{"role": "user", "content": prompt}],
                timeout=30
            )
            return response
        except Exception as e:
            if attempt == max_retries - 1:
                raise
            time.sleep(2 ** attempt)  # Exponential backoff

This looks reasonable. But what happens when OpenAI has a bad day and timeouts spike to 15% of requests?

15% of your 1000 daily requests now make 2-3 attempts
Each retry sends the full prompt again (including that 800-token system prompt)
Your token consumption jumps by 20-40% instantly

Worse yet: I've seen validation retry loops where the agent's output doesn't match the expected schema, so the developer adds logic to retry with error feedback. Each failed parse triggers another full API call with the previous attempt's context. A single malformed JSON response can cascade into 5-10 retry attempts.

Tool Call Explosion

Function calling feels efficient—until you look at the token counts.

Every tool call follows this pattern:

Model decides to call a function (outputs JSON)
Your code executes the function
Results are formatted and sent back
Model processes results and decides next step

Each step includes the full conversation history and system prompt. A research agent that calls search(), then fetch_url(), then extract_data() isn't making three cheap calls—it's making three increasingly expensive calls as the context window fills with previous tool results.

The economics get brutal with GPT-4. A complex agent workflow that feels like "just a few tool calls" can easily consume 15,000-20,000 tokens per user request.

Production Reality Check

After you've shipped and the costs are running hot, you need visibility and hard limits. Setting OpenAI spending limits helps but doesn't give you per-agent granularity or prevent runaway costs before they hit your credit card.

For production deployments where budget control is non-negotiable, tools like AWX Shredder (awx-shredder.fly.dev) act as a hard circuit breaker—an OpenAI-compatible proxy that blocks requests when an agent exceeds its daily budget. It takes one environment variable change and gives you real-time spend tracking with alerts before you blow through your allocation.

What You Should Do Today

Audit your actual token consumption. Log usage.total_tokens from every API response for a week. Calculate the median and p95. You'll be surprised.
Count your system prompt tokens. Use tiktoken to get exact counts. If your system prompt is over 500 tokens, consider whether every instruction is essential.
Track retry rates. Add metrics for how often your retry logic actually fires. Set alerts when retry rates exceed 5%.
Model tool call patterns. Log how many function calls the average request triggers. If it's more than 3, consider whether you can combine tools or reduce the decision tree.
Set hard budget limits per agent. Don't rely on cost estimates. Implement actual spending caps that prevent runaway costs.

The gap between estimated and actual LLM costs isn't a rounding error—it's the difference between a sustainable product and a budget crisis. The math is straightforward once you account for what actually gets billed.

[Boost]

AwxGlobal — Fri, 01 May 2026 22:55:32 +0000

AwxGlobal

May 1

Per-agent daily spend limits: the architecture every AI team needs

#ai #architecture #openai #agents

Comments

4 min read

Per-agent daily spend limits: the architecture every AI team needs

AwxGlobal — Fri, 01 May 2026 21:07:07 +0000

Originally published at awx-shredder.fly.dev/blog

Per-agent daily spend limits: the architecture every AI team needs

Your Slack bot just burned through $847 in four hours because a junior dev accidentally pushed a loop that called gpt-4-turbo on every message edit event. Your customer support agent hit an infinite reasoning loop and racked up $2,300 in o1-preview costs before anyone noticed. These aren't hypothetical scenarios—they're the kind of incidents that happen weekly across AI engineering teams.

The problem isn't that developers are careless. It's that LLM APIs have fundamentally different cost characteristics than traditional APIs. A single malformed request can cost $50. A logic bug can drain thousands before your monitoring alerts even fire. And when you're running multiple agents—research bots, customer support, data analysis tools—the blast radius of a single misconfigured agent can take down your entire API budget.

Why application-level budget checks fail

Most teams start with application-level budget enforcement. You add a counter in your database, increment it on each API call, and check before making requests:

async def call_llm(agent_id: str, messages: list):
    current_spend = await db.get_daily_spend(agent_id)
    if current_spend >= DAILY_LIMIT:
        raise BudgetExceededError()

    response = await openai.chat.completions.create(
        model="gpt-4-turbo",
        messages=messages
    )

    # Calculate and record cost
    cost = calculate_cost(response.usage)
    await db.increment_spend(agent_id, cost)
    return response

This looks reasonable until you hit production. The cost calculation happens after the API call completes. If your database write fails, you've lost spend tracking. If the process crashes between the API call and the database update, that cost vanishes. Race conditions mean multiple requests can check the budget simultaneously, all see they're under the limit, and fire off requests that collectively exceed it.

More critically: this pattern requires every callsite in your codebase to route through your budget enforcement logic. Third-party libraries that call OpenAI directly bypass it entirely. That LangChain agent you integrated? It's not checking budgets. The new engineer who doesn't know about your internal wrapper? They import openai directly and circumvent everything.

The proxy architecture

The robust solution is budget enforcement at the network layer. Every LLM API call flows through a proxy that:

Authenticates the agent making the request
Checks current spend against the daily limit before forwarding to the LLM provider
Blocks the request immediately if the limit is exceeded
Records actual costs from the LLM response
Aggregates spend across all instances of your application

This architecture makes budget enforcement impossible to bypass. Applications can't accidentally route around it because the proxy is configured at the network level via OPENAI_BASE_URL. Multiple application instances automatically share the same spend tracking because it's centralized in the proxy.

Here's what the client-side configuration looks like:

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
  baseURL: process.env.OPENAI_BASE_URL, // points to proxy
  defaultHeaders: {
    'X-Agent-ID': 'customer-support-bot'
  }
});

// This call is budget-enforced automatically
const response = await client.chat.completions.create({
  model: 'gpt-4-turbo',
  messages: [{ role: 'user', content: 'Hello' }]
});

The proxy intercepts the request, checks if customer-support-bot has budget remaining today, and either forwards it to OpenAI or returns a 429 error. Your application code doesn't need to think about budgets—they're enforced infrastructure-level.

Building vs. buying

Implementing a production-grade proxy requires solving several non-trivial problems:

Streaming support: LLM streaming responses require careful proxy handling to calculate costs from partial responses
Token counting accuracy: Different models have different pricing for input/output tokens, and your cost calculations need to match OpenAI's billing exactly
Atomic spend updates: You need transactional guarantees that spend increments don't get lost
Multi-region deployment: Low latency requires running the proxy close to your application
Alert fatigue: Teams need warnings before hitting limits, not just hard blocks

For teams that need this now, AWX Shredder is a production-ready proxy that handles all of this. Change OPENAI_BASE_URL to https://awx-shredder.fly.dev/proxy/v1, set per-agent daily budgets, and get email alerts at 50%/80%/100% thresholds. It's OpenAI-compatible, so existing code works unchanged.

For teams building internally, the core architecture is straightforward:

Run a lightweight HTTP proxy (Node.js with http-proxy-middleware or Python with aiohttp)
Use Redis for atomic spend tracking with daily key expiration
Parse token usage from OpenAI responses and multiply by model-specific pricing
Return 429 errors when budgets are exceeded
Implement request signing or API keys to authenticate agents

The tricky parts are handling streaming correctly (you need to buffer the response to extract token counts while still streaming to the client) and keeping your pricing table up to date as OpenAI changes model costs.

The enforcement guarantee

The key insight is that budget enforcement must happen before cost is incurred, not after. Application-level tracking is audit logging. Proxy-level blocking is actual enforcement.

When your proxy returns 429, that request never reaches OpenAI. No tokens are consumed. No cost is charged. The agent is hard-stopped until the daily limit resets. This guarantee—that exceeding a budget is architecturally impossible—is what lets you safely increase agent autonomy without fear of runaway costs.

What to do today

If you're running multiple AI agents in production, implement per-agent spend limits this week. The next production incident will happen—the question is whether it costs $50 or $5,000. Pick a proxy architecture (build or buy), assign realistic daily budgets to each agent (10-20% above their typical daily spend), and configure alerts before you hit limits. Your infrastructure should make expensive mistakes impossible, not just unlikely.

I woke up to a $400 OpenAI bill. Here's what I built to make sure it never happens again.

AwxGlobal — Fri, 01 May 2026 15:52:35 +0000

It was 2am when the email came in.

"Your OpenAI usage has exceeded $400."

An agent I'd been testing had hit a loop. It kept retrying a failed call, over and over, for six hours while I slept. By the time I saw it, the damage was done.

I went looking for a tool that would let me set a hard daily limit per agent — something that would block the call before it ever reached OpenAI. I couldn't find one.

So I built it.

The problem with AI agents and money

When you build with AI APIs, you're essentially writing a blank check. Your agent makes calls, OpenAI charges you, and you find out later. There's no circuit breaker. There's no per-agent budget. There's no "stop after $10."

Soft limits and billing alerts exist, but they notify you after the money is gone.

What I needed was a hard block. Something that intercepts the call and returns an error before it reaches the API.

What AWX Shredder does

It's a proxy. Instead of your agent calling api.openai.com directly, it calls your AWX endpoint. AWX checks the agent's daily budget, and either forwards the call or blocks it with a 402.

Setup is two lines:

OPENAI_BASE_URL=https://awx-shredder.fly.dev/proxy/v1
OPENAI_API_KEY=your_awx_key
You create agents in a dashboard, set a daily budget per agent, and that's it. Every call is logged. When an agent hits its limit, calls stop — not after, not eventually, right then.

Why per-agent matters

Most billing tools are org-level. You get one limit for everything.

But in a real system you might have five agents running at once — a researcher, a writer, a code reviewer, a scraper, a scheduler. If one goes rogue, you want to block that agent, not your whole org.

AWX gives each agent its own daily budget. One agent burning money doesn't affect the others.

It works with Claude too

Not just OpenAI. Point your Anthropic client at /proxy/v1/messages and the same budget enforcement applies.

Try it

It's free to start: awx-shredder.fly.dev

If you've ever had an unexpected AI bill, or you're building agents and want to sleep at night — this is for you.

I'm building this in public. Follow along if you want to see how it grows.

ai #openai #webdev #productivity