Forem: Bogdan Pistol

Cut Your LLM Costs by ~30% With Prompt Optimization (What Actually Works in Production)

Bogdan Pistol — Sat, 13 Dec 2025 17:07:39 +0000

If you’ve shipped an LLM-powered feature to production, you’ve probably experienced this:

The feature works
Users like it
Traffic grows
Your AI bill quietly explodes

The problem is not usually one big mistake. It’s death by a thousand small inefficiencies: bloated prompts, wrong model choices, missing caching, and zero visibility into what’s actually driving cost.

Over the past months, while working on LLM-heavy applications, we consistently saw 20–40% of spend wasted. Not because models are bad, but because prompts and execution paths aren’t treated as production assets.

This post breaks down where LLM costs really leak and the practical prompt optimization techniques that actually reduce spend in real systems.

Why LLM Costs Are Hard to Control

LLM pricing is simple on paper: you pay per token.

In practice, token usage is wildly unpredictable.

A single request cost depends on:

Prompt length
Context size
Model choice
Tool calls and retries
Hidden system prompts
User behavior

Once an app scales, these variables compound fast. Without instrumentation, most teams only notice the issue when the invoice lands.

The “Hidden 30%”: Where the Money Goes

Here are the most common cost leaks we see in production systems.

Overly Verbose Prompts

Prompts tend to grow organically:

Extra instructions
Repeated constraints
Historical context that is no longer relevant

Every additional token is paid every single request.

What works

Ruthlessly trim instructions
Remove redundant phrasing
Prefer clear structure over verbosity
Measure tokens per prompt version

Even small reductions add up at scale.

Bloated Context Windows

A common anti-pattern:

“Just pass everything to the model and let it decide.”

Large context windows are expensive and often unnecessary.

What works

Retrieve only relevant context (RAG with filtering)
Cap historical messages
Summarize long threads before reinjecting
Avoid entire-database prompts

Context relevance matters more than context size.

Using Frontier Models for Simple Tasks

Many systems default to a single top-tier model for everything:

Classification
Extraction
Formatting
Simple reasoning

This is convenient, but expensive.

What works

Route tasks by complexity
Use cheaper models for deterministic or shallow tasks
Reserve frontier models for reasoning-heavy paths

def route(task, input):
    if task in SIMPLE_TASKS:
        return call_model("cheap-model", input)
    return call_model("frontier-model", input)

This alone can cut costs dramatically.

No Caching or Reuse

LLMs are often called for:

Repeated prompts
Identical user actions
Static system instructions

Without caching, you pay repeatedly for the same output.

What works

Cache deterministic responses
Cache embeddings
Reuse prompt fragments instead of regenerating them
Detect duplicate calls

Caching is one of the highest ROI optimizations.

No Guardrails or Visibility

Most teams cannot answer:

Which prompt is the most expensive?
Which model drives most spend?
Where retries or failures inflate cost?

Without observability, optimization is guesswork.

What works

Track cost per request
Track tokens per prompt
Alert on abnormal usage
Compare prompt versions over time

At some point, most teams realize the issue isn’t knowing what to optimize —
it’s not having a single place to see prompts, executions, tokens, models, and cost together.

Once prompts are scattered across code, config files, and notebooks, even simple questions like
“Which prompt is costing us the most this week?” become hard to answer.

Prompt Optimization Is a Production Discipline

The key mindset shift is this:

Prompts are not static strings.

They are production assets.

That means:

Version them
Measure them
Optimize them
Roll back when needed

Treat prompt changes like code changes. The cost savings compound as usage grows.

Where This Breaks Down at Scale

Most of the techniques above are straightforward in isolation.

What becomes difficult at scale is:

Tracking prompt versions over time
Correlating prompts with executions and cost
Comparing models and optimizations objectively
Enforcing guardrails across teams

This is where teams usually move from ad-hoc scripts and dashboards
to a dedicated LLM observability and cost-control layer.

For context, this is exactly the problem space we’re working on with Dakora —
giving teams a unified view of prompts, executions, models, and cost, so prompt optimization stops being guesswork and becomes measurable.

The goal isn’t more dashboards.
It’s making cost, performance, and prompt changes visible in the same place.

A Simple Cost-Reduction Checklist

Measure tokens per request
Trim prompt verbosity
Reduce unnecessary context
Route tasks to cheaper models
Add caching for repeated calls
Track cost by prompt and model
Set basic spend alerts

Most teams that apply these systematically recover ~30% of wasted spend without changing product behavior.

Final Thoughts

LLMs are powerful, but they are not free.

As usage scales, prompt quality becomes a cost lever, not just a UX concern.

If you’re dealing with LLM cost surprises in production,
I’m happy to compare notes or walk through how other teams are approaching this.

You can explore what we’re building at https://dakora.io

feedback is very welcome.

Originally published on dakora.io

Building Intelligent Research Agents with OpenAI's Agents Framework

Bogdan Pistol — Mon, 06 Oct 2025 10:28:01 +0000

Multi-agent systems are transforming how we build AI applications. Instead of relying on a single large language model to handle every task, we can now orchestrate specialized agents that work together, each focused on what it does best. In this tutorial, we'll build a practical research assistant using OpenAI's Agents Framework—and see how clean prompt management makes agent development faster and more reliable.

Why OpenAI's Agents Framework?

Released in March 2025 as a production-ready evolution of OpenAI's experimental Swarm project, the Agents Framework stands out in a crowded field of agent frameworks for several key reasons:

Lightweight by Design

Unlike heavyweight frameworks that require mastering complex graph structures or conversation patterns, OpenAI's Agents SDK introduces just four core primitives:

Agents - LLMs equipped with instructions and tools
Handoffs - Enable agents to delegate tasks to specialists
Guardrails - Validate inputs and outputs for safety
Sessions - Automatically maintain conversation history

This minimalist approach means you can build sophisticated multi-agent systems without wrestling with unnecessary abstraction layers.

Production-Ready from Day One

While frameworks like LangGraph excel at complex stateful workflows and AutoGen shines in multi-role conversations, OpenAI's Agents SDK focuses on production readiness. It includes:

Built-in tracing for debugging and monitoring
Automatic schema generation for function tools
Seamless integration with GPT-4o and other OpenAI models
Human-in-the-loop (HITL) approval for critical decisions

Provider-Agnostic with 100+ LLM Support

Despite being an OpenAI product, the framework is compatible with over 100 different language models, giving you flexibility as the AI landscape evolves.

What We're Building

Our research assistant demonstrates real multi-agent coordination:

Research Planner breaks topics into subtopics and questions
Analyst agents dive deep into each subtopic
Synthesizer combines findings into coherent insights

This pattern applies to countless use cases: competitive analysis, market research, technical documentation, and more.

Setup: Getting Started

Prerequisites

You'll need:

Python 3.11.5+ or 3.12+ (earlier 3.11 versions have typing compatibility issues)
An OpenAI API key (get one here)

Installation

# Clone the repository
git clone https://github.com/bogdan-pistol/dakora.git
cd dakora/examples/openai-agents

# Create virtual environment
python3 -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

Configure Your API Key

# Copy the example environment file
cp .env.example .env

# Edit .env and add your OpenAI API key
# OPENAI_API_KEY=sk-proj-...

Where to get your API key:

Visit OpenAI Platform
Navigate to API Keys in your account settings
Create a new secret key
Copy and paste it into your .env file

Running Your First Research

Let's see it in action:

python research_assistant.py "AI agent frameworks in 2025"

Example Output

🔬 Starting research on: AI agent frameworks in 2025

📋 Planning research strategy...

📊 Research Plan:
Strategy: This research plan will employ a literature review, case studies, and expert
interviews to gather data on the evolution, integration, and ethical considerations of
AI agent frameworks, aiming to provide a comprehensive understanding of their state in 2025.
Subtopics: 3

🔍 Analyzing subtopic 1/3: Evolution of AI Agent Frameworks
✓ Analysis complete

🔍 Analyzing subtopic 2/3: Integration and Interoperability of AI Agents
✓ Analysis complete

🔍 Analyzing subtopic 3/3: Ethical and Regulatory Considerations
✓ Analysis complete

🔄 Synthesizing findings...
✓ Synthesis complete

================================================================================

📄 RESEARCH REPORT

================================================================================
**Executive Summary**

The evolution of AI agent frameworks in 2025 is marked by significant advancements
in technology, fostering sophisticated autonomous systems capable of handling complex
tasks across industries. These systems integrate machine learning models like GPT and
BERT, allowing for enhanced natural language processing and multi-agent collaboration...

[Full report output...]

================================================================================

💾 Full report saved to: research_output_20251006_103427.json

Understanding the Architecture

The Multi-Agent Workflow

Our system follows a clear pattern:

User Query → Research Planner → Multiple Analysts (parallel) → Synthesizer → Report

Each agent has a specific role, making the system easier to debug, test, and extend.

Key Code Components

Let's walk through the main building blocks:

1. Agent Creation with Managed Prompts

from agents import Agent
from dakora.vault import Vault

vault = Vault(config_path="dakora.yaml")

def create_research_planner():
    prompt = vault.get("planner_system")
    return Agent(
        name="Research Planner",
        instructions=prompt.render(),
        model="gpt-4o-mini"
    )

Why this matters: Instead of hardcoding prompts in Python, we store them as versioned YAML files. This separation of concerns means prompt engineers can iterate on instructions without touching code.

2. Dynamic Prompt Rendering

analyst_prompt = vault.get("analyst_system")
analyst_instructions = analyst_prompt.render(
    subtopic=subtopic["title"],
    questions=subtopic.get("questions", []),
    context=f"Part of broader research on: {topic}",
    analysis_depth="standard",
    source_types=subtopic.get("sources", [])
)

analyst = Agent(
    name=f"Analyst-{idx}",
    instructions=analyst_instructions,
    model="gpt-4o-mini"
)

Why this matters: Each analyst gets customized instructions based on its specific subtopic. The same prompt template adapts to different contexts, reducing duplication.

3. Agent Execution with OpenAI's Runner

from agents import Runner

plan_result = Runner.run_sync(planner, f"Create a research plan for: {topic}")

# Process the output
plan_data = json.loads(plan_result.final_output)

Why this matters: The Runner handles session management, conversation history, and tool execution automatically. You focus on orchestration, not plumbing.

4. Parallel Analysis

findings = []

for idx, subtopic in enumerate(plan_data.get("subtopics", []), 1):
    analyst = Agent(
        name=f"Analyst-{idx}",
        instructions=analyst_instructions,
        model="gpt-4o-mini"
    )

    analysis_result = Runner.run_sync(analyst, f"Analyze: {subtopic['title']}")

    findings.append({
        "subtopic": subtopic["title"],
        "analysis": analysis_result.final_output
    })

Why this matters: Each subtopic gets dedicated analysis. While this example processes sequentially, the pattern easily extends to parallel execution for faster results.

Introducing Dakora: Clean Prompt Management

As our research assistant grew, a challenge emerged: managing increasingly complex agent instructions. This is where Dakora proved invaluable.

The Prompt Management Problem

Multi-agent systems often mean:

Dozens of prompt variations
Dynamic content injection
Version tracking needs
Type safety requirements

Hardcoding prompts becomes unmaintainable. Template strings help, but lack structure. We needed something purpose-built.

Why Dakora?

Dakora is a lightweight Python library designed specifically for prompt template management. Here's what made it the right choice for this project:

Type-Safe Inputs

# prompts/analyst_system.yaml
id: analyst_system
version: 1.0.0
description: System prompt for deep analysis

inputs:
  subtopic:
    type: string
    required: true
  questions:
    type: array<string>
    required: true
  analysis_depth:
    type: string
    required: false
    default: standard
  source_types:
    type: array<string>
    required: false
    default: []

Input validation happens before the LLM call, catching errors early and saving API costs.

Hot Reload During Development

Edit a prompt file, re-run your script—changes apply immediately. No restart required.

# Edit analyst prompt to add new analysis section
vim prompts/analyst_system.yaml

# Changes apply immediately
python research_assistant.py "your topic"

Version Control Integration

id: analyst_system
version: 1.1.0  # Bumped from 1.0.0
description: Added economic impact analysis section

template: |
  You are a research analyst specializing in deep, structured analysis.

  Analysis Framework:
  1. Current State
  2. Key Developments
  3. Economic Impact  # New section
  4. Challenges & Opportunities
  5. Future Outlook

Track prompt evolution alongside code changes. Roll back if needed. It's all in git.

Visual Prompt Editor

Dakora includes an interactive web playground for testing templates:

dakora playground

This browser-based editor lets you:

Test prompts with different inputs
See rendered output instantly
Validate type safety
Share templates with your team

Try it online at playground.dakora.io

How Dakora Fits In

In our research assistant, Dakora handles:

Storage - All prompts live in prompts/ as YAML files
Validation - Type-safe inputs catch errors before API calls
Rendering - Jinja2 templates with custom filters
Versioning - Semantic versioning built-in

The OpenAI Agents Framework handles:

Execution - Agent coordination and tool calling
State - Conversation history and session management
Integration - OpenAI API communication

This separation means cleaner code, faster iteration, and easier collaboration between developers and prompt engineers.

Exploring the Prompts

Let's look at how prompts are structured:

Research Planner Prompt

id: planner_system
version: 1.0.0
description: Creates research strategies and identifies subtopics

template: |
  You are a research planning specialist. Your task is to analyze research
  topics and create comprehensive research strategies.

  Research Topic: {{topic}}
  Number of Subtopics: {{num_subtopics}}
  {% if focus_areas|length > 0 %}
  Focus Areas: {{focus_areas|yaml}}
  {% endif %}

  IMPORTANT: You must respond with ONLY valid JSON, no additional text.

  Format your response as this exact JSON structure:
  {
    "subtopics": [
      {
        "title": "Subtopic title",
        "questions": ["Q1", "Q2", "Q3"],
        "sources": ["Source type 1", "Source type 2"],
        "priority": 1
      }
    ],
    "research_strategy": "Brief overview of the research approach"
  }

inputs:
  topic:
    type: string
    required: true
  num_subtopics:
    type: number
    default: 3
  focus_areas:
    type: array<string>
    default: []

Analyst Prompt with Conditional Logic

id: analyst_system
version: 1.0.0
description: Deep analysis framework with conditional depth

template: |
  You are a research analyst specializing in deep, structured analysis.

  Assignment:
  Subtopic: {{subtopic}}
  Research Questions: {{questions|yaml}}

  Analysis Depth: {{analysis_depth}}
  {% if analysis_depth == "comprehensive" %}
  Provide detailed analysis with specific examples, data points, and multiple viewpoints.
  {% elif analysis_depth == "standard" %}
  Provide balanced analysis covering main points with supporting examples.
  {% else %}
  Provide concise analysis focusing on the most critical insights.
  {% endif %}

inputs:
  subtopic:
    type: string
    required: true
  questions:
    type: array<string>
    required: true
  analysis_depth:
    type: string
    default: standard

The {% if %} conditionals let one template handle multiple depth levels—no need for separate prompts.

Project Structure

Here's how everything is organized:

openai-agents/
├── research_assistant.py       # Main orchestration script
├── prompts/                    # Dakora template directory
│   ├── coordinator_system.yaml # Workflow orchestration
│   ├── planner_system.yaml     # Research strategy
│   ├── analyst_system.yaml     # Deep analysis
│   ├── synthesizer_system.yaml # Finding synthesis
│   └── report_template.yaml    # Output formatting
├── dakora.yaml                 # Dakora configuration
├── requirements.txt            # Python dependencies
├── .env                        # API keys (not in git)
└── README.md

The dakora.yaml config file:

registry: local
prompt_dir: ./prompts
logging:
  enabled: false

Extending the System

Add a New Agent Type

Create a prompt template:

# prompts/fact_checker.yaml
id: fact_checker
version: 1.0.0
description: Validates claims against sources

template: |
  You are a fact-checking specialist. Verify the following claims:

  {{claims}}

  For each claim, provide:
  - Verification status (verified/unverified/false)
  - Supporting evidence
  - Confidence level

inputs:
  claims:
    type: string
    required: true

Load it in your code:

fact_checker_prompt = vault.get("fact_checker")
fact_checker = Agent(
    name="Fact Checker",
    instructions=fact_checker_prompt.render(claims=findings_text)
)

Implement Agent Handoffs

Use the framework's handoff mechanism:

from agents import Agent, Handoff

fact_checker = Agent(
    name="Fact Checker",
    instructions=vault.get("fact_checker").render()
)

analyst = Agent(
    name="Analyst",
    instructions=analyst_instructions,
    handoffs=[Handoff(target=fact_checker)]
)

When the analyst encounters uncertain claims, it can hand off to the fact checker automatically.

Command Line Interface

Dakora includes a CLI for prompt management:

# List all templates
dakora list

# View a specific prompt
dakora get analyst_system

# Bump version
dakora bump analyst_system --minor

# Watch for changes (hot reload)
dakora watch

Key Takeaways

OpenAI Agents Framework Strengths

Simplicity - Four primitives cover most agent patterns
Production-Ready - Built-in tracing, guardrails, and HITL
Flexibility - Works with 100+ LLMs, not just OpenAI
Growing Ecosystem - TypeScript support, voice agents, and more

When to Use This Framework

Great fit for:

Production applications requiring reliability
Projects already using OpenAI models
Teams valuing simplicity over complex workflows
Applications needing human oversight (HITL)

Consider alternatives if:

You need complex graph-based workflows (try LangGraph)
Multi-role conversations are central (try AutoGen)
You're building on non-OpenAI infrastructure

Dakora for Prompt Management

Use Dakora when:

Managing 10+ prompts across multiple agents
Collaborating between developers and prompt engineers
Version tracking and rollback are important
Type safety prevents costly API errors

Benefits we experienced:

3x faster prompt iteration
Zero hardcoded strings in agent code
Easy A/B testing of prompt variations
Clear audit trail of prompt changes

Resources

Code & Documentation

This Example: github.com/bogdan-pistol/dakora/examples/openai-agents
Dakora Repository: github.com/bogdan-pistol/dakora
OpenAI Agents SDK: github.com/openai/openai-agents-python
Dakora Website: dakora.io
Interactive Playground: playground.dakora.io

Official Documentation

OpenAI Agents Guide: openai.github.io/openai-agents-python

Community

Dakora Discord: Join the community
OpenAI Forum: community.openai.com

Next Steps

Try building your own multi-agent system:

Clone the example and run it locally
Modify the analyst prompt to add new analysis sections
Add a fourth agent type (summarizer, fact-checker, etc.)
Experiment with different models (GPT-4o, GPT-4o-mini, Claude, etc.)
Integrate with your own data sources or APIs

The combination of OpenAI's Agents Framework and Dakora's prompt management creates a powerful foundation for building reliable, maintainable AI agents. Start simple, iterate fast, and scale as you learn.

Built something cool with OpenAI Agents and Dakora? Share it in the Dakora Discord—we'd love to see what you create.

Stop Hardcoding Prompts: A Practical Workflow for AI Teams

Bogdan Pistol — Thu, 02 Oct 2025 14:32:10 +0000

If you’ve built anything with OpenAI or other LLMs, chances are your prompts live as strings in your codebase:

from openai import OpenAI
client = OpenAI()

response = client.responses.create(
    model="gpt-5",
    input="Write a short bedtime story about a unicorn."
)

print(response.output_text)

This works fine. Everything is in one place, and you don’t need to think twice about it.

But this example is a single, one-line prompt. Most production-ready apps use dozens, sometimes hundreds. Some prompts stretch over hundreds of lines, with parameters and variations.

The Problem: Prompt Hell

Inlining prompts directly into your code doesn’t scale:

Files become filled with long blocks of text.
Every tweak means hunting down the right string, editing it, then rebuilding and redeploying.
Larger prompts with multiple parameters quickly get unreadable.
Copy-pasting across repos or services introduces errors.

You end up with what many call prompt hell — messy, time-consuming, and error-prone.

A Naive Fix (and Its Limits)

You might try moving prompts into YAML, JSON, or .env files. That’s better than inline, but it brings its own pain:

Manually editing YAML is brittle (one wrong indent breaks everything).
Validation is poor — missing parameters or wrong types only show up at runtime.
Collaboration gets awkward when multiple people are changing raw text files.

This is duct tape, not a workflow.

A Better Way: Externalize and Manage Prompts

Prompts deserve the same treatment as code and config: versioned, editable, and testable.

That’s why tools like Dakora exist — lightweight, open source, and built for small teams who don’t want to fight with prompt sprawl.

With Dakora you can:

Keep prompts in a central vault.
Edit them in a clean web UI instead of raw YAML.
Sync changes into your app instantly.
Back everything with local files under Git for transparency and history.

Quickstart

Getting started is a two-liner:

pip install dakora
dakora init
dakora playground

This opens a simple UI playground where you can manage and edit prompts in real time. No redeploys, no fiddly JSON.

Why It Matters

LLM apps evolve fast. If you’re shipping features, testing variations, or working with a team, you can’t afford to redeploy every time you tweak wording.

Prompt management isn’t just a “nice to have.” It’s the difference between hacking on the weekend and running a reliable product.

Try It Out

Dakora is open source and ready to use today:

⭐ Star it on GitHub
📖 Read the docs

Stop hardcoding prompts. Your team — and your future self — will thank you.

Photo by Mii Luthman on Unsplash