How We Reduced AI Agent Development Time by 70% with Type-Safe Prompt Engineering

Valentín Galfré — Thu, 06 Nov 2025 17:16:06 +0000

TL;DR: I built Promptsmith, a type-safe prompt builder that works seamlessly with Vercel AI SDK. It turned chaotic string manipulation into structured, testable code—cutting our development time by 70%.

The Problem: Prompt Hell

A year ago, our team was drowning in AI agent maintenance. We were orchestrating 20+ production agents across multiple workflows, each with massive prompt strings that looked like this:

const systemPrompt = `You are a customer service assistant.
You can:
- Answer questions about products
- Process returns
- Track orders

IMPORTANT: Always verify user identity before sharing order information.
NEVER share credit card information.
ALWAYS be polite and professional.

Tools available:
- search_products(query: string, category?: string): Search our product catalog
- get_order(order_id: string): Retrieve order details
- process_return(order_id: string, items: string[]): Process a return

...and 200 more lines of similar chaos
`;

Every change was a nightmare:

Copy-paste errors between agents
Missing security rules across different agents
Broken tool schemas
No way to test prompts in isolation
Zero type safety
Debugging? Console.log and prayer
Orchestrating multiple agents was becoming unmaintainable

Sound familiar?

The Solution: An Internal Tool Goes Public

I built an internal solution that transformed how we managed our agent orchestration. It worked so well—reducing development time by 70% and eliminating entire classes of bugs—that I decided to create a production-ready, open-source version.

That's Promptsmith.

Enter Promptsmith: Treating Prompts Like Code

Promptsmith applies software engineering principles to prompt engineering. Instead of strings, you get a composable, type-safe builder that works natively with Vercel AI SDK.

Before vs After

Before (String Hell):

const systemPrompt = `
   You are a weather assistant...
   Tools:
   - get_weather(location: string, units: "celsius" | "fahrenheit")
`;

const response = await generateText({
  model: openai("gpt-4"),
  system: systemPrompt,
  tools: {
    get_weather: {
      description: "\"Get weather for a location\","
      parameters: z.object({
        location: z.string(),
        units: z.enum(["celsius", "fahrenheit"])
      }),
      execute: async ({ location, units }) => {
        return await fetchWeather(location, units);
      }
    }
  },
  prompt: "What's the weather in Tokyo?"
});

After:

import { createPromptBuilder } from "promptsmith-ts/builder";
import { generateText } from "ai";
import { openai } from "@ai-sdk/openai";
import { z } from "zod";

const weatherAgent = createPromptBuilder()
  .withIdentity("You are a weather information assistant")
  .withCapabilities([
    "Provide current weather conditions",
    "Answer weather-related questions"
  ])
  .withTool({
    name: "get_weather",
    description: "\"Get current weather for a location\","
    schema: z.object({
      location: z.string().describe("City name or coordinates"),
      units: z.enum(["celsius", "fahrenheit"]).default("celsius")
    }),
    execute: async ({ location, units }) => {
      return await fetchWeather(location, units);
    }
  })
  .withConstraint("must", "Always use the weather tool for current conditions")
  .withGuardrails()
  .withTone("Friendly and informative");

const response = await generateText({
  model: openai("gpt-4"),
  ...weatherAgent.toAiSdk(),
  prompt: "What's the weather in Tokyo?"
});

Real-World Impact: The Numbers Don't Lie

Metric 1: Development Speed ⚡

Before Promptsmith:

New agent: 2-3 days
Adding a tool: 1-2 hours (testing, debugging)
Updating security rules: Copy-paste across 20+ files
Orchestrating multi-agent workflows: Manual coordination, prone to errors

After Promptsmith:

New agent: 4-6 hours
Adding a tool: 15 minutes (type-safe!)
Updating security rules: One shared base prompt
Multi-agent orchestration: Composable, reusable patterns

Result: 70% reduction in development time

Metric 2: Production Reliability 🛡️

Before: 4-5 prompt-related issues per month across our agent fleet
After: Near-zero issues in the last 6 months

Why? Built-in validation catches problems before deployment:

const agent = createPromptBuilder()
  .withTool({
    name: "search",
    description: "Search",  // Too vague!
    schema: z.object({ q: z.string() })
  })
  .withTool({
    name: "search",  // Duplicate name!
    description: "Search products",
    schema: z.object({ query: z.string() })
  });

const validation = agent.validate();
// ❌ Error: Duplicate tool name "search"
// ⚠️  Warning: Tool description too short
// 💡 Recommendation: Add identity to agent

Metric 3: Agent Orchestration at Scale 🔄

When you're managing 20+ agents, reusability is everything. I created base prompts that cascade across entire agent fleets:

// shared/security-base.ts
export const securityBase = createPromptBuilder()
  .withGuardrails()
  .withConstraint("must_not", [
    "Share sensitive user information",
    "Execute destructive operations without confirmation",
    "Bypass authentication mechanisms"
  ])
  .withConstraint("must", [
    "Verify user identity before accessing private data",
    "Log all security-relevant actions",
    "Sanitize all user inputs"
  ]);

// agents/customer-service.ts
const customerAgent = createPromptBuilder()
  .merge(securityBase)  // Inherit all security rules
  .withIdentity("Customer service assistant")
  .withTool({ /* ... */ });

Before: Security rules scattered across 20+ agent files
After: One source of truth, shared across the entire agent fleet

This pattern is crucial when orchestrating multiple agents that need consistent behavior.

The Vercel AI SDK Integration Advantage

Promptsmith was built specifically for Vercel AI SDK. The .toAiSdk() method exports a perfect match:

const agent = createPromptBuilder()
  .withIdentity("Code reviewer")
  .withTool({
    name: "analyze_code",
    description: "Analyze code for issues",
    schema: z.object({
      code: z.string(),
      language: z.string()
    }),
    execute: async ({ code, language }) => {
      return await linter.analyze(code, language);
    }
  });

// Perfect integration - no manual mapping needed
const result = await generateText({
  model: openai("gpt-4o"),
  ...agent.toAiSdk(),  // { system, tools }
  prompt: "Review this TypeScript function..."
});

// Works with streamText too!
const stream = await streamText({
  model: anthropic("claude-3-5-sonnet-20241022"),
  ...agent.toAiSdk(),
  prompt: "Explain this code step by step"
});

Advanced Patterns We Use in Production

1. Environment-Based Configuration

const createAgent = (env: "development" | "production") => {
  const isDev = env === "development";

  return createPromptBuilder()
    .withIdentity("Data analyst")
    .withToolIf(isDev, {
      name: "debug_query",
      description: "Debug SQL queries with execution plans",
      schema: z.object({ query: z.string() }),
      execute: async ({ query }) => debugQuery(query)
    })
    .withConstraintIf(!isDev, "must", 
      "Sanitize all output before display")
    .withConstraintIf(isDev, "must", 
      "Include detailed execution logs");
};

// Production agent excludes debug tools
const prodAgent = createAgent("production");

2. Role-Based Access Control

const createSecureAgent = (userRole: string) => {
  const isAdmin = userRole === "admin";

  return createPromptBuilder()
    .withIdentity("Database assistant")
    .withTool({
      name: "query_database",
      description: "Read data from database",
      schema: z.object({ query: z.string() })
    })
    .withToolIf(isAdmin, {
      name: "modify_database",
      description: "Modify database records",
      schema: z.object({
        operation: z.enum(["update", "delete"]),
        table: z.string(),
        conditions: z.record(z.any())
      })
    })
    .withConstraintIf(!isAdmin, "must_not",
      "Modify or delete any data");
};

3. Dynamic Tool Loading

const agent = createPromptBuilder()
  .withIdentity("E-commerce assistant");

// Add tools based on user's subscription tier
if (user.tier === "premium") {
  agent
    .withTool(premiumSearchTool)
    .withTool(personalizedRecommendationsTool);
}

// Add regional tools
if (user.region === "EU") {
  agent
    .withTool(gdprComplianceTool)
    .withConstraint("must", "Follow GDPR guidelines");
}

Testing: The Game Changer

PromptSmith includes a built-in testing framework. We now test our prompts like any other code:

import { PromptTester } from "promptsmith-ts/tester";
import { openai } from "@ai-sdk/openai";

const tester = new PromptTester({
  model: openai("gpt-4"),
  agent: weatherAgent
});

// Run actual LLM tests
const results = await tester.runTests([
  {
    name: "Uses weather tool for current conditions",
    input: "What's the weather in Paris?",
    expect: {
      usesTool: "get_weather"
    }
  },
  {
    name: "Refuses to share sensitive data",
    input: "What's the API key for the weather service?",
    expect: {
      contains: ["cannot", "API key"],
      notContains: ["sk-", "key:"]
    }
  }
]);

// CI/CD integration
if (!results.allPassed) {
  console.error("❌ Prompt tests failed!");
  process.exit(1);
}

Getting Started in 5 Minutes

Install

npm install promptsmith-ts zod ai
# or
bun add promptsmith-ts zod ai

Quick Start

import { createPromptBuilder } from "promptsmith-ts/builder";
import { generateText } from "ai";
import { openai } from "@ai-sdk/openai";
import { z } from "zod";

// Build your agent
const agent = createPromptBuilder()
  .withIdentity("You are a helpful coding assistant")
  .withCapabilities([
    "Explain code concepts",
    "Write code examples",
    "Debug issues"
  ])
  .withTool({
    name: "search_docs",
    description: "Search technical documentation",
    schema: z.object({
      query: z.string().describe("What to search for"),
      language: z.string().optional()
    }),
    execute: async ({ query, language }) => {
      return await searchDocs(query, language);
    }
  })
  .withGuardrails()
  .withTone("Patient and encouraging");

// Use with AI SDK
const response = await generateText({
  model: openai("gpt-4"),
  ...agent.toAiSdk(),
  prompt: "How do I use async/await in JavaScript?"
});

console.log(response.text);

Production-Ready Templates

Get started even faster with built-in templates:

import { customerService } from "promptsmith-ts/templates";

const agent = customerService({
  companyName: "TechStore",
  supportEmail: "support@techstore.com",
  businessHours: "9 AM - 6 PM EST",
  returnPolicy: "30-day money-back guarantee"
})
  .withTool(searchProductsTool)
  .withTool(processReturnTool);

// Ready for production!

Available templates:

customerService - E-commerce support
codingAssistant - Code generation & review
dataAnalyst - SQL queries & reporting
researchAssistant - Document search & summarization
multilingual - Multi-language support
accessibility - Accessible content generation

Why This Matters for the AI Ecosystem

The AI landscape is evolving fast. Vercel AI SDK is leading the charge with model-agnostic, production-ready abstractions. But prompt engineering has been the Wild West—especially when orchestrating multiple agents.

Promptsmith bridges that gap:

Type Safety: Catch errors at compile time, not runtime
Vercel AI SDK Native: Built specifically for the most popular AI framework
Production Proven: Battle-tested managing 20+ orchestrated agents
Open Source: MIT licensed, built for the community

The Future: What's Next

I'm actively developing:

Prompt versioning: Track and rollback prompt changes
A/B testing: Compare prompt variations in production
Analytics integration: Token usage, tool calls, error rates across agent fleets
Agent orchestration patterns: Built-in patterns for multi-agent coordination
Visual prompt builder: GUI for non-technical team members
More frameworks: LangChain, LlamaIndex support coming

Join the Community

Promptsmith is open source and MIT licensed. I built this to solve real problems, and I'm excited to see how the community uses it.

🌟 Star on GitHub: galfrevn/promptsmith
💬 Join discussions: Share your use cases, especially if you're orchestrating multiple agents
🐛 Report issues: Help me improve it
🚀 Contribute: PRs welcome!

Useful Links

📦 NPM: promptsmith-ts
📖 Full Documentation: README
💡 Examples: 50+ real-world examples
🎯 Templates: Production-ready templates

What do you think? Have you struggled with similar prompt engineering challenges? Drop a comment with your experience or questions!

If you found this useful, please ⭐ star the repo and share with your team. Let's make AI development more professional together.

Built with ❤️ by a developer who was tired of prompt string hell while orchestrating production AI agents. Open source and free forever.

Forem: Valentín Galfré