i spent 2 weeks comparing deep research APIs so you don’t have to

Vani Agarwal — Wed, 24 Dec 2025 23:28:01 +0000

deep research APIs are a new category. OpenAI, Perplexity, Google, and startups like Parallel are all shipping systems that can browse the web, synthesize sources, and return cited answers in a single API call. these tools are powerful, BUT choosing between them currently is not.

over the last couple weeks, i kept running into the same problem:

pricing scattered across docs
capabilities buried in changelogs
benchmarks inconsistent, missing, or not public
“it returns citations” used as a proxy for quality

so instead of guessing, i decided to actually compare them. this post is about what i learned, how i think about evaluation now, and why i built a public index to track this space.

how i approached evaluation

instead of reading docs in isolation, i did three things in parallel:

ran real queries: i tested market analysis, technical research, competitive intel, and product strategy prompts across providers — not just trivia or synthetic benchmarks.
read provider docs + benchmarks side-by-side: this made inconsistencies obvious very quickly, such as different definitions of “grounding”, benchmarks that aren’t comparable and pricing models that hide real cost drivers
tracked tradeoffs: there isn’t a single “best” deep research api. there are tradeoffs, specifically, depth vs latency, or cost vs accuracy or batch workflows vs interactive use. once i framed it that way, the space became much clearer.

why i built the deep research api index

after doing this manually, i realized there was no neutral, centralized place to:

compare providers side-by-side
run the same prompt across them
track how things change over time

so i built one at deep-research-index.vercel.app

it’s an independent index that:

compares providers across concrete metrics
lets you run identical prompts across multiple apis
makes tradeoffs explicit instead of hiding them

this isn’t a startup pitch, it's more it’s a reference i wanted to exist.

Forem: Vani Agarwal

i spent 2 weeks comparing deep research APIs so you don’t have to

how i approached evaluation

why i built the deep research api index