Forem: Adarsh Kant

Building Real-Time Voice Forms with Google Gemini API: Architecture & Learnings

Adarsh Kant — Sun, 05 Apr 2026 21:43:54 +0000

When you want to build voice-input forms that feel responsive and intuitive, the key challenge isn't transcription—modern APIs handle that well. It's latency. Transcription that takes 2 seconds to return feels broken. Transcription that streams back in real-time (200-400ms for first token) feels magical.

This post walks through the architecture we built at Anve Voice Forms to make real-time voice transcription feel fast and seamless in the browser.

The Challenge: Why Basic Transcription APIs Feel Slow

Most voice API approaches work like this:

User speaks for N seconds
Collect all audio
Send entire audio file to API
Wait for transcription response
Display result

Round-trip latency: 2-5 seconds. That's dead time where the user is waiting and nothing is happening.

The better approach is streaming: send audio chunks as they arrive, start processing immediately, and stream back results in real-time.

The Architecture

Here's the high-level flow:

Browser (Frontend)
  Microphone API → WebAudio Processor → WebSocket Client
                                              │ Chunks
                                              ▼
Backend (Node.js/Python)
  WebSocket Server → Audio Processor → Gemini API (Streaming)
                          │
                          ▼
                    Transcript Builder → Browser updates UI

1. Browser-Side Audio Capture

// Capture audio from microphone
const audioContext = new (window.AudioContext || window.webkitAudioContext)();
const mediaStream = await navigator.mediaDevices.getUserMedia({ audio: true });
const source = audioContext.createMediaStreamAudioSource(mediaStream);

const processor = audioContext.createScriptProcessor(4096, 1, 1);

processor.onaudioprocess = (event) => {
  const audioData = event.inputBuffer.getChannelData(0);
  const pcmData = new Float32Array(audioData);
  const int16Data = float32ToInt16(pcmData);
  socket.emit('audio_chunk', int16Data);
};

source.connect(processor);
processor.connect(audioContext.destination);

function float32ToInt16(float32Array) {
  const int16Array = new Int16Array(float32Array.length);
  for (let i = 0; i < float32Array.length; i++) {
    int16Array[i] = float32Array[i] < 0
      ? float32Array[i] * 0x8000
      : float32Array[i] * 0x7fff;
  }
  return int16Array;
}

Key decisions:

4096 sample chunk size: 93ms at 44.1kHz (good balance between latency and overhead)
Int16 encoding: most APIs expect 16-bit PCM audio
Send immediately: don't buffer, start streaming as chunks arrive

2. Streaming to Gemini API

This is where real-time transcription happens:

const { GoogleGenerativeAI } = require("@google/generative-ai");
const genAI = new GoogleGenerativeAI(process.env.GEMINI_API_KEY);

async function transcribeAudioStream(ws, audioChunks) {
  const model = genAI.getGenerativeModel({ model: "gemini-2.0-flash" });

  const response = await model.generateContentStream({
    contents: [{
      role: "user",
      parts: [
        { inlineData: { mimeType: "audio/mp3", data: audioStream } },
        { text: "Transcribe this audio. Return ONLY the transcription." }
      ]
    }]
  });

  for await (const chunk of response.stream) {
    const text = chunk.text();
    if (text) {
      ws.send(JSON.stringify({
        type: 'partial_transcript',
        text: text,
        timestamp: Date.now()
      }));
    }
  }
}

3. Handling Codec Mismatches

This was our biggest surprise issue. Browsers capture audio as PCM (44.1kHz, 16-bit mono). But APIs have different requirements — some want WAV, some MP3, some raw PCM.

const ffmpeg = require('fluent-ffmpeg');

async function convertAudioCodec(inputBuffer, outputFormat) {
  return new Promise((resolve, reject) => {
    ffmpeg(inputBuffer)
      .format(outputFormat)
      .audioFrequency(16000)
      .audioChannels(1)
      .on('end', () => resolve(outputBuffer))
      .on('error', reject)
      .pipe(outputBuffer);
  });
}

4. Latency Optimization

Real-time means <500ms perception. Our latency breakdown:

Browser capture: 93ms (chunk size)
Network round-trip: 50ms
Gemini processing: 150ms
Response streaming: 20ms
Total: ~310ms before transcription appears

5. Cost Optimization

// Don't send silence
function shouldSendChunk(audioData, threshold = 0.01) {
  const rms = Math.sqrt(
    audioData.reduce((sum, s) => sum + s ** 2, 0) / audioData.length
  );
  return rms > threshold;
}

We estimate $0.0005 per form submission at scale.

Lessons Learned

Streaming changes everything. 500ms feels slow. 200ms feels responsive.
Test with real audio. Background noise, accents, quiet voices — test aggressively.
Browser audio APIs are still janky. ScriptProcessorNode is deprecated but most compatible.
Don't ignore codec issues. We lost 2 weeks to garbage transcription from wrong formats.
Frontend UX matters. Debounce updates, show partial results clearly.

Production Stack

Frontend: React + WebSocket client
Backend: Node.js with ws library
API: Google Gemini 2.0 Flash
Codec: ffmpeg-wasm (browser) + ffmpeg (backend)
Hosting: Render + Cloudflare CDN

Building something with voice? We'd love to hear about it. Drop a comment or check out Anve Voice Forms if you want to see this architecture in action.

—Adarsh, Founder @ Anve Voice Forms

I Built a Voice-Powered Form Builder and 87% of Users Complete It

Adarsh Kant — Sat, 04 Apr 2026 21:54:07 +0000

Every developer has built a form. And every developer knows the pain: you spend hours perfecting the UX, adding validation, making it responsive... and then 85% of users abandon it halfway through.

I got tired of this. So I built Anve Voice Forms — a form builder where users can speak their answers instead of typing them.

The Problem With Text Forms

Here's what the data actually shows:

Average form completion rate: 15-20% (Formstack, 2024)
Average time to complete a 10-field form: 4 minutes 23 seconds
#1 reason for abandonment: "Too many fields" / "Takes too long"
Mobile form completion is 30% lower than desktop

We've been building forms the same way since the 90s. Text input, validation, submit. The entire interaction model assumes users want to type. But 40% of the world's population prefers voice input — whether due to accessibility needs, mobile context, or just convenience.

What I Built

Anve Voice Forms lets you create forms where users can speak their answers. The voice engine (powered by Google Gemini's multimodal API) transcribes responses in real-time across 40+ languages.

The tech stack:

React + TypeScript + Vite (frontend)
Tailwind CSS (styling)
Supabase (database + auth + edge functions)
Clerk (authentication)
Google Gemini API (voice processing via real-time WebSocket streaming)
Razorpay (payments)

How it works:

You build a form (drag-and-drop, just like Typeform)
Each field can accept text OR voice input
When a user clicks the mic, Gemini processes their speech in real-time
The response is transcribed, validated, and stored
You get analytics on completion rates, voice vs text usage, and more

The Results

After testing with early users across education, HR, and customer feedback use cases:

87%+ completion rates (vs ~15-20% industry average for text)
3x faster form completion time
40+ languages supported out of the box
Users on mobile completed forms 2.5x faster with voice

The biggest surprise? Users who had the option of voice but chose text still completed at higher rates. Just having voice as a fallback reduced anxiety about long forms.

Why Voice Changes Everything for Forms

1. Accessibility is built-in, not bolted on
1.3 billion people globally have some form of disability. Voice input isn't a nice-to-have — it's how a huge chunk of the world interacts with technology.

2. Multilingual by default
If your form serves users in multiple languages, voice forms handle it natively. No translation layers, no per-language form variants. A user in Tamil Nadu speaks Tamil, a user in Berlin speaks German — same form.

3. Mobile-first UX
Typing on a phone is slow and error-prone. Voice is the natural input method for mobile. Forms that support voice see significantly higher mobile completion rates.

The Architecture

The voice processing pipeline:

User speaks → WebSocket to Gemini API → Real-time transcription → Client-side validation → Supabase insert → Analytics event

Key technical decisions:

WebSocket streaming over REST for real-time feel
Client-side audio processing — only processed text is stored
Supabase Edge Functions for server-side logic
Progressive enhancement — voice is additive, text always works

Try It / Get a Lifetime Deal

I'm running a limited launch: 500 lifetime licenses at $199 (one-time payment, lifetime access).

What you get:

Unlimited text form submissions (forever)
50 voice responses/month
Analytics dashboard
API access + webhooks
40+ languages
Lifetime updates

Live demo: voiceforms.anvevoice.app/lifetime/

Main app: forms.anvevoice.app

What's Next

Currently working on:

Zapier + Make integrations
Conditional logic for voice flows
Team collaboration features
White-label option for agencies

Would love feedback from the dev community. What would you build with voice-powered forms? Drop a comment.

Built by Adarsh — indie founder from India.

How I Built a Voice AI That Takes Real DOM Actions on Websites

Adarsh Kant — Sat, 21 Mar 2026 19:31:19 +0000

Every voice AI tool I evaluated did the same thing: listen to speech, convert to text, send to an LLM, return audio. Essentially a chatbot with a microphone.

But I wanted something different. I wanted voice AI that could actually do things on a website — click buttons, fill forms, navigate pages. A voice agent, not a voice chatbot.

So I built AnveVoice.

The Problem with Voice Chatbots

Here's what most "voice AI" tools do:

User speaks
Speech-to-text converts it
Text goes to an LLM
LLM generates a response
Text-to-speech reads it back

That's it. The AI talks back, but it doesn't do anything. It can't click your "Book Appointment" button. It can't fill in your contact form. It can't navigate to your pricing page.

For websites, this is a huge missed opportunity. 96.3% of websites fail basic accessibility standards (WebAIM 2025). Voice navigation isn't just a feature — it's an accessibility requirement.

The Architecture: Voice → Intent → DOM Action

Here's how AnveVoice works differently:

User Speech → STT (sub-200ms) → Intent Parser → Action Router
                                                    ↓
                                    ┌───────────────┼───────────────┐
                                    ↓               ↓               ↓
                              DOM Actions      Navigation      Form Fill
                              (click, scroll)  (page redirect)  (input values)
                                    ↓               ↓               ↓
                              Visual Feedback → TTS Response → State Update

The key innovation is the Action Router. Instead of just generating text responses, the AI interprets user intent and maps it to real DOM actions using 46 MCP (Model Context Protocol) tools over JSON-RPC 2.0.

Real DOM Actions

When a user says "Book an appointment for Tuesday," AnveVoice doesn't just say "I'd be happy to help you book an appointment." It actually:

Identifies the booking form on the page
Fills in the date field with next Tuesday's date
Clicks the submit button
Confirms the booking with voice feedback

This is possible because we maintain a real-time DOM map of the page and use semantic understanding to match user intents to actionable elements.

The Technical Challenge: Sub-700ms Latency

End-to-end voice latency needs to be under 1 second to feel natural. Here's our pipeline:

Stage	Target	Actual
STT	< 200ms	~180ms
Intent Parse	< 100ms	~80ms
Action Execution	< 200ms	~150ms
TTS	< 200ms	~190ms
Total	< 700ms	~600ms

We achieve this by:

Streaming STT — processing audio chunks as they arrive, not waiting for silence detection
Pre-computed DOM maps — indexing actionable elements on page load so we don't need to traverse the DOM at query time
Parallel TTS — starting speech synthesis while the action is still executing
Edge inference — running intent classification at the edge, not round-tripping to a central server

The Embed: One Script Tag

The entire integration is a single script tag:

<script 
  src="https://widget.anvevoice.app/embed.js" 
  data-agent-id="YOUR_AGENT_ID">
</script>

That's it. No WebRTC server management. No complex API integration. Works with React, Vue, Angular, Next.js, Shopify, WordPress, or any HTML page.

The widget handles:

Microphone permission and audio capture
Real-time speech recognition in 50+ languages
Intent classification and action routing
DOM manipulation and visual feedback
Text-to-speech response in the detected language

50+ Languages (Including 22 Indian Languages)

This was a non-negotiable for us. India has 700M+ smartphone users, and 65 of every 100 mobile searches happen in non-English languages.

We support all 22 scheduled Indian languages plus Hinglish (Hindi-English code-switching), which is how most urban Indians actually communicate with technology.

The language detection works automatically — if a user starts speaking Hindi, the system detects it, locks to Hindi for the session, and responds in Hindi. No configuration needed.

Pricing: Flat-Rate vs. Per-Minute

Most voice AI tools charge per minute:

Retell AI: ~$0.13-0.31/min
Vapi: ~$0.15-0.33/min
ElevenLabs: ~$0.08-0.10/min

At 1,000 minutes/month, that's $80-$330.

AnveVoice uses flat-rate token pricing:

Free: $0/mo (50K tokens)
Growth: $35/mo (500K tokens, 3 bots)
Enterprise: Custom

Predictable costs. No surprise bills.

What's Next

We're currently focused on:

Healthcare — 94% appointment booking success rate in pilot clinics
E-commerce — Voice-powered product discovery and checkout
Government portals — Citizen services in vernacular languages
Accessibility — Making WCAG 2.1 AA compliance achievable through voice

Try It

You can try AnveVoice at anvevoice.app or see the experience hub at experience.anvevoice.app.

The embed is free to start. If you're building a website that needs voice interaction — especially if accessibility or multilingual support matters — give it a try.

I'm Adarsh, founder of ANVE.AI. I'm a cybersecurity professional (CISA/CEH certified) who got obsessed with making the web more accessible through voice. If you have questions about the architecture or want to discuss voice AI, drop a comment below or find me on LinkedIn.

From 0 to 100+ Users: What Actually Worked After 20,000 SEO Pages Got Us Nothing

Adarsh Kant — Thu, 19 Mar 2026 08:49:49 +0000

A few weeks ago, I published a post here about adding voice AI to any website with one script tag. Today I'm sharing the business side of that story — because the technical win meant nothing without users.

The Before: 6 Months of Beautiful Failure

I built AnveVoice — a Voice OS for websites. One script tag. Agentic DOM actions (navigates, fills forms, clicks buttons). 53 languages. Sub-700ms latency.

Then I did what every blog told me to do: I went all in on SEO.

20,253 pages of content written
1,000+ monthly visitors from Google
$3,200/month infrastructure costs
0 signups. Zero.

Cost per signup: undefined (can't divide by zero).

The Pivot That Changed Everything

The product didn't change. The positioning did.

Before: "Voice OS for websites" — so broad that nobody saw themselves in it.

After: Three specific verticals with urgent deadlines:

Healthcare — WCAG 2.1 AA deadline April 24, 2026. Telemedicine platforms face legal exposure if patient intake forms aren't accessible.
Government — Same deadline. $55,000/day penalties for non-compliance.
International e-commerce — 53 languages as a competitive moat for global stores.

What Actually Drove the First 100+ Users

1. Multi-platform content blitz

Published the raw, honest failure story simultaneously on Dev.to, Indie Hackers, Medium, Hacker News, and LinkedIn. The vulnerability resonated — founders DM'd saying they'd been through the same thing.

2. Vertical positioning

Stopped saying "voice for everyone." Started saying "voice for healthcare sites facing the April 2026 WCAG deadline." Same product. Completely different conversion rate.

3. Cold outreach to deadline-driven buyers

When you email someone facing a compliance deadline with $55K/day penalties, the conversation is fundamentally different from cold outreach to someone who might find your product interesting.

4. "Powered by AnveVoice" badge

Every free tier widget shows this. Each user becomes a distribution channel. It compounds silently.

5. Directory submissions

Listed on 10+ directories (Product Hunt, BetaList, SaaSHub, AlternativeTo). Each listing is a permanent backlink and discovery channel.

The Numbers Today

Users:          100+ (up from 0)
Verticals:      3 (healthcare, government, e-commerce)
Languages:      53
Latency:        <700ms end-to-end
Free tier:      60 conversations/month
Growth plan:    $36/month

The Lesson

Your product is probably fine. Your positioning might be the problem.

Find people who need what you built urgently. Not people who think it's cool.

enthusiasm !== customers;
urgency === customers;

If you're building something and struggling with traction, ask yourself: Who has a deadline? Who faces a penalty without a solution? Who is actively shopping right now?

Those are your first 100 users.

AnveVoice is live at anvevoice.app. Free tier, no credit card. If you're in healthcare, government, or e-commerce and face accessibility deadlines — happy to help.

Building in public on X/Twitter.

I Added Voice AI to Any Website with One Script Tag

Adarsh Kant — Wed, 18 Mar 2026 08:36:49 +0000

What if you could add a voice AI assistant to any website with a single line of code?

That's what I built. One <script> tag. The user talks. The AI listens, understands, and takes real actions on the page — clicking buttons, filling forms, navigating pages.

Here's how it works under the hood.

The Problem

Most websites are built for mouse-and-keyboard users. But:

15-20% of the global population has some form of disability
Voice search is growing 35% year over year
WCAG 2.1 AA compliance is now legally required for government and healthcare sites (deadline: April 24, 2026)
Mobile users on the go need hands-free interaction

Traditional chatbots just answer questions. They don't do anything on the page. I wanted to build something that actually takes action.

The Architecture

AnveVoice has three core layers:

1. Speech-to-Text (STT)

We use a streaming STT pipeline that achieves sub-200ms first-token latency. The audio is captured via the Web Audio API:

// Simplified audio capture
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
const audioContext = new AudioContext({ sampleRate: 16000 });
const source = audioContext.createMediaStreamSource(stream);
// Stream chunks to STT service via WebSocket

We support 53 languages with automatic language detection. The system identifies the language within the first 500ms of audio.

2. Intent Resolution + DOM Mapping

This is the hard part. Once we have the transcribed text, we need to:

Understand intent: "I want to buy the blue shoes in size 10" maps to {action: "click", target: "product-variant-blue", then: "select-size-10", then: "add-to-cart"}
Map to DOM elements: We crawl the page's accessibility tree and semantic HTML to find matching elements
Execute actions: Click, scroll, fill form fields, navigate

// Simplified DOM action executor
async function executeVoiceAction(intent) {
  const { action, target, value } = intent;

  // Find the target element using multiple strategies
  const element = await findElement(target, [
    'aria-label',      // ARIA attributes first
    'data-testid',     // Test IDs
    'innerText',       // Visible text matching
    'semantic-role',   // HTML5 semantic roles
  ]);

  switch (action) {
    case 'click':
      element.click();
      break;
    case 'fill':
      element.value = value;
      element.dispatchEvent(new Event('input', { bubbles: true }));
      break;
    case 'navigate':
      window.location.href = element.href;
      break;
  }
}

3. Text-to-Speech (TTS) Response

After executing the action, the system confirms what it did via natural speech. We use streaming TTS for sub-300ms response time.

The total pipeline: STT (200ms) + Intent (100ms) + Action (50ms) + TTS (300ms) = under 700ms end-to-end.

The One-Tag Integration

Here's what the actual integration looks like:

<script src="https://app.anvevoice.app/widget.js"
        data-key="your-api-key">
</script>

That's it. The script:

Injects a floating voice button into the page
Handles microphone permissions
Streams audio to our STT service
Resolves intents against the current page's DOM
Executes actions and provides voice feedback

No server-side changes. No framework dependencies. Works with React, Vue, Angular, vanilla HTML, Shopify, WordPress — anything with a DOM.

What It Can Actually Do

Real examples from production:

E-commerce: "Show me red dresses under fifty dollars" → filters products, scrolls to results
Healthcare forms: "Fill in my date of birth, March 15, 1985" → finds the DOB field, enters the date
Government portals: "Navigate to the benefits application page" → clicks through menu navigation
Multi-language: A user says the same command in Hindi, Spanish, or Japanese — same result

The WCAG Compliance Angle

The April 24, 2026 WCAG 2.1 AA deadline affects:

Government sites serving 50,000+ people
Healthcare organizations receiving federal funding
Any site that wants to avoid accessibility lawsuits ($55K+/day penalties for government entities)

Voice interfaces aren't just nice to have anymore. They're becoming a legal requirement for accessible web experiences.

Performance Numbers

After 6 months of optimization:

Metric	Target	Actual
End-to-end latency	<1000ms	680ms avg
Language detection	<500ms	420ms
DOM action execution	<100ms	45ms
Languages supported	20+	53
Integration time	<5 min	~60 seconds

Try It

AnveVoice is live at anvevoice.app.

Free tier: 60 conversations/month
Growth: $36/month for 2,100 conversations
Scale: $120/month for high-volume sites

If you're working on accessibility, multilingual support, or just want to make your site more interactive — I'd love to hear what you think.

Drop a comment if you have questions about the architecture, the DOM mapping approach, or the STT/TTS pipeline. Happy to go deeper on any of these.

I'm Adarsh, solo founder building AnveVoice. Currently pivoting from horizontal positioning to three urgent verticals: healthcare, government, and international e-commerce. Building in public on Twitter/X.

We Built LinuxOS-AI: The First Step Toward an AI-Native Linux OS

Adarsh Kant — Wed, 02 Jul 2025 19:20:02 +0000

Hey folks 👋

I'm excited to share something we’ve been quietly working on — LinuxOS-AI, an AI-powered Linux terminal built on top of Google’s Gemini CLI.

It’s open-source. It’s safe by default. And it’s a glimpse of what a future AI-native operating system might feel like.

🧠 Why We Built This
Traditional terminals are powerful but rigid. You have to remember flags, read man pages, and always worry about breaking things.

We asked: what if you could just tell your Linux shell what you want in plain English — and it would do it safely and intelligently?

So we built LinuxOS-AI. A terminal where you can say:

🗣️ “Install Oracle DB”
🛡️ “Configure firewall to allow SSH only”
📁 “List all Python files over 1MB”

🔧 What Makes It Different
✅ Natural Language System Admin (powered by Gemini CLI)
✅ Dry-run & sudo confirmation for safety
✅ Built-in agents for Shell, Filesystem, and Firewall tasks
✅ Reskinned UX for clarity + extensibility
✅ Fully open source and customizable

This is just v0.1.0 — but we believe it’s the starting point for something big.

🌐 Try It / Support It
🔗 GitHub: github.com/ANVEAI/linuxos-ai

🚀 Product Hunt launch: producthunt.com/products/linuxos-ai

We’d love your feedback, feature ideas, or even just a GitHub ⭐️ if you like where this is going.

🧩 What’s Next?
We're exploring:

Built-in package manager hooks

AI-powered cron/scheduling

Plugin support (think: agents.d)

Voice module (in alpha 👀)

If you’ve ever wished your terminal understood you better, we’d love to hear from you.

💬 What’s one thing you’d want your terminal to do if it was truly intelligent?
Drop a comment — let’s reimagine the shell together.

– Adarsh Kant
Founder, ANVE.AI
LinuxOS-AI Maintainer