Forem: Imisioluwa Elijah

Vertalk — Real-Time AI Call Agent Powered by Redis 8

Imisioluwa Elijah — Mon, 11 Aug 2025 04:39:10 +0000

This is a submission for the Redis AI Challenge: Beyond the Cache.

What I Built

Vertalk is a real-time AI voice call assistant built to showcase what is possible when Redis 8 is used as the primary data and event backbone, not only a cache. The platform integrates RedisJSON, RediSearch (text and vector with HNSW), Streams, and Pub/Sub to power low‑latency, multi‑tenant AI operations that ingest knowledge, search semantically, stream live events to the UI, and adapt the assistant’s behavior per Knowledge base vertical in real time.

Demo

Live Demo:

vertalk.vercel.app

Backup: https://vertalk-n8bypytee-imisi-dahunsis-projects.vercel.app/

Demo Video:

Repo:

ImisiDahunsi4 / vertalk

This is a Next.js project bootstrapped with create-next-app.

Getting Started

First, run the development server:

npm run dev
# or
yarn dev
# or
pnpm dev
# or
bun dev

Open http://localhost:3000 with your browser to see the result.

You can start editing the page by modifying pages/index.tsx. The page auto-updates as you edit the file.

API routes can be accessed on http://localhost:3000/api/hello. This endpoint can be edited in pages/api/hello.ts.

The pages/api directory is mapped to /api/*. Files in this directory are treated as API routes instead of React pages.

This project uses next/font to automatically optimize and load Inter, a custom Google Font.

Redis Integration Overview

Docs: See docs/redis-report.md for architecture, data models, routes, and evaluation.
Modules used: RedisJSON, RediSearch (full-text + vector/HNSW), Streams, Pub/Sub.
Primary keys: company:<id> (JSON), kb:<companyId>:... (HASH+vector), call:<id> (JSON), ticket:<id> (JSON), show:<slug> (JSON).
Real-time: Pub/Sub channels…

View on GitHub

How I Used Redis 8

Redis as the Primary System of Record

Vertalk centralizes data in Redis. Knowledge base configuration is stored via RedisJSON and drives the AI assistant’s behavior and tone. The default and active Knowledge-base are coordinated through simple keys while the full JSON document captures brand voice, vertical, and enabled functions. The following excerpt from pages/api/admin/company.ts shows how Vertalk persists knowledge base configuration and switches the active tenant instantly:

// pages/api/admin/company.ts (excerpt)
await client.sendCommand(["JSON.SET", `company:${id}`, "$", JSON.stringify(companyDoc)]);
if (makeActive) {
  await client.set("currentCompany", id);
}

Low‑Latency Redis Client and Subscriber

A robust Redis client layer underpins the app. A singleton is created for the main client and another is duplicated for a dedicated subscriber. This pattern minimizes overhead and supports server‑sent events (SSE) without reconnect churn. The following snippet from lib/redis.ts illustrates the approach.

// lib/redis.ts (excerpt)
export async function getRedis(): Promise<RedisClientType> {
  if (!g.__redisClient) {
    const client = createClient({ url: process.env.REDIS_URL });
    client.on("error", (err) => console.error("Redis Client Error", err));
    await client.connect();
    g.__redisClient = client;
  }
  return g.__redisClient as RedisClientType;
}

export async function getRedisSubscriber(): Promise<RedisClientType> {
  if (!g.__redisSubscriber) {
    const client = await getRedis();
    const subscriber = client.duplicate();
    subscriber.on("error", (err) => console.error("Redis Subscriber Error", err));
    await subscriber.connect();
    g.__redisSubscriber = subscriber;
  }
  return g.__redisSubscriber as RedisClientType;
}

Real‑Time Streaming with Pub/Sub, SSE, and Streams

Vertalk uses Redis Pub/Sub to push live events to the UI over SSE, and Redis Streams to persist the same events for history and replay. The SSE endpoint bridges a Redis subscription to the browser through an EventSource. The following excerpt from pages/api/rt/subscribe.ts shows the bridge implementation:

// pages/api/rt/subscribe.ts (excerpt)
const subscriber = await getRedisSubscriber();
const channel = `ch:call:${callId}`;

res.setHeader("Content-Type", "text/event-stream");
res.write(`: connected to ${channel}\n\n`);

const onMessage = (message: string) => {
  res.write(`data: ${message}\n\n`);
};

await subscriber.subscribe(channel, onMessage);

To publish operational or test events, the platform exposes a publish endpoint that writes to both a Redis Stream and Pub/Sub channel in one roundtrip using a pipeline. This ensures live updates and durable history with minimal latency. The following code is from pages/api/rt/publish.ts:

// pages/api/rt/publish.ts (excerpt)
const streamKey = `stream:call:${callId}`;
const channel = `ch:call:${callId}`;
const eventStr = JSON.stringify(event);

const pipe = client.multi();
pipe.xAdd(streamKey, "*", { event: eventStr });
pipe.publish(channel, eventStr);
await pipe.exec();

On the client side, the admin UI subscribes to ingestion progress using the SSE bridge. The call ID convention doubles as a logical session name, keeping the pattern consistent across features.

// pages/admin/index.tsx (excerpt)
const es = new EventSource(`/api/rt/subscribe?callId=${encodeURIComponent(`ingest:${company.id}`)}`);

This dual-write pattern, using both Streams and Pub/Sub, provides live updates to the browser and a durable event log for audit or UX replay while keeping latency ultra low.

Vector Search and Knowledge Ingestion

Vertalk implements a vector search stack using RediSearch HNSW, coupled with a dependency‑free embedding stub that can be swapped for a production provider. Knowledge text is chunked, embedded, and stored as Redis Hashes with ID tags and a vector field. The following snippets from lib/kb.ts create the index and ingest chunks.

// lib/kb.ts (excerpt)
await client.sendCommand([
  "FT.CREATE", "idx:kb", "ON", "HASH", "PREFIX", "1", "kb:", "SCHEMA",
  "companyId", "TAG",
  "title", "TEXT",
  "chunk", "TEXT",
  "vector", "VECTOR", "HNSW", "6",
  "TYPE", "FLOAT32", "DIM", EMBEDDING_DIM.toString(), "DISTANCE_METRIC", "COSINE",
]);

// lib/kb.ts (excerpt)
const docId = `kb:${companyId}:${Date.now()}:${Math.random().toString(36).slice(2, 8)}`;
await client.hSet(docId, {
  companyId,
  title: input.title,
  chunk,
  source: input.source || "manual",
  createdAt: Date.now().toString(),
  vector: buf as unknown as any,
} as any);

Ingestion broadcasts real‑time progress via Pub/Sub and mirrors events into Redis Streams for history, using the same call ID convention. The following excerpt shows both operations:

// lib/kb.ts (excerpt)
const evt = { type: "ingest-progress", companyId, processed, total, t: Date.now() };
await client.publish(`ch:call:ingest:${companyId}`, JSON.stringify(evt));
await client.sendCommand([
  "XADD", `stream:call:ingest:${companyId}`, "*", "event", JSON.stringify(evt),
]);

A terminal event marks completion and is likewise published to both channels.

Vector and Text Search APIs

The knowledge base search endpoint supports HNSW vector KNN or classic text search in a single API, parameterized by company and query mode. This demonstrates Redis handling both semantic and lexical search with a uniform operational surface. The following snippet from pages/api/kb/search.ts shows vector KNN using a bound parameter for the query vector.

// pages/api/kb/search.ts (vector excerpt)
const embedding = await embedText(q || "");
const buf = float32ToBuffer(embedding);
const query = `(@companyId:{${companyId}})=>[KNN ${k} @vector $B]`;
const reply = await client.sendCommand([
  "FT.SEARCH", "idx:kb", query,
  "PARAMS", "2", "B", buf as any,
  "RETURN", "3", "title", "chunk", "__vector_score",
  "SORTBY", "__vector_score", "ASC",
  "DIALECT", "2",
]);

For text mode, the same endpoint sanitizes input and runs a RediSearch text query scoped to the tenant, ensuring safe, scoped lexical matching.

// pages/api/kb/search.ts (text excerpt)
const safe = sanitizeSearchQuery(q || "");
const query = `(@companyId:{${companyId}}) ${safe || "*"}`;
const reply = await client.sendCommand([
  "FT.SEARCH", "idx:kb", query,
  "RETURN", "2", "title", "chunk",
  "LIMIT", "0", String(k),
]);

Embedding Stub: Production‑Ready Interface, Demo‑Ready Defaults

To keep the app fully runnable without external APIs, Vertalk ships a simple embedding stub that creates normalized bag‑of‑words vectors. The interface, namely the fixed dimension and buffer conversion, matches production providers so a real embedding backend can be dropped in without any changes to persistence or query logic. The following excerpt is from lib/embedding.ts:

// lib/embedding.ts (excerpt)
export const EMBEDDING_DIM = 256;

export async function embedText(text: string): Promise<Float32Array> {
  const tokens = text.toLowerCase().replace(/[^a-z0-9\s]/g, " ").split(/\s+/).filter(Boolean);
  const vec = new Float32Array(EMBEDDING_DIM);
  for (const t of tokens) vec[hashStr(t) % EMBEDDING_DIM] += 1.0;
  // L2 normalize...
  return vec;
}

export function float32ToBuffer(vec: Float32Array): Buffer {
  return Buffer.from(vec.buffer);
}

Full‑Text Search Over JSON:

Beyond knowledge search, Vertalk uses RedisJSON and RediSearch to model entities. The seed endpoint creates a JSON index and inserts demo content, enabling fast facet and numeric sorts.

// pages/api/redis/seed.ts (excerpt)
await client.sendCommand([
  "FT.CREATE", "idx:shows", "ON", "JSON", "PREFIX", "1", "show:", "SCHEMA",
  "$.title", "AS", "title", "TEXT",
  "$.description", "AS", "description", "TEXT",
  "$.theatre", "AS", "theatre", "TEXT",
  "$.venue", "AS", "venue", "TEXT",
  "$.price", "AS", "price", "NUMERIC", "SORTABLE",
  "$.date", "AS", "date", "NUMERIC", "SORTABLE",
]);

End‑to‑End Real‑Time UX

Once a Demo company data is saved in the admin UI, you can ingest knowledge chunks. The UI opens an EventSource to /api/rt/subscribe, receives ingest-progress and ingest-complete events, and displays updates live. You can immediately search the knowledge base using either vector or text mode, demonstrating that the same Redis index can power multiple retrieval paradigms with strict tenant scoping.

How to Run the Demo

Set REDIS_URL in your environment using .env or your shell. Start the Next.js app with your preferred package manager. Visit /admin to save a and add knowledge base, and start ingestion to see live SSE updates. Use the Test Knowledge Search panel to run vector or text queries. Optionally, seed data and test RediSearch over JSON records.

# Install deps and run (example with pnpm)
pnpm install
pnpm dev

You can seed demo show data using the seed endpoint. This creates the RediSearch JSON index and inserts the demo documents.

curl -X POST http://localhost:3000/api/redis/seed

You can publish a test event to a call channel using the publish endpoint. This writes to both the Stream and the Pub/Sub channel and is immediately visible in any open SSE subscribers.

curl -X POST http://localhost:3000/api/rt/publish \
  -H "Content-Type: application/json" \
  -d '{"callId":"ingest:default","event":{"type":"test","t":'"$(date +%s)"'}}'

You can test vector search over the knowledge base using the KB search endpoint. The type parameter can be vector or text, and the query is scoped by companyId.

curl "http://localhost:3000/api/kb/search?companyId=default&type=vector&q=booking%20policy"

Conclusion

Vertalk demonstrates Redis 8 as a multi‑model platform for real‑time AI. RedisJSON stores configuration and content, RediSearch powers both full‑text and vector search with HNSW, Streams provide durable event history, and Pub/Sub delivers instantaneous UI updates. With multi‑tenant isolation, live ingestion, and semantic search all running through Redis, Vertalk goes far beyond caching and offers a unified runtime for modern AI agents.

SwiftPad: AI Powered Transcription with AssemblyAI

Imisioluwa Elijah — Mon, 28 Jul 2025 05:47:15 +0000

This is a submission for the AssemblyAI Voice Agents Challenge

What I Built

Inspired by the need for effortless audio transcription and content creation, I built Swift-Pad, a powerful audio transcription and transformation tool leveraging AssemblyAI's Universal-Streaming technology.

Swift-Pad allows users to easily record or upload audio, transcribe it with remarkable accuracy, and instantly transform those transcriptions into summaries, emails, blogs, quick notes, and more. The goal was simple: remove the hassle from turning audio conversations into actionable content.

This submission addresses the Business Automation Voice Agent prompt with:

Automated Audio Transcription: Real-time, high-accuracy speech-to-text transcription for business meetings, calls, and interviews.
AI-driven Content Transformation: Converts transcriptions into summaries, emails, blog posts, lists, and notes instantly.
Speaker Diarization and Highlight Extraction: Automatically identifies speakers and key moments, enhancing readability and usability.
Seamless File Management: Securely manages audio files with automatic storage and retrieval through Supabase integration.
Scalable and Secure User Authentication: Provides robust user authentication and data security through Clerk and Prisma ORM integration.
Usage Tracking & Rate Limiting: Ensures fair and controlled usage through Upstash Redis integration.

Core Problem Addressed

Professionals, creators, and businesses frequently face the time-consuming task of manually transcribing audio recordings and converting conversations into actionable formats. Swift-Pad automates this workflow, reducing hours of manual effort into minutes, allowing users to quickly capture, manage, and leverage valuable insights from audio content.

Demo

Live Demo: https://swift-pad.vercel.app/

Screenshots:

All Notes Page

Recording & Uploading Interface

Transcription Display & Transformation Options

Generated Content from Transcription

GitHub Repository

ImisiDahunsi4 / swift-pad

Swift Pad

Swift Pad is a web application for audio transcription and transformation. Record or upload audio files, get accurate transcriptions, and transform them into various formats like summaries, blog posts, emails, and more.

Features

🎙️ Audio Recording: Record audio directly in your browser
📤 File Upload: Upload existing audio files (MP3, WAV, M4A, etc.)
🔤 Accurate Transcription: Powered by AssemblyAI's state-of-the-art speech recognition
🌐 Multi-language Support: Transcribe audio in multiple languages
🔄 Text Transformations: Convert transcriptions into summaries, blog posts, emails, and more
👥 Speaker Diarization: Identify different speakers in conversations
📊 Auto Highlights: Automatically extract key moments from transcriptions
🔒 Secure Storage: Files stored securely with Supabase Storage

Technology Stack

Frontend: Next.js 15, React 19, TailwindCSS
Backend: Next.js API routes, tRPC
Database: PostgreSQL via Prisma ORM
Authentication: Clerk Auth
Storage: Supabase Storage
AI Services
- …

View on GitHub

Technical Implementation & AssemblyAI Integration

Here's your summarized Technical Implementation & AssemblyAI Integration section styled similarly to the provided example:

Technical Implementation & AssemblyAI Integration

1. Flexible Dual-SDK Architecture

Swift Pad uses a dual-SDK architecture to maximize flexibility with AssemblyAI integration, enabling both direct SDK access and AI SDK compatibility:

// assemblyaiClient.ts
import { createAssemblyAI } from '@ai-sdk/assemblyai';
import { AssemblyAI } from 'assemblyai';

// AI SDK Integration
export const assemblyaiClient = createAssemblyAI({ apiKey });

// Direct AssemblyAI SDK
export function getAssemblyAIClient(apiKey?: string) {
  return new AssemblyAI({ apiKey: apiKey || defaultApiKey });
}

This setup supports advanced features, user-provided keys (BYOK), and easy integration with multiple AI providers.

2. Robust Asynchronous Transcription with Intelligent Polling

To handle AssemblyAI’s asynchronous transcription efficiently, Swift Pad implements intelligent polling with error handling and timeout protection:

// whisper.ts
async function pollForCompletion(transcriptId: string) {
  let attempts = 0;
  while (attempts < 200) {
    const transcript = await client.transcripts.get(transcriptId);
    if (transcript.status === 'completed') return transcript;
    if (transcript.status === 'error') throw new Error(transcript.error);
    await new Promise(res => setTimeout(res, 3000));
    attempts++;
  }
  throw new Error('Transcription timeout');
}

This ensures reliable completion tracking and user-friendly error messaging.

3. Enhanced Audio Processing Features

Swift Pad leverages AssemblyAI’s advanced audio processing capabilities through careful parameter configuration:

// whisper.ts
const params = {
  audio_url: input.audioUrl,
  language_code: input.language || "en",
  speaker_labels: true,
  auto_highlights: true,
};

This enables speaker identification, key moment extraction, and multilingual transcription.

4. Optimized Browser-based Audio Recording

High-quality audio recording in Swift Pad captures audio directly in-browser optimized for AssemblyAI processing:

// useAudioRecording.ts
const mediaRecorder = new MediaRecorder(stream, { mimeType: "audio/webm" });
mediaRecorder.ondataavailable = e => chunks.push(e.data);
mediaRecorder.onstop = () => setAudioBlob(new Blob(chunks));

This setup provides optimal compatibility and user-friendly controls.

5. Multi-tier Storage & Fallback Strategy

Swift Pad employs a robust storage upload mechanism with multiple fallback methods ensuring reliability:

// useSupabaseUpload.ts
try {
  return await uploadFileToS3(file);
} catch {
  try {
    return await uploadViaServerAPI(file);
  } catch {
    return await supabase.storage.from(bucket).upload(path, file);
  }
}

This multi-tier approach ensures uploads succeed under various conditions.

6. End-to-End Workflow Integration

Swift Pad seamlessly integrates recording, storage, transcription, and display into a unified workflow:

// RecordingModal.tsx
const url = await uploadToSupabase(audioBlob);
const result = await transcribeFromS3.mutateAsync({ audioUrl: url });
router.push(`/whispers/${result.id}`);

Users experience effortless audio-to-text transformations from start to finish.

7. Integration Architecture Overview

Complete AssemblyAI integration workflow:

Recording/Upload → Supabase Storage → AssemblyAI Transcription
      ↓                                       ↓
 User Interface ← Database (Prisma) ← Google Gemini (Title & Transformation)

This architecture ensures clarity, scalability, and optimal user experience.