Forem: Iftekhairul Alam

Connecting AI Voice Agents to SIP & PSTN Using NextGenSwitch

Iftekhairul Alam — Fri, 06 Feb 2026 05:14:52 +0000

Bridging AI Voice Agents with Real Phone Calls

Building an AI voice agent is relatively easy today.

Connecting that agent to real phone calls (SIP, PBX, PSTN) is not.

Most AI voice systems are designed to work with WebSockets and raw audio streams, while production telephony still relies on SIP, RTP, and PSTN infrastructure. This mismatch is where many voice-AI projects struggle to move beyond demos.

This post explains how NextGenSwitch bridges that gap—allowing any AI voice system to interact with real phone callers using a Twilio-style streaming interface, without exposing SIP or RTP complexity to AI developers.

🔗 Original post:

https://nextgenswitch.com/blog/connecting-ai-voice-agents-to-sip-pstn-using-nextgenswitch/

The Core Problem

AI voice systems typically expect:


WebSocket → PCM audio → AI pipeline → PCM audio

Telephony systems operate very differently:


PSTN → SIP Trunk → PBX → RTP (μ-law / A-law)

Key challenges include:

SIP and RTP are stateful and codec-sensitive
AI systems expect clean, ordered audio frames
Handling barge-in, latency, and scaling is non-trivial
Most AI frameworks are not PBX-aware

The Role of NextGenSwitch

NextGenSwitch acts as a telephony abstraction layer between traditional phone systems and modern AI services.

It provides:

SIP & PSTN termination
Integration with PBX systems (Asterisk / FreeSWITCH)
A Twilio-style Programmable Voice API
Real-time WebSocket audio streaming
Codec and sample-rate normalization

Your AI service never has to interact directly with SIP or RTP.

High-Level Architecture


Caller
|
[PSTN / SIP Trunk]
|
[Asterisk / FreeSWITCH]
|
[NextGenSwitch]
| <WebSocket Audio Stream>
|
[AI Voice Service]

The AI voice service can be:

A custom WebSocket server
A cloud-based AI endpoint
An on-prem STT + LLM + TTS stack
Any framework capable of handling real-time audio

Twilio-Style XML Call Control

When a call reaches NextGenSwitch, it fetches XML instructions—similar to Twilio’s TwiML.

Minimal XML (only the stream URL is required)

xml <Response> <Connect> <Stream url="wss://ai.yourdomain.com/ws/voice-agent"/> </Connect> </Response>`

This instruction:

Answers the call
Opens a bidirectional WebSocket
Starts real-time audio streaming

Optional Parameters (Examples Only)

Parameters are not mandatory.
They are passed as metadata to your AI service.

`xml <Response> <Connect> <Stream url="wss://ai.yourdomain.com/ws/voice-agent"> <Parameter name="agent" value="support-bot"/> <Parameter name="tenant_id" value="company-01"/> <Parameter name="language" value="en-US"/> </Stream> </Connect> </Response> `

These values appear in the JSON start event and can be used for routing, prompts, or CRM lookups.

WebSocket Streaming Protocol (JSON)

NextGenSwitch uses a Twilio Media Streams–style JSON protocol.

Your AI service only needs to handle a small set of events.

`start` Event

Sent once when the stream begins.

`json { "event": "start", "streamId": "NGS_STREAM_123456", "start": { "callId": "NGS_CALL_abc", "from": "+8801XXXXXXXXX", "to": "5000", "customParameters": { "agent": "support-bot", "tenant_id": "company-01" } } } `

Save the streamId—it must be included in outbound audio messages.

`media` Event (Inbound Audio)

`json { "event": "media", "streamId": "NGS_STREAM_123456", "media": { "payload": "BASE64_AUDIO_BYTES==" } } `

Audio characteristics:

Codec: G.711 μ-law
Sample rate: 8 kHz
Payload: base64-encoded audio frames

`media` Event (Outbound Audio)

Your AI service responds using the same structure:

`json { "event": "media", "streamId": "NGS_STREAM_123456", "media": { "payload": "BASE64_AUDIO_BYTES==" } } `

NextGenSwitch converts this audio back into telephony format and sends it to the caller.

`stop` Event

`json { "event": "stop", "streamId": "NGS_STREAM_123456", "stop": { "reason": "hangup" } } `

AI Stack: Fully Flexible

NextGenSwitch does not require any specific AI framework.

You can use:

Any STT engine
Any LLM
Any TTS engine
Any programming language

Frameworks like Pipecat can be used as reference implementations, but they are not required.

Why This Architecture Works

No SIP or RTP handling in AI code
Twilio-style, developer-friendly interface
Real-time, low-latency audio streaming
Vendor-neutral AI integration
Production-ready PSTN scalability

Common Use Cases

AI receptionist
AI call center agents
Voice-based order processing
Appointment booking
IVR replacement
Multilingual voice bots

Key Takeaways

Only the <Stream url> is mandatory
XML parameters are optional metadata
Streaming protocol is Twilio-style JSON
Telephony audio uses μ-law @ 8kHz
AI logic is completely decoupled from PBX logic

Learn More

Programmable Voice Stream API
https://nextgenswitch.com/docs/programmable-voice-api/#stream
AI streaming examples
https://github.com/nextgenswitch/ai_agents

NextGenSwitch: Build AI-Powered Telephony (Features, Examples, and a Developer Quickstart)

Iftekhairul Alam — Wed, 29 Oct 2025 10:33:17 +0000

If you’ve ever hacked together SIP trunks, WebRTC agents, IVRs, recordings—and then tried to bolt on an AI receptionist—you know the pain. NextGenSwitch (NGS) is a modern softswitch with programmable voice and first-class AI hooks that makes this stack delightfully straightforward.
Repo: github.com/nextgenswitch/nextgenswitch

This post gives you:

A whirlwind tour of features and architecture
Exactly how AI integrations plug in (LLMs, realtime voice, function-calling)
A copy-paste Quickstart (outbound calls, XML verbs, status callbacks, live call modify)
Dev procedures and a production checklist

TL;DR: NGS offers Twilio-style call control with softswitch power under the hood—plus clean integration points for LLMs and your backend tools.

What is NextGenSwitch?

NextGenSwitch (NGS) is a cloud softswitch + programmable voice layer:

SIP + WebRTC: bridge PSTN/SIP customers to browser-based agents.
Programmable Calls with XML/JSON-style responses from your webhooks.
IVR, Gather, Queueing, Recording built-in.
Events & CDRs for observability.
AI-native: stream audio to a realtime LLM, or do turn-by-turn ASR ? LLM ? TTS; call your own tools (helpdesk, CRM, calendar, etc.).

Feature Highlights

Outbound API: originate calls and drive the flow with hosted XML or inline responseXml.
Inbound Webhooks: answer URL returns NGS verbs like <Say>, <Play>, <Gather>, <Dial>, <Record>.
Status Callbacks: get DIALING, RINGING, ESTABLISHED, DISCONNECTED, and more.
Modify-Live: push new instructions mid-call (e.g., speak a message, transfer, or bridge).
Recording + Transcription: trigger, store, and process (e.g., summarize with LLMs).
Queues & Skills: route by priority, business hours, or sticky agent.

Architecture at a Glance

Caller (PSTN/SIP)
        ¦
        ?
   NGS Ingress --? Your Webhook
        ¦              ¦
        ¦              +-? LLM (realtime or text)
        ¦              +-? Tools (tickets, CRM, calendar)
        ¦              +-? Business logic (IVR/queues/policies)
        ?
   Media Engine (SIP/WebRTC, TTS/ASR)
        ¦
        +-? Agent Browser (WebRTC)
        +-? Upstream SIP / PSTN

You keep your business logic server-side; NGS handles signaling, media, and scale.

AI Integrations (How it actually fits)

You’ve got two common patterns:

Turn-based
NGS records/streams ? your server gets transcript ? you ask an LLM what to do ? return NGS actions (<Say>, <Gather>, enqueue, etc.).
Realtime voice
Full-duplex audio between caller and LLM (e.g., OpenAI Realtime, Gemini Live). The LLM can produce audio or directives; your webhook can still inject actions (transfer, record, ticketing).

Function-calling / Tools: define JSON schemas like create_support_ticket, schedule_appointment, lookup_customer. When the LLM calls a tool, you do the API call, then continue the conversation with the caller.

Developer Quickstart (Copy-Paste)

Replace YOUR_NGS_BASE_URL, YOUR_AUTH_ID, YOUR_AUTH_SECRET and public URLs where applicable.

1) Outbound Call — Hosted XML (via `response`) or Inline (via `responseXml`)

Option A: hosted XML

curl -X POST \
  -H "X-Authorization: YOUR_AUTH_ID" \
  -H "X-Authorization-Secre: YOUR_AUTH_SECRET" \
  -d "to=1002" \
  -d "from=1001" \
  -d "statusCallback=https://example.com/voice/status" \
  -d "response=https://example.com/voice/hello.xml" \
  https://YOUR_NGS_BASE_URL/api/v1/call

Option B: inline XML

curl -X POST \
  -H "X-Authorization: YOUR_AUTH_ID" \
  -H "X-Authorization-Secre: YOUR_AUTH_SECRET" \
  --data-urlencode 'to=1002' \
  --data-urlencode 'from=1001' \
  --data-urlencode 'statusCallback=https://example.com/voice/status' \
  --data-urlencode 'responseXml=<?xml version="1.0" encoding="UTF-8"?><Response><Say>Hello from inline XML</Say></Response>' \
  https://YOUR_NGS_BASE_URL/api/v1/call

Minimal XML you can host at https://example.com/voice/hello.xml:

<?xml version="1.0" encoding="UTF-8"?>
<Response>
  <Say>Hello, world! This call is powered by NextGenSwitch.</Say>
</Response>

2) Status Callbacks (Receive live events)

Node/Express

import express from "express";
const app = express();
app.use(express.urlencoded({ extended: false })); // NGS posts form-encoded

app.post("/voice/status", (req, res) => {
  console.log("NGS status:", req.body); // e.g., { call_id, status, timestamp, ... }
  res.sendStatus(200);                   // respond fast; do heavy work async
});

app.listen(3000, () => console.log("Status callback on :3000"));

Sample payload (form-encoded):

call_id=abc123
status=RINGING
timestamp=2025-10-29T08:00:00Z

3) Modify a Live Call (push new instructions)

curl -X PUT \
  -H "X-Authorization: YOUR_AUTH_ID" \
  -H "X-Authorization-Secre: YOUR_AUTH_SECRET" \
  --data-urlencode 'responseXml=<Response><Pause length="2"/><Say>Call has been modified</Say><Dial>1000</Dial></Response>' \
  https://YOUR_NGS_BASE_URL/api/v1/call/REPLACE_WITH_CALL_ID

4) Multi-language Examples (Node, Python, PHP)

Node (fetch)

import fetch from "node-fetch"; // Node <18; otherwise use global fetch
const base = "https://YOUR_NGS_BASE_URL";

const res = await fetch(`${base}/api/v1/call`, {
  method: "POST",
  headers: {
    "Content-Type": "application/x-www-form-urlencoded",
    "X-Authorization": "YOUR_AUTH_ID",
    "X-Authorization-Secre": "YOUR_AUTH_SECRET"
  },
  body: new URLSearchParams({
    to: "1002",
    from: "1001",
    statusCallback: "https://example.com/voice/status",
    responseXml: `<?xml version="1.0"?><Response><Say>Hello from Node</Say></Response>`
  })
});

console.log(await res.text());

Python (requests)

import requests

base = "https://YOUR_NGS_BASE_URL"
headers = {
    "X-Authorization": "YOUR_AUTH_ID",
    "X-Authorization-Secre": "YOUR_AUTH_SECRET",
}
data = {
    "to": "1002",
    "from": "1001",
    "statusCallback": "https://example.com/voice/status",
    "responseXml": '<?xml version="1.0"?><Response><Say>Hello from Python</Say></Response>'
}

r = requests.post(f"{base}/api/v1/call", headers=headers, data=data)
print(r.text)

PHP (Guzzle)

<?php
require 'vendor/autoload.php';
use GuzzleHttp\Client;

$client = new Client(['base_uri' => 'https://YOUR_NGS_BASE_URL']);
$response = $client->post('/api/v1/call', [
  'headers' => [
    'X-Authorization' => 'YOUR_AUTH_ID',
    'X-Authorization-Secre' => 'YOUR_AUTH_SECRET'
  ],
  'form_params' => [
    'to' => '1002',
    'from' => '1001',
    'statusCallback' => 'https://example.com/voice/status',
    'responseXml' => '<?xml version="1.0"?><Response><Say>Hello from PHP</Say></Response>'
  ]
]);

echo $response->getBody();

XML Verb Cheatsheet (NGS Actions)

<Say>: TTS

  <Response><Say loop="2">This message repeats twice.</Say></Response>

<Play>: play a media file

  <Response><Play loop="3">https://example.com/audio/connecting.mp3</Play></Response>

<Gather>: collect DTMF and POST to your action URL

  <Response>
    <Gather action="https://example.com/process_input" method="POST" maxDigits="4" timeout="10">
      <Say>Please enter your 4-digit PIN.</Say>
    </Gather>
  </Response>

<Dial>: bridge to a number/endpoint

  <Response>
    <Dial to="+1234567890" answerOnBridge="true" record="record-from-answer">
      <Play>https://example.com/audio/connecting.mp3</Play>
    </Dial>
  </Response>

<Record>: start a recording

  <Response>
    <Record action="https://example.com/handle_recording" method="POST" timeout="5" finishOnKey="#" beep="true"/>
  </Response>

Other useful controls: <Hangup>, <Pause>, <Redirect>, <Bridge>, <Leave>.

Development Procedures (What to actually do)

Create envs & keys

NGS_AUTH_ID, NGS_AUTH_SECRET, WEBHOOK_SECRET (if verifying signatures)
LLM keys, Helpdesk/CRM tokens

Expose your server

ngrok http 3000 (or Cloudflare Tunnel) for /voice/answer, /voice/status, /voice/ai

Provision routing

Buy/assign a number ? set Answer URL (returns XML) and Status Callback URL
Add queues (sales/support) with skills/hours if needed

Logging & Idempotency

Treat status webhooks as retryable; de-dupe by event_id if provided
Correlate call_id across logs, DB, and tickets

Security

Verify webhook signatures/timestamps
Encrypt recordings; redact sensitive entities from transcripts
Rotate secrets, enforce MFA

Testing

Unit test your XML builders
Simulate DTMF and edge flows (timeouts, invalid digits)
Load test queue wait, ASR ? LLM latency

Troubleshooting (Quick)

401/403 ? Check X-Authorization / secret headers (your deployment may use X-Authorization-Secre).
400 ? XML missing/invalid; start with <Response><Say>…</Say></Response>.
“Rings but no audio” ? If using hosted response, ensure your XML URL is public & returns valid XML with correct Content-Type.
No status events ? Verify public statusCallback URL; log incoming requests; keep handler fast.
Live modify does nothing ? Confirm call_id and that your PUT payload contains a valid responseXml.

Links & Credits

Website: nextgenswitch.com
GitHub: github.com/nextgenswitch/nextgenswitch

Wrap-Up

With NextGenSwitch, you write simple webhooks and XML while the platform handles media and scale. Add AI for triage, summarization, or full conversational agents, and route to humans when it matters. The examples above are ready to paste—use them as a base, then layer in your product logic.

If you ship something with this, share a link—I’d love to see your call flows, response times, and UI patterns!