Twilio and Deepgram TTS

This guide walks through real-time text-to-speech on a phone call: a caller dials a Twilio number and immediately hears a message that your server generates and streams into the call as it’s spoken. The server never listens — this is the speaking half of a voice pipeline, in isolation.

The architecture is small. Twilio opens a bidirectional audio stream to your server; your server turns text into speech with Deepgram Flux TTS and streams the resulting audio back into the call. By the end you have a working “speak into a call” service you can dial from any phone, and a clear view of how to push generated audio into a live call in real time.

How it works

Playback is a one-directional flow in the opposite direction from transcription: text goes in, audio comes out and into the caller’s ear. Your server generates the message with Deepgram and streams it back to Twilio as media frames. It ignores whatever the caller says.

The implementation is a single WebSocket handler per call. When the call connects, it opens a Flux TTS socket, sends the message, and streams the audio it gets back to Twilio in real-time-paced 20 ms frames. No audio path runs from the caller into your logic, and that absence keeps this simpler than a full agent.

Which Twilio and Deepgram integration do you need?

This guide builds playback only — text in, speech out, no listening. If you need more of the pipeline, two companion guides cover those cases.

If you only need to say something — IVR prompts, notifications, announcements, dynamic confirmations — this guide is the whole job. Reach for the Voice Agent API when you also need to listen and respond.

Before you begin

You need the following accounts, keys, and tools.

• A Twilio account with a voice-capable phone number.
• A Deepgram API key.
• Python 3.10 or later.
• A tunneling tool to expose your local server to Twilio. This guide uses ngrok.

Install ngrok and authenticate it once with the token from your ngrok dashboard.

$
brew install ngrok
$
ngrok config add-authtoken YOUR_NGROK_AUTHTOKEN

Step 1: Set up the project

Clone the companion repository, which holds the complete app.py and requirements.txt referenced throughout this guide.

$
git clone https://github.com/deepgram-devs/twilio-tts.git
$
cd twilio-tts

Install the dependencies and set up configuration. The dependency list is short: fastapi, uvicorn, deepgram-sdk, python-dotenv, and twilio.

$
pip install -r requirements.txt
$
cp .env.example .env

The .env file holds two values your application reads on startup, plus two optional security values covered later.

$
DEEPGRAM_API_KEY=YOUR_DEEPGRAM_API_KEY
$
PUBLIC_HOSTNAME=your-host.ngrok-free.app   # the public host Twilio reaches, no scheme

This app needs no LLM key and no speech-to-text — it only generates speech. Text-to-speech runs through the official Deepgram Python SDK (deepgram-sdk, version 7.6.0 or later).

Verify: running python -c "import app" with the two environment variables set imports cleanly.

Step 2: Serve TwiML with <Connect><Stream>

When a call connects, Twilio asks your webhook what to do. You answer with TwiML, Twilio’s XML instruction set. The key instruction here is <Connect><Stream>, which opens a bidirectional WebSocket — that return path lets you send generated audio back into the call.

1
@app.post("/twiml")
2
async def twiml(request: Request) -> Response:
3
    xml = f"""<?xml version="1.0" encoding="UTF-8"?>
4
<Response>
5
  <Connect>
6
    <Stream url="wss://{PUBLIC_HOSTNAME}/media" />
7
  </Connect>
8
</Response>"""
9
    return Response(content=xml, media_type="application/xml")

That one design choice separates playback from transcription. Playing audio into a call requires a channel that carries audio back to Twilio, and only <Connect><Stream> (bidirectional) does that. Its one-way sibling, <Start><Stream>, can fork the caller’s audio to you but cannot carry any audio back — fine for a transcriber, useless for playback. Unlike the STT guide, no <Say> or <Pause> follows: with <Connect>, the stream is the call, and it stays up until you close the socket.

Verify: curl -X POST https://YOUR_HOST/twiml returns the XML above with your wss:// URL and a <Connect> (not <Start>) element.

Step 3: Generate speech with Deepgram Flux TTS

Flux TTS is Deepgram’s streaming, turn-based text-to-speech built for voice-agent pipelines. You open a WebSocket, send text as a Speak message, and Flush to end the turn. The server replies with a SpeechStarted marker, a run of binary audio frames, and a SpeechMetadata summary once the turn’s audio is complete — that last message is your cue to stop reading. Requesting mulaw at 8 kHz matches Twilio’s Media Streams format exactly, so the bytes drop straight into media frames with no transcoding.

1
from deepgram.speak.v2 import SpeakV2Error, SpeakV2Speak, SpeakV2SpeechMetadata
2
3
async def deepgram_tts(text: str):
4
    async with dg_client.speak.v2.connect(
5
        model="flux-alexis-en",
6
        encoding="mulaw",
7
        sample_rate="8000",                             # note: string, not int
8
    ) as socket:
9
        await socket.send_speak(SpeakV2Speak(text=text))
10
        await socket.send_flush()
11
        async for message in socket:
12
            if isinstance(message, bytes):
13
                yield message                           # a chunk of generated audio
14
            elif isinstance(message, SpeakV2SpeechMetadata):
15
                break                                   # turn complete — all audio arrived
16
            elif isinstance(message, SpeakV2Error):
17
                raise RuntimeError(f"Flux TTS error: {message.description}")
18
        await socket.send_close()

Flux model strings follow the format flux-{voice}-{language} (for example, flux-alexis-en); swap the voice to change how the message sounds. See Flux TTS voices for the current list.

Step 4: Stream the audio into the call

As audio arrives from Flux, forward it to Twilio as media frames. Flux emits arbitrarily sized chunks, so buffer them and re-slice into exact 20 ms frames (160 bytes of mulaw). Pace the frames at real time so you don’t dump the whole clip into Twilio’s buffer at once. After the last frame, send a mark; Twilio echoes it back once playback reaches that point, which is your cue that the message finished playing.

1
async def speak(twilio_ws, stream_sid, text):
2
    buffer = bytearray()
3
4
    async def send_frame(frame):
5
        await twilio_ws.send_text(json.dumps({
6
            "event": "media", "streamSid": stream_sid,
7
            "media": {"payload": base64.b64encode(frame).decode()},
8
        }))
9
        await asyncio.sleep(0.02)                        # real-time pacing
10
11
    async for chunk in deepgram_tts(text):
12
        buffer.extend(chunk)
13
        while len(buffer) >= 160:                        # 160 bytes = 20ms of mulaw 8k
14
            await send_frame(bytes(buffer[:160]))
15
            del buffer[:160]
16
    if buffer:                                           # trailing partial frame
17
        await send_frame(bytes(buffer))
18
19
    await twilio_ws.send_text(json.dumps({
20
        "event": "mark", "streamSid": stream_sid, "mark": {"name": "end-of-speech"},
21
    }))

Every message you send back must carry the call’s streamSid, which you capture from Twilio’s start event.

Step 5: Drive the call from the media socket

The /media WebSocket ties it together. Accept the socket, and on Twilio’s start event generate and stream the message. Then wait for your end-of-speech mark to come back and hang up. The handler skips inbound media events (the caller’s own audio), because this app never listens.

1
@app.websocket("/media")
2
async def media(twilio_ws: WebSocket) -> None:
3
    await twilio_ws.accept()
4
    async for raw in twilio_ws.iter_text():
5
        msg = json.loads(raw)
6
        event = msg.get("event")
7
        if event == "start":
8
            stream_sid = msg["start"]["streamSid"]
9
            await speak(twilio_ws, stream_sid, MESSAGE)   # generate + stream
10
        elif event == "mark" and msg["mark"]["name"] == "end-of-speech":
11
            break                                         # playback done -> end the call
12
        elif event == "media":
13
            pass                                          # caller audio -> ignored
14
        elif event == "stop":
15
            break

Waiting for the mark before closing matters: if you close the socket the instant you send the last frame, you cut off the audio Twilio still has buffered. The mark tells you Twilio actually played it.

With TwiML answering the call, Flux generating the audio, and the mark timing the hangup, the playback service is complete — time to dial it from a real phone.

Step 6: Run and test

Start the application and open a tunnel so Twilio can reach it.

$
python app.py                   # or: uvicorn app:app --port 5050 --reload
$
ngrok http 127.0.0.1:5050       # in another terminal; copy the forwarding host into .env PUBLIC_HOSTNAME

Next, connect the phone number to your webhook. In the Twilio Console, open Phone Numbers → Manage → Active numbers → [your number] → Voice Configuration, set A call comes in to a Webhook pointing at https://YOUR_HOST/twiml with method HTTP POST, and save.

Now place the call and confirm playback end to end.

1. Call the number.
2. Confirm you hear the generated MESSAGE in a natural Flux voice.
3. Confirm the call ends cleanly once the message finishes.
4. Watch the console for [call] started stream ... and then [call] playback finished.

Hearing the full message without clipping confirms the bridge and Deepgram are working together.

Secure the endpoints

Your tunnel exposes both endpoints to the public internet. The companion app.py ships two optional guards that two environment variables switch on.

• TWILIO_AUTH_TOKEN — validates the X-Twilio-Signature header so /twiml answers only real Twilio requests. Find it in the Twilio Console under Account → API keys & tokens → Auth Token.
• STREAM_SECRET — a random string the TwiML passes as a <Parameter> and the app checks on the /media start event, so /media accepts only the sockets your own TwiML opened. Generate one with python -c "import secrets; print(secrets.token_urlsafe(32))".

Go further with Deepgram

Once the core playback runs, several enhancements build on the same streaming approach.

• Speak dynamic text. Instead of a constant MESSAGE, drive the text from a database lookup, an incoming webhook, or query parameters passed through the TwiML — order confirmations, appointment reminders, account balances.
• Choose a different voice. Swap the speak model for another Flux voice to change tone, gender, or accent.
• Play several messages. Call speak(...) more than once (each with its own mark) to chain prompts, or keep the socket open and stream new audio whenever you have something to say.
• Add listening for true barge-in. Barge-in here is manual only — with no transcription, the app can’t detect the caller talking over the message. Add Deepgram speech-to-text, or move to the Voice Agent API, to stop playback the instant the caller speaks.

What’s next

• Flux TTS overview
• Text-to-Speech REST API
• Twilio and Deepgram STT
• Twilio and Deepgram Voice Agent
• Twilio Media Streams