LiveKit and Deepgram

This guide walks you through building a voice AI agent that uses LiveKit Agents for real-time audio transport and Deepgram for speech-to-text (STT) and text-to-speech (TTS). By the end, you will have a working voice agent that listens to a user, generates a response with an LLM, and speaks back in real time.

Deepgram is available in LiveKit Agents through two paths:

This guide starts with LiveKit Inference for the fastest setup, then shows how to switch to the Deepgram Plugin for direct API access and advanced features.

Before you begin

You need:

• A LiveKit Cloud account (or a self-hosted LiveKit server)
• An LLM API key. This guide uses OpenAI, but LiveKit Agents supports other providers
• Python 3.10+ or Node.js 18+

Step 1: Create the project

The fastest way to scaffold a new agent project is with the LiveKit CLI (lk). Alternatively, clone a starter template from GitHub (Python, Node.js).

$
lk agent init my-agent --template agent-starter-python
$
cd my-agent
$
pip install -r requirements.txt

When the CLI finishes, it prints a sandbox URL (for example, https://my-agent-xxxxx.sandbox.livekit.io). Note this URL. You will use it to test your agent later.

Step 2: Configure your environment

The CLI creates a .env.local file during setup. Open it and confirm your API keys are set:

$
LIVEKIT_URL=YOUR_LIVEKIT_CLOUD_URL
$
LIVEKIT_API_KEY=YOUR_LIVEKIT_API_KEY
$
LIVEKIT_API_SECRET=YOUR_LIVEKIT_API_SECRET
$
OPENAI_API_KEY=YOUR_OPENAI_API_KEY

Step 3: Use Deepgram for TTS

The starter template uses Deepgram for STT but Cartesia for TTS. To use Deepgram for both, find the tts argument in the AgentSession constructor in src/agent.py (Python) or src/main.ts (Node.js) and replace it:

1
# Replace the existing tts line with:
2
tts=inference.TTS(model="deepgram/aura-2", voice="thalia"),

The agent now uses Deepgram for both STT and TTS. No additional dependencies or API keys are needed because both run through LiveKit Inference.

Browse available voices in the Deepgram voice library.

Step 4: Run the agent

Download the required model files (VAD, turn detection), then start the agent in development mode:

$
python src/agent.py download-files
$
python src/agent.py dev

The dev command connects your agent to LiveKit Cloud’s sandbox environment. Open your sandbox URL in your browser to talk to your agent.

Step 5: Test the conversation

Once the agent is running, you should hear a greeting. Try these interactions to verify everything works:

1. Ask a question and confirm the agent responds with speech.
2. Start speaking while the agent is talking. It should stop and listen.
3. Pause after speaking. The agent should detect the end of your turn and respond.

If any of these fail, check your API keys and confirm the agent process is running.

Use the Deepgram Plugin for advanced features

The steps above use LiveKit Inference, which hosts Deepgram models through LiveKit Cloud. If you need direct access to Deepgram features like speaker diarization, keyterm prompting, or fine-grained parameter control, use the Deepgram Plugin instead. This connects directly to Deepgram’s API with your own API key.

Install the plugin

$
pip install "livekit-agents[deepgram]~=1.4"

Set your Deepgram API key

Add DEEPGRAM_API_KEY to your .env.local file:

$
DEEPGRAM_API_KEY=YOUR_DEEPGRAM_API_KEY

Update the AgentSession

Add the Deepgram import at the top of your entrypoint file, then replace the stt and tts arguments in the AgentSession:

1
from livekit.plugins import deepgram
2
3
# Replace the stt and tts lines in your AgentSession:
4
stt=deepgram.STT(
5
    model="nova-3",
6
    language="en",
7
    punctuate=True,
8
    interim_results=True,
9
),
10
tts=deepgram.TTS(model="aura-2-thalia-en"),

Use Flux for turn detection

Flux is Deepgram’s conversational STT model with built-in turn detection. It uses acoustic and semantic cues to determine when a speaker has finished their turn, resulting in more natural conversations with fewer awkward pauses.

To use Flux, replace the stt configuration with STTv2 and set turn detection to "stt":

1
# Replace the stt line and add turn_detection:
2
stt=deepgram.STTv2(model="flux-general-en"),
3
turn_detection="stt",

Choose a different voice

Deepgram offers 60+ voices across seven languages. Replace the model parameter in TTS with any supported voice:

1
tts = deepgram.TTS(model="aura-2-andromeda-en")

Browse all available voices in the Deepgram voice library.

Go further with Deepgram

• Use keyterm prompting to improve recognition of domain-specific vocabulary.
• Enable speaker diarization to assign a speaker label to each word in the transcript.
• See the full list of plugin parameters in the LiveKit Deepgram reference for Python and Node.js.

Resources

• Deepgram Models & Languages
• Deepgram TTS Voices
• LiveKit Agents Docs
• LiveKit Deepgram STT Guide
• LiveKit Deepgram TTS Guide
• livekit-plugins-deepgram on PyPI
• @livekit/agents-plugin-deepgram on npm