Skip to navigation
Build with Deepgram

Use in the Voice Agent

Run Think-Fast or phoneLLM mini as the think stage of a Deepgram Voice Agent with the deepgram provider.

Early Access. Deepgram LLMs are available only to current Deepgram contract customers enrolled in the Early Access program (talk to sales for access). Usage is free during Early Access, requests run on the global endpoint only, SDKs do not support the endpoint yet, and the API can change before general availability. Do not run production traffic on it.

Use the Voice Agent API when you want Deepgram to run the full pipeline (listen, think, and speak) over one WebSocket. To use a Deepgram LLM as the agent’s LLM, set the think provider type to deepgram.

Configure the think stage

Set agent.think.provider in your Settings message:

{
  "type": "Settings",
  "agent": {
    "think": {
      "provider": {
        "type": "deepgram",
        "model": "think-fast",
        "temperature": 0.7
      },
      "prompt": "You are a friendly voice agent. Reply in one or two short sentences."
    }
  }
}
FieldValue
agent.think.provider.typedeepgram
agent.think.provider.modelthink-fast or phonellm-mini-alpha-2-fp8
agent.think.provider.temperatureOptional. From 0 to 2. Pipecat recommends 0.0 for phoneLLM mini.
agent.think.provider.reasoning_modeOptional. One of none, minimal, low, medium, or high. For voice, leave it unset or set it to none. With phoneLLM mini, low and higher make the agent speak its reasoning aloud.
agent.think.provider.voice_prompt_opt_outOptional Boolean, default false. Think-Fast only. Set to true to send your prompt unchanged. See Voice style instructions.
agent.think.endpointOmit. Deepgram hosts these models.

/v1/think calls the same control reasoning_effort. In the Voice Agent, use reasoning_mode: a Settings message with reasoning_effort is rejected with UNPARSABLE_CLIENT_MESSAGE.

Think-Fast can only be used with the Model Improvement Program. A Settings message that selects think-fast in a session opted out of MIP (mip_opt_out: true at the top level of Settings, or a project that requires opt-out) is rejected with INVALID_SETTINGS. Use phonellm-mini-alpha-2-fp8 for sessions that opt out. See Data retention and MIP.

Voice style instructions

With think-fast, Deepgram places instructions on spoken response style ahead of your prompt and sends a Warning with code VOICE_PROMPT_ADDED after Settings:

The think model think-fast is tuned for voice, so Deepgram places instructions on spoken response style ahead of your prompt. Set agent.think.provider.voice_prompt_opt_out to true to send your prompt unchanged.

Set agent.think.provider.voice_prompt_opt_out to true to send your prompt unchanged. phoneLLM mini sessions do not get these instructions.

SDK support

Typed SDK support for the deepgram think provider is coming. The Voice Agent works with Think-Fast and phoneLLM mini from every Deepgram SDK today:

SDKWhat to do
GoSet the provider type to deepgram as usual. The provider is a map, so it passes through.
.NETSet Provider.Type = "deepgram" as usual.
JavaScriptCast the provider to Deepgram.ThinkSettingsV1Provider, because strict TypeScript rejects type: "deepgram".
PythonSend the settings as a plain dict with send_raw.
JavaLeave .think() unset and pass the think block as a map with additionalProperty("think", ...).

In Python, do not build typed settings with model_construct to get around validation. It sends "provider": null, and the server rejects the message with UNPARSABLE_CLIENT_MESSAGE and no hint about why. Send a plain dict instead.

import { Deepgram } from "@deepgram/sdk";
connection.sendSettings({
  type: "Settings",
  audio: {
    input: { encoding: "linear16", sample_rate: 24000 },
    output: { encoding: "linear16", sample_rate: 24000, container: "wav" },
  },
  agent: {
    listen: { provider: { type: "deepgram", version: "v1", model: "nova-3" } },
    think: {
      // The SDK types don't list the deepgram provider yet, so cast it.
      provider: { type: "deepgram", model: "think-fast" } as unknown as Deepgram.ThinkSettingsV1Provider,
      prompt: "You are a friendly voice agent. Reply in one or two short sentences.",
    },
    speak: { provider: { type: "deepgram", model: "aura-2-thalia-en" } },
  },
});

voice_prompt_opt_out reaches the server the same way: add it to the provider object.

Function calls and context work the same way as with the other Voice Agent LLM providers:

When to call /v1/think directly instead

Use the Voice Agent API when you want Deepgram to manage turn-taking, barge-in, and audio. Call /v1/think directly when you run your own orchestration, such as a custom STT, LLM, and TTS loop. See Build a cascaded agent.