LLM Models

An overview of the LLM providers and models you can use with the Voice Agent API.

Defines the LLM (Large Language Model) to be used with your Agent. The provider.type field specifies the format or protocol of the API.

For example:

  • open_ai means the API follows OpenAI’s Chat Completions format.
  • This option can be used with OpenAI, Azure OpenAI, or Amazon Bedrock — as long as the endpoint behaves like OpenAI’s Chat Completion API.

You can set your Voice Agent’s LLM model in the Settings Message See the docs for more information.

Supported LLM providers

You can query the following endpoint to check the supported models for each provider:

1curl https://agent.deepgram.com/v1/agent/settings/think/models

Example Payload

1{
2  "models": [
3    {
4      "id": "gpt-5.2-instant",
5      "name": "GPT-5.2 Instant",
6      "provider": "open_ai"
7    },
8   {
9      "id": "gpt-5.2",
10      "name": "GPT-5.2 Thinking",
11      "provider": "open_ai"
12    },
13    {
14      "id": "gpt-5.1-chat-latest",
15      "name": "GPT-5.1 Instant",
16      "provider": "open_ai"
17    },
18    {
19      "id": "gpt-5.1",
20      "name": "GPT-5.1 Thinking",
21      "provider": "open_ai"
22    },
23    {
24      "id": "gpt-5-nano",
25      "name": "GPT-5 Nano",
26      "provider": "open_ai"
27    },
28    {
29      "id": "gpt-5-mini",
30      "name": "GPT-5 Mini",
31      "provider": "open_ai"
32    },
33    {
34      "id": "gpt-5",
35      "name": "GPT-5",
36      "provider": "open_ai"
37    },
38    {
39      "id": "gpt-4.1",
40      "name": "GPT-4.1",
41      "provider": "open_ai"
42    },
43    {
44      "id": "gpt-4.1-mini",
45      "name": "GPT-4.1 mini",
46      "provider": "open_ai"
47    },
48    {
49      "id": "gpt-4.1-nano",
50      "name": "GPT-4.1 nano",
51      "provider": "open_ai"
52    },
53    {
54      "id": "gpt-4o",
55      "name": "GPT-4o",
56      "provider": "open_ai"
57    },
58    {
59      "id": "gpt-4o-mini",
60      "name": "GPT-4o mini",
61      "provider": "open_ai"
62    },
63    {
64      "id": "claude-4-5-haiku-latest",
65      "name": "Claude Haiku 4.5",
66      "provider": "anthropic"
67    },
68    {
69      "id": "claude-sonnet-4-5",
70      "name": "Claude Sonnet 4.5",
71      "provider": "anthropic"
72    },
73    {
74      "id": "claude-3-5-haiku-latest",
75      "name": "Claude Haiku 3.5",
76      "provider": "anthropic"
77    },
78    {
79      "id": "claude-sonnet-4-20250514",
80      "name": "Claude Sonnet 4",
81      "provider": "anthropic"
82    },
83    {
84      "id": "gemini-3-flash-preview",
85      "name": "Gemini 3.0 Flash Preview",
86      "provider": "google"
87    },
88    {
89      "id": "gemini-3-pro-preview",
90      "name": "Gemini 3.0 Pro Preview",
91      "provider": "google"
92    },
93    {
94      "id": "gemini-2.5-flash-lite",
95      "name": "Gemini 2.0 Flash Lite",
96      "provider": "google"
97    },
98    {
99      "id": "gemini-2.5-flash",
100      "name": "Gemini 2.5 Flash",
101      "provider": "google"
102    },
103    {
104      "id": "gemini-2.0-flash",
105      "name": "Gemini 2.0 Flash",
106      "provider": "google"
107    },
108    {
109      "id": "openai/gpt-oss-20b",
110      "name": "GPT OSS 20B",
111      "provider": "groq"
112    }
113  ]
114}

If you don’t specify agent.think.provider.type the Voice Agent will use Deepgram’s default managed LLMs. For managed LLMs, supported model names are predefined in our configuration.

Parameteropen_aianthropicaws_bedrockgooglegroq
agent.think.provider.typeopen_aianthropicaws_bedrockgooglegroq
agent.think.endpointoptionaloptionalrequiredoptionalrequired

The agent.think.endpoint is optional or required based on the provider type:

  • For open_ai, anthropic, and google, the endpoint field is optional because Deepgram provides managed LLMs for these providers.
  • For groq and aws_bedrock provider types, endpoint is required because Deepgram does not manage those LLMs.
  • If an endpoint is provided the url is required but headers are optional.

When using aws_bedrock as the provider type, you must also provide AWS credentials in the agent.think.provider.credentials field. This should include:

  • type: Either “iam” or “sts”
  • region: AWS region (e.g., “us-east-2”)
  • access_key_id: Your AWS access key ID
  • secret_access_key: Your AWS secret access key
  • session_token: Required only when type is “sts”

Supported LLM models

OpenAI

ProviderModelPricing Tier
open_aigpt-5.2-instantStandard
open_aigpt-5.2Advanced
open_aigpt-5.1Advanced
open_aigpt-5.1-chat-latestAdvanced
open_aigpt-5Advanced
open_aigpt-5-miniStandard
open_aigpt-5-nanoStandard
open_aigpt-4.1Advanced
open_aigpt-4.1-miniStandard
open_aigpt-4.1-nanoStandard
open_aigpt-4oAdvanced
open_aigpt-4o-miniStandard

Anthropic

ProviderModelPricing Tier
anthropicclaude-sonnet-4-5Advanced
anthropicclaude-4-5-haiku-latestStandard
anthropicclaude-3-5-haiku-latestStandard
anthropicclaude-sonnet-4-20250514Advanced

Google

ProviderModelPricing Tier
googlegemini-3-flash-previewStandard
googlegemini-3-pro-previewAdvanced
googlegemini-2.5-flashStandard
googlegemini-2.0-flashStandard
googlegemini-2.0-flash-liteStandard

Example using Deepgram’s managed Google LLM

JSON
1  // ... other settings ...
2  "think": {
3    "provider": {
4      "type": "google",
5      "model": "gemini-2.5-flash",
6      "temperature": 0.5
7    }
8  }
9  // ... other settings ...

Example using a custom Google endpoint (BYO)

When using a custom endpoint, the model property is not supported. The desired model is specified as part of the endpoint URL instead.

Use API keys from Google AI Studio for Gemini models. Keys from Vertex AI, Workspace Gemini, or Gemini Enterprise will not work with the Agent API.

JSON
1  // ... other settings ...
2  "think": {
3    "provider": {
4      "type": "google",
5      "temperature": 0.5
6    },
7    "endpoint": {
8      "url": "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:streamGenerateContent?alt=sse",
9      "headers": {
10        "x-goog-api-key": "xxxxxxxxx"
11      }
12    }
13  }
14  // ... other settings ...

Groq

ProviderModelPricing Tier
groqopenai/gpt-oss-20bStandard

Example Payload

JSON
1// ... other settings ...
2 "think": {
3      "provider": {
4        "type": "open_ai",
5        "model": "gpt-4o-mini",
6        "temperature": 0.7
7      },
8      "endpoint": { // Optional if LLM provider is open_ai, anthropic, or google. Required for 3rd party LLM providers such as groq
9        "url": "https://api.example.com/llm", // Required if endpoint is provided
10        "headers": { // Optional if an endpoint is provided
11          "authorization": "Bearer {{token}}"
12        }
13      },
14    }
15// ... other settings ...

Passing a custom (BYO) LLM through a Cloud Provider

For Bring Your Own (BYO) LLMs, any model string provided is accepted without restriction.

Deepgram tests against major LLM providers including OpenAI, Anthropic, and Google. When bringing your own LLM, you have two options:

  • Use an OpenAI-compatible LLM service or gateway. Set provider.type to open_ai and point the endpoint.url to your service. Any LLM endpoint that conforms to the OpenAI Chat Completions API format will work, including third-party LLM gateways.
  • Use a custom endpoint from one of the supported major LLM providers. If you have your own contract or deployment with a supported provider (such as OpenAI, Anthropic, or Google), set the provider.type to match that provider and supply your own endpoint.url and endpoint.headers.

In both cases, configure the provider.type to one of the supported provider values and set the endpoint.url and endpoint.headers fields to the correct values for your provider or gateway.

JSON
1  // ... other settings ...
2"think": {
3      "provider": {
4        "type": "open_ai",
5        "model": "gpt-4",
6        "temperature": 0.7
7      },
8      "endpoint": { // Required for a custom LLM
9        "url": "https://cloud.provider.com/llm", // Required for a custom LLM
10        "headers": { // Optional for a custom LLM
11          "authorization": "Bearer {{token}}"
12        }
13      },
14    }
15  // ... other settings ...

Using Multiple LLM Providers

If you need to set a fallback LLM provider, you can define multiple LLM providers for your Voice Agent. The thinkobject supports both a single provider and an array of providers.

Example

JSON
1{
2"agent": {
3  "think": [
4    {
5      "provider": {
6        "type": "open_ai",
7        "model": "gpt-4o-mini",
8        "temperature": 0.7
9      }
10    },
11    {
12      "provider": {
13        "type": "anthropic",
14        "model": "claude-4-5-haiku-latest",
15        "temperature": 0.7
16      }
17    }
18  ]
19}
20}