LLM Models | Deepgram's Docs

Defines the LLM (Large Language Model) to be used with your Agent. The provider.type field specifies the format or protocol of the API.

For example:

open_ai means the API follows OpenAI’s Chat Completions format.
This option can be used with OpenAI, Azure OpenAI, or Amazon Bedrock — as long as the endpoint behaves like OpenAI’s Chat Completion API.

You can set your Voice Agent’s LLM model in the Settings Message See the docs for more information.

Supported LLM providers

You can query the following endpoint to check the supported models for each provider:

1 curl https://agent.deepgram.com/v1/agent/settings/think/models

Example Payload

1 {
2   "models": [
3     {
4       "id": "gpt-4.1",
5       "name": "GPT-4.1",
6       "provider": "open_ai"
7     },
8     {
9       "id": "gpt-4.1-mini",
10       "name": "GPT-4.1 mini",
11       "provider": "open_ai"
12     },
13     {
14       "id": "gpt-4.1-nano",
15       "name": "GPT-4.1 nano",
16       "provider": "open_ai"
17     },
18     {
19       "id": "gpt-4o",
20       "name": "GPT-4o",
21       "provider": "open_ai"
22     },
23     {
24       "id": "gpt-4o-mini",
25       "name": "GPT-4o mini",
26       "provider": "open_ai"
27     },
28     {
29       "id": "claude-3-5-haiku-latest",
30       "name": "Claude Haiku 3.5",
31       "provider": "anthropic"
32     },
33     {
34       "id": "claude-sonnet-4-20250514",
35       "name": "Claude Sonnet 4",
36       "provider": "anthropic"
37     },
38     {
39       "id": "gemini-2.0-flash",
40       "name": "Gemini 2.0 Flash",
41       "provider": "google"
42     },
43     {
44       "id": "gemini-2.5-flash-lite",
45       "name": "Gemini 2.0 Flash Lite",
46       "provider": "google"
47     },
48     {
49       "id": "gemini-2.5-flash",
50       "name": "Gemini 2.5 Flash",
51       "provider": "google"
52     },
53     {
54       "id": "openai/gpt-oss-20b",
55       "name": "GPT OSS 20B",
56       "provider": "groq"
57     }
58   ]
59 }

If you don’t specify agent.think.provider.type the Voice Agent will use Deepgram’s default managed LLMs. For managed LLMs, supported model names are predefined in our configuration.

Parameter	`open_ai`	`anthropic`	`aws_bedrock`	`google`	`groq`
`agent.think.provider.type`	`open_ai`	`anthropic`	`aws_bedrock`	`google`	`groq`
`agent.think.endpoint`	optional	optional	required	required	required

The agent.think.endpoint is optional or required based on the provider type:

For open_ai and anthropic, the endpoint field is optional because Deepgram provides managed LLMs for these providers.
For google, groq, and aws_bedrock provider types, endpoint is required because Deepgram does not manage those LLMs.
If an endpoint is provided the url is required but headers are optional.

When using aws_bedrock as the provider type, you must also provide AWS credentials in the agent.think.provider.credentials field. This should include:

type: Either “iam” or “sts”
region: AWS region (e.g., “us-east-2”)
access_key_id: Your AWS access key ID
secret_access_key: Your AWS secret access key
session_token: Required only when type is “sts”

Supported LLM models

OpenAI

Provider	Model	Pricing Tier
`open_ai`	`gpt-5`	`Advanced`
`open_ai`	`gpt-5-mini`	`Standard`
`open_ai`	`gpt-5-nano`	`Standard`
`open_ai`	`gpt-4.1`	`Advanced`
`open_ai`	`gpt-4.1-mini`	`Standard`
`open_ai`	`gpt-4.1-nano`	`Standard`
`open_ai`	`gpt-4o`	`Advanced`
`open_ai`	`gpt-4o-mini`	`Standard`

Anthropic

Provider	Model	Pricing Tier
`anthropic`	`claude-3-5-haiku-latest`	`Standard`
`anthropic`	`claude-sonnet-4-20250514`	`Advanced`

Google

Provider	Model	Pricing Tier
`google`	`gemini-2.5-flash`	`Standard`
`google`	`gemini-2.0-flash`	`Standard`
`google`	`gemini-2.0-flash-lite`	`Standard`

Example

The google LLM provider does not allow specifying the model property. The desired model is specified as part of the endpoint URL instead.

Use API keys from Google AI Studio for Gemini models. Keys from Vertex AI, Workspace Gemini, or Gemini Enterprise will not work with the Agent API.

JSON

1   // ... other settings ...
2   "think": {
3     "provider": {
4       "type": "google",
5       "temperature": "0.5" // Omit or customize this, based on your needs
6     },
7     "endpoint": {
8       // Specify which model you want to use in the URL
9       "url": "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:streamGenerateContent?alt=sse",
10       "headers": {
11         "x-goog-api-key": "xxxxxxxxx", // Use your Gemini API key from AI Studio here
12       }
13     }
14   }
15   // ... other settings ...

Groq

Provider	Model	Pricing Tier
`groq`	`openai/gpt-oss-20b`	`Standard`

Example Payload

JSON

1 // ... other settings ...
2  "think": {
3       "provider": {
4         "type": "open_ai",
5         "model": "gpt-4o-mini",
6         "temperature": 0.7
7       },
8       "endpoint": { // Optional if LLM provider is open_ai or anthropic. Required for 3rd party LLM providers such as groq and google
9         "url": "https://api.example.com/llm", // Required if endpoint is provided
10         "headers": { // Optional if an endpoint is provided
11           "authorization": "Bearer {{token}}"
12         }
13       },
14     }
15 // ... other settings ...

Passing a custom (BYO) LLM through a Cloud Provider

For Bring Your Own (BYO) LLMs, any model string provided is accepted without restriction.

You can use a custom LLM hosted by a 3rd party Cloud Provider by setting the provider.type to one of the supported provider values and setting the endpoint.url and endpoint.headers fields to the correct values for your Cloud Provider.

JSON

1   // ... other settings ...
2 "think": {
3       "provider": {
4         "type": "open_ai",
5         "model": "gpt-4",
6         "temperature": 0.7
7       },
8       "endpoint": { // Required for a custom LLM
9         "url": "https://cloud.provider.com/llm", // Required for a custom LLM
10         "headers": { // Optional for a custom LLM
11           "authorization": "Bearer {{token}}"
12         }
13       },
14     }
15   // ... other settings ...