LLM Models

Defines the LLM (Large Language Model) to be used with your Agent. The provider.type field specifies the format or protocol of the API.

For example:

• open_ai means the API follows OpenAI’s Chat Completions format.
• This option can be used with OpenAI, Azure OpenAI, or Amazon Bedrock — as long as the endpoint behaves like OpenAI’s Chat Completion API.

Supported LLM providers

You can query the following endpoint to check the supported models for each provider:

1
curl https://agent.deepgram.com/v1/agent/settings/think/models

Example Payload

1
{
2
  "models": [
3
    {
4
      "id": "gpt-4.1-mini",
5
      "name": "GPT-4.1 mini",
6
      "provider": "open_ai"
7
    },
8
    {
9
      "id": "gpt-4.1-nano",
10
      "name": "GPT-4.1 nano",
11
      "provider": "open_ai"
12
    },
13
    {
14
      "id": "gpt-4o-mini",
15
      "name": "GPT-4o mini",
16
      "provider": "open_ai"
17
    },
18
    {
19
      "id": "claude-3-haiku-20240307",
20
      "name": "Claude Haiku 3",
21
      "provider": "anthropic"
22
    },
23
    {
24
      "id": "claude-3-5-haiku-latest",
25
      "name": "Claude Haiku 3.5",
26
      "provider": "anthropic"
27
    }
28
  ]
29
}

The agent.think.endpoint is optional or required based on the provider type:

• For open_ai and anthropic, the endpoint field is optional because Deepgram provides managed LLMs for these providers.
• For all other provider types, endpoint is required because Deepgram does not manage those LLMs.
• If an endpoint is provided the url is required but headers are optional.

Supported LLM models

OpenAI

Anthropic

Google

Example Payload

1
// ... other settings ...
2
 "think": {
3
      "provider": {
4
        "type": "open_ai",
5
        "model": "gpt-4o-mini",
6
        "temperature": 0.7
7
      },
8
      "endpoint": { // Optional if LLM provider is open_ai or anthropic. Required for non-Deepgram LLM providers like x_ai
9
        "url": "https://api.example.com/llm", // Required if endpoint is provided
10
        "headers": { // Optional if an endpoint is provided
11
          "authorization": "Bearer {{token}}"
12
        }
13
      },
14
    }
15
// ... other settings ...

Passing a custom (BYO) LLM through a Cloud Provider

You can use a custom LLM hosted by a 3rd party Cloud Provider by setting the provider.type to one of the supported provider values and setting the endpoint.url and endpoint.headers fields to the correct values for your Cloud Provider.

1
  // ... other settings ...
2
"think": {
3
      "provider": {
4
        "type": "open_ai",
5
        "model": "gpt-4",
6
        "temperature": 0.7
7
      },
8
      "endpoint": { // Required for a custom LLM
9
        "url": "https://cloud.provider.com/llm", // Required for a custom LLM
10
        "headers": { // Optional for a custom LLM
11
          "authorization": "Bearer {{token}}"
12
        }
13
      },
14
    }
15
  // ... other settings ...