LLM Models — Deepgram | Documentation

Defines the LLM (Large Language Model) to be used with your Agent.

You can set your LLM model in the Setting Configuration for your Voice Agent. See the docs for more information.

Supported LLM providers and models

think.provider.type	think.model values	think.provider.url values	think.provider.headers values
`open_ai`	`gpt-4o-mini`	none	none
`anthropic`	`claude-3-haiku-20240307`	none	none
`custom`	`YOUR_MODEL_NAME`	`YOUR_LLM_COMPLETIONS_URL`	`Array of Headers to pass with your auth token`

Example

JSON

1 {
2   "think": {
3     "provider": {
4       "type": "open_ai"
5     },
6     "instructions": "this is the LLM System prompt",
7     "model": "gpt-4o-mini"
8   }
9 }

Passing a custom LLM through a Cloud Provider

You can pass a custom chat completions URL and headers to set your authorization or api-key etc

JSON

1 {
2   "think": {
3     "provider": {
4       "type": "custom",
5       "url": "YOUR_LLM_CHAT_COMPLETIONS_URL",
6       "headers": [
7         {
8           "key": "Authorization",
9           "value": "Bearer sk-xxx"
10         }
11       ]
12     },
13     "instructions": "this is the LLM System prompt",
14     "model": "your-model-name"
15   }
16 }

Azure OpenAI Service

Checkout the docs on how to create and deploy an Azure OpenAI service resource.
See this code repository for a getting started example.

Amazon Bedrock

Checkout our workshop on using the Deepgram Voice Agent API with Amazon Bedrock.
See this code repository for a getting started example.

1	{
2	"think": {
3	"provider": {
4	"type": "open_ai"
5	},
6	"instructions": "this is the LLM System prompt",
7	"model": "gpt-4o-mini"
8	}
9	}

1	{
2	"think": {
3	"provider": {
4	"type": "custom",
5	"url": "YOUR_LLM_CHAT_COMPLETIONS_URL",
6	"headers": [
7	{
8	"key": "Authorization",
9	"value": "Bearer sk-xxx"
10	}
11	]
12	},
13	"instructions": "this is the LLM System prompt",
14	"model": "your-model-name"
15	}
16	}