LLM Models

An overview of the LLM providers and models you can use with the Voice Agent API.

Voice Agent

Defines the LLM (Large Language Model) to be used with your Agent.

You can set your Voice Agent’s LLM model in the Settings Message See the docs for more information.

Supported LLM providers

Provider endpoint and headers are optional if the LLM provider is Deepgram but are required for non-Deepgram LLM providers.

agent.think.provider.typeagent.think.provider.endpointagent.think.provider.headers
open_aioptionaloptional
anthropicoptionaloptional
x_aioptionaloptional

Example Payload

JSON
1// ... other settings ...
2 "think": {
3      "provider": {
4        "type": "open_ai",
5        "model": "gpt-4",
6        "temperature": 0.7
7      },
8      "endpoint": { // Optional if LLM provider is Deepgram. Required for non-Deepgram LLM providers
9        "url": "https://api.example.com/llm",
10        "headers": {
11          "authorization": "Bearer {{token}}"
12        }
13      },
14// ... other settings ...

Passing a custom LLM through a Cloud Provider

You can use a custom LLM hosted by a 3rd party Cloud Provider by setting the provider.type to one of the supported provider values and setting the provider.endpoint.url and provider.headers fields to the correct values for your Cloud Provider.

JSON
1{
2  // ... other settings ...
3"think": {
4      "provider": {
5        "type": "open_ai",
6        "model": "gpt-4",
7        "temperature": 0.7
8      },
9      "endpoint": { // Optional if LLM provider is Deepgram. Required for non-Deepgram LLM providers
10        "url": "https://cloud.provider.com/llm",
11        "headers": {
12          "authorization": "Bearer {{token}}"
13        }
14      },
15  // ... other settings ...
16}

Azure OpenAI Service

Built with