LLM Models

An overview of the LLM providers and models you can use with the Voice Agent API.

Defines the LLM (Large Language Model) to be used with your Agent.

📘
You can set your LLM model in the Setting Configuration for your Voice Agent. See the docs for more information.

Supported LLM providers and models

think.provider.type	think.model values	think.provider.url values	think.provider.headers values
`open_ai`	`gpt-4o-mini`	none	none
`anthropic`	`claude-3-haiku-20240307`	none	none
`groq`	`Coming Soon!`	none	none
`custom`	`YOUR_MODEL_NAME`	`YOUR_LLM_COMPLETIONS_URL`	`Array of Headers to pass with your auth token`

Example

{
  "think": {
    "provider": {
      "type": "open_ai"
    },
    "instructions": "this is the LLM System prompt",
    "model": "gpt-4o-mini"
  }
}

Passing a custom LLM through a Cloud Provider

You can pass a custom chat completions URL and headers to set your authorization or api-key etc

{
  "think": {
    "provider": {
      "type": "custom",
      "url": "YOUR_LLM_CHAT_COMPLETIONS_URL",
      "headers": [
        {
          "key": "Authorization",
          "value": "Bearer sk-xxx"
        }
      ]
    },
    "instructions": "this is the LLM System prompt",
    "model": "your-model-name"
  }
}

Azure OpenAI Service

Checkout the docs on how to create and deploy an Azure OpenAI service resource.
See this code repository for a getting started example.

Amazon Bedrock

Checkout our workshop on using the Deepgram Voice Agent API with Amazon Bedrock.
See this code repository for a getting started example.

Updated 21 days ago