Agent LLM Models

An overview of the LLM providers and models you can use with the Voice Agent API.

Defines the LLM (Large Language Model) to be used with your Agent.

📘
You can set your LLM model in the Setting Configuration for your Voice Agent. See the docs for more information.

Supported LLM providers and models

think.provider.type	think.model values	think.provider.url values	think.provider.key values
`open_ai`	`gpt-4o-mini`	none	none
`anthropic`	`claude-3-haiku-20240307`	none	none
`groq`	`Coming Soon!`	none	none
`custom`	`YOUR_MODEL_NAME`	`YOUR_LLM_COMPLETIONS_URL`	`SECRET_KEY_FOR_CUSTOM_URL`

Example

{
  "think": {
    "provider": {
      "type": "open_ai"
    },
    "instructions": "optional: this is the LLM System prompt",
    "url": "YOUR_LLM_COMPLETIONS_URL",
    "key": "SECRET_KEY_FOR_CUSTOM_URL"
  }
  "model": "gpt-4o-mini",
}

Passing a custom LLM through a Cloud Provider

Azure OpenAI Service

Checkout the docs on how to create and deploy an Azure OpenAI service resource.
See this code repository for a getting started example.

Amazon Bedrock

Checkout our workshop on using the Deepgram Voice Agent API with Amazon Bedrock.
See this code repository for a getting started example.

Updated about 7 hours ago