Configure the Voice Agent

Learn about the voice agent configuration options for the agent, and both input and output audio.

Voice Agent

To configure the voice agent, you’ll need to send a Settings Configuration message immediately after connection. Below is a detailed explanation of the configurations available.

Configuration Parameters

ParameterDescription
audio.inputThe speech-to-text audio media input you wish to send. The audio must be uncontainerized and have the following encodings:
- Linear16
- alaw
- mulaw
audio.outputThe text-to-speech audio media output you wish to receive. See options
agent.listen.modelThe Deepgram speech-to-text model to be used. See options
agent.think.modelDefines the LLM Model to be used. See options
agent.think.provider.typeDefines the LLM Provider See options
agent.think.instructionsDefines the System Prompt for the LLM
agent.think.functionsPass functions to the agent that will be called throughout the conversation if/when needed as described per function. See options
agent.speak.modelThe Deepgram text-to-speech model to be used. See options
context.messagesUsed to restore existing conversation if websocket connection break.
context.replayUsed to to replay the last message, if it is an assistant message.

Full Example

Below is an in-depth example showing all the fields available fields for SettingConfigurations highlighting default values and optional properties.

JSON
1{
2  "type": "SettingsConfiguration",
3  "audio": {
4    "input": { // optional. defaults to 16kHz linear16
5      "encoding": "",
6      "sample_rate": 24000 // defaults to 24k 
7    },
8    "output": { // optional. see table below for defaults and allowed combinations
9      "encoding": "",
10      "sample_rate": 24000, // defaults to 24k
11      "bitrate": 0,
12      "container": ""
13    }
14  },
15  "agent": {
16    "listen": {
17      "model": "" // optional. default 'nova-2'
18    },
19    "think": {
20      "provider": {
21        "type": "" // see `LLM providers and models` table below
22      },
23      "model": "", // see `LLM providers and models` table below
24      "instructions": "", // optional (this is the LLM System prompt)
25      "functions": [
26        {
27          "name": "", // function name
28          "description": "", // tells the agent what the function does, and how and when to use it
29          "url": "", // the endpoint where your function will be called
30          "headers": [ // optional. Http headers to pass when calling the function. Only supports 'authorization'
31            {
32              "key": "authorization",
33              "value": ""
34            }
35          ],
36          "method": "post", // the http method to use when calling the function
37          "parameters": {
38            "type": "object",
39            "properties": {
40              "item": { // the name of the input property
41                "type": "string", // the type of the input
42                "description":"" // the description of the input so the agent understands what it is
43              }
44            },
45            "required": ["item"] // the list of required input properties for this function to be called
46          }
47        }
48      ]
49    },
50    "speak": {
51      "model": "" // default 'aura-asteria-en' for other providers see TTS Models documentation
52    }
53  },
54  "context": {
55    "messages": [], // LLM message history (e.g. to restore existing conversation if websocket connection breaks)
56    "replay": false // whether to replay the last message, if it is an assistant message
57  }
58}
Built with