What is Function Calling?

LLM function calling refers to the ability of large language models (LLMs) to invoke external functions or APIs in response to user queries, rather than simply generating text based on their training data. This capability allows LLMs to enhance their functionality by integrating with other systems, services, or databases to provide real-time data, perform specific tasks, or trigger actions.

How Function Calling Works

• User Query: A user asks the LLM something that requires external data or specific action (e.g., "Check the weather in New York" or "Book an appointment").
• Function Identification: The LLM identifies that the query requires a specific function to be called. For instance, if the user asks for the weather, the model recognizes that it needs to call a weather API rather than generate a general response.
• Parameter Extraction: The LLM analyzes the user's query to extract the required parameters (e.g., location, date, or other variables). For example, in the weather query, "New York" would be extracted as the location parameter.
• Call the Function: The LLM triggers an external function or API with the appropriate parameters. This could involve fetching live data, performing a task (e.g., making a booking), or retrieving information that is outside the LLM’s static knowledge.
• Return the Result: The function returns the result (such as the current weather data), which the LLM incorporates into its response back to the user.

Here's a step-by-step breakdown of what the diagram illustrates:

1. Your application calls the API

Your application sends a prompt to the LLM, along with definitions of the available functions that the LLM can call. These functions can represent specific operations or tasks that the LLM might trigger.

2. The model makes a decision

The LLM analyzes the input and determines whether to respond directly to the user or call one or more functions. This decision depends on the content of the prompt and whether the LLM deems a function execution necessary to fulfill the user's request.

3. API response with function specification
The API responds to your application by specifying which function (or functions) should be called and what arguments should be passed to it. This is the step where the LLM returns a clear action for your application to take.

4. Your application executes the function

After receiving the function and arguments from the API, your application executes the function with the specified arguments, completing the operation.

5. Your application sends the result back to the API

Once the function has been executed, your application sends the result (along with the original prompt) back to the API. This step provides the final output that can either be returned to the user or used for further operations.

Function Call Example

Below is very simple example of a function call with the Deepgram Voice Agent API.

{
  "name": "function_to_call", // The function name you wish to call.
  "description": "This is a function that does something.", // tells the agent what the function does, and how and when to use it
  "url": "https://your-function-endpoint.com", // the endpoint where your function will be called
  "headers": [ // optional. Http headers to pass when calling the function. Only supports 'authorization'
    {
      "key": "authorization",
      "value": "auth_token"
    }
  ],
  "method": "post", // the http method to use when calling the function
  "parameters": {
    "type": "object",
    "properties": {
      "input_property_name": { // the name of the input property
        "type": "string", // the type of the input
        "description": "The input to be processed." // the description of the input so the agent understands what it is
      }
    },
    "required": [
      "input_property_name"
    ] // the list of required input properties for this function to be called
  }
}

Function Calling Messages

Below are other important concepts to understand about performing end to end Function Calling.

Client: Function Call Response

The Function Call Response message is a JSON command that the client should reply with every time there is a Function Call Request received.

Server: Function Call Request

The Function Call Request message is used to call a function from the server.

Server: Function Calling

The Function Calling message is sent by the server to aid the client in debugging.

Function Code Samples

📘

For more ideas and function code samples for using Function Calling with your Agent check out this repository.

Function Calling Demo

📘

For a reference implementation of integrating a function call with Deepgram Voice Agent checkout this repository.