Automatically Transcribe and Summarize Phone Calls

One use for the Deepgram API is to transcribe calls between a caller and an agent. When paired with Twilio, a cloud communication platform that lets developers integrate a number of communication technologies into their applications, Deepgram's API can be used to streamline your workflow by providing bite-sized versions of call recordings.

In this guide, you'll learn how to use Twilio Functions and Deepgram's Summarization feature to send phone call summaries via SMS once a conversation has ended. Using Twilio, you will build a phone number that forwards callers to your agent and begins recording. When the call is complete, Deepgram will provide both a transcript and summary of the call. Finally, the transcript and summary will be sent to both the caller and agent via SMS.

If you need reference material or you'd rather not follow along with this guide, we provide a full version of the sample code after the tutorial.

Before You Start

Before you run the code, you'll need to do a few things.

Create a Deepgram Account

📘

Before you can use Deepgram, you'll need to create a Deepgram account. Signup is free and includes $200 in free credit and access to all of Deepgram's features!

Create a Deepgram API Key

📘

Before you start, you'll need to follow the steps in the Make Your First API Request guide to obtain a Deepgram API key, and configure your environment if you are choosing to use a Deepgram SDK.

Create a Twilio Account

Before you can use Twilio, you'll need to sign up for a Twilio account. Once signed up, make sure you have a phone number with SMS and Voice capabilities set up in your account.

Have Access to Two Phones

To test your project, you'll need access to two phones--one to make a call and one to receive a call.

Setting Up Twilio

To use Twilio, you will need to create a new service, add the Deepgram SDK as a dependency, and add the appropriate environment variables.

Create a Service

Create a new service, which can contain multiple Twilio Functions and assets related to a single project.

1. Log in to the Twilio Console, and navigate to Developer Tools > Functions & Assets.
2. Create a new service. It’s important that you create a new service rather than a standalone function.

Add Dependencies

Inside your new service, add the Deepgram SDK as a dependency:

1. Locate the Dependencies section.
2. Add @deepgram/sdk. To get the latest version, omit the version number.

Add Environment Variables

Inside your new service, add your Deepgram API Key and your agent phone number as environment variables:

1. Locate the Environment Variables section.

2. Add the following variables:

   Variable name	Variable content
   DEEPGRAM_KEY	Value of your Deepgram API Key generated in your Deepgram Console
   FORWARDING_NUMBER	Value of your agent phone number in E.164 formatting (example: +14155552671)

Recording and Forwarding Inbound Calls

Create a Twilio function that receives incoming call data and forwards it to your agent while recording it:

1. Rename the /welcome function to /inbound.

2. Replace the entire file with the following code:

   JavaScript

   exports.handler = function (context, event, callback) {
   	let twiml = new Twilio.twiml.VoiceResponse();
   	const dial = twiml.dial({
   		record: "record-from-answer-dual",
   		recordingStatusCallback: "/recordings",
   	});
   	dial.number(process.env.FORWARDING_NUMBER);
   	return callback(null, twiml);
   };
   

📘

When the call is completed, call data will be sent to /recordings, which you will create later in this guide.

3. Save the function, and select Deploy All. Once deployed, this function is ready to be used.

Configure Your Twilio Number

Now that you have created a function to receive incoming calls, apply it to your Twilio number:

1. Navigate to your Twilio number settings.
2. Under A Call Comes In, select Function.
3. Under Service, select your service.
4. Under Function Path, select /inbound.

Transcribing and Summarizing Calls

Now that your Twilio number is configured to record the phone call, you can use Deepgram to transcribe and summarize it.

Transcribe the Call

When a call is received, use Deepgram to transcribe it:

1. Create a new Twilio function named /transcribe.

2. Replace the boilerplate code with the following code:

   JavaScript

   import { createClient } from "@deepgram/sdk";
   const deepgram = createClient(process.env.DEEPGRAM_API_KEY);
   
   exports.handler = async function (context, event, callback) {
   	const { RecordingUrl, CallSid } = event;
   	const twilioClient = context.getTwilioClient();
   	const { from: caller, to: twilioNumber } = await twilioClient.calls(CallSid).fetch();
   
   	// Further code here
   
   	return callback(null, true);
   };
   

📘

This code uses the CallSid to look up the call to find additional call information. Once done, the caller’s phone number will be available in a variable called caller, and the number they placed the call to will be available in a variable called twilioNumber.

3. Generate a transcription of the call using Deepgram’s Node.js SDK:

   JavaScript

   const options = { punctuate: true, tier: "enhanced", summarize: true };
   const { result, error } = await deepgram.listen.prerecorded.transcribeUrl({ url: RecordingUrl }, options);
   

Summarize the Call

From Deepgram's transcription, isolate the summary:

const { summaries } = result.results.channels[0].alternatives[0];

Sending Summary Messages

Now that you have a summary of the call, you can send it to both the caller and the agent.

Format the Summary

Because summaries is an array of objects (containing summary text and the time period being summarized), you will need to turn summaries into one string that can be sent via SMS:

const summary = summaries.map((s) => s.summary).join("\n\n");

Send the Summary via SMS

Finally, you can send Deepgram's summary via SMS:

1. Just below the previous summary code, add the following:

   JavaScript

   for (let number of [process.env.FORWARD_NUMBER, caller]) {
   	await twilioClient.messages.create({
   		body: summary,
   		to: number,
   		from: twilioNumber,
   	});
   }
   

2. Save both files again, and deploy all functions in your service.

Testing Your Implementation

To test your implementation, call your Twilio number, pick it up on your "agent device", speak, and hang up. You should receive a summary message via SMS a few seconds later.

Full Sample Code

In case you need it for reference, we provide the full sample code used in this tutorial below:

// /inbound
exports.handler = function (context, event, callback) {
	let twiml = new Twilio.twiml.VoiceResponse();
	const dial = twiml.dial({
		record: "record-from-answer-dual",
		recordingStatusCallback: "/transcribe",
	});
	dial.number(process.env.FORWARDING_NUMBER);
	return callback(null, twiml);
};

// /transcribe
import { createClient } from "@deepgram/sdk";
const deepgram = createClient(process.env.DEEPGRAM_API_KEY);

exports.handler = async function (context, event, callback) {
	const { RecordingUrl, CallSid } = event;
	const twilioClient = context.getTwilioClient();
	const { from: caller, to: twilioNumber } = await twilioClient.calls(CallSid).fetch();

	const options = { punctuate: true, tier: "enhanced", summarize: true };
	const { result, error } = await deepgram.listen.prerecorded.transcribeUrl({ url: RecordingUrl }, options);

	if (error) {
 		throw error;
	}

	const { summaries } = result.results.channels[0].alternatives[0];
	const summary = summaries.map((s) => s.summary).join("\n\n");

	for (let number of [process.env.FORWARDING_NUMBER, caller]) {
		await twilioClient.messages.create({
			body: summary,
			to: number,
			from: twilioNumber,
		});
	}

	return callback(null, true);
};