Invoke a Deepgram SageMaker Endpoint

Once your endpoint is deployed and in service, you invoke it to transcribe audio. The endpoint supports three invocation modes, depending on which endpoint type you deployed and how you need the response returned.

Complete, runnable examples for all three modes — in Python, TypeScript, and Java — are maintained in the deepgram-devs/dg-sagemaker repository. The sections below explain each mode and link to the corresponding example. See the repository’s README for setup and prerequisites.

Use the Deepgram SDKs with the SageMaker transport

You don’t have to call the AWS APIs directly. The Deepgram SDKs can target a SageMaker endpoint through a SageMaker transport, so you keep the same client-side request and response patterns whether you call the Deepgram-hosted API or your own SageMaker deployment. You swap the transport; your listen request and result-handling code stays the same.

For example, the Deepgram Java SDK pairs with the Deepgram SageMaker transport (com.deepgram:deepgram-sagemaker):

import com.deepgram.DeepgramClient;
import com.deepgram.sagemaker.SageMakerConfig;
import com.deepgram.sagemaker.SageMakerTransportFactory;
import com.deepgram.resources.listen.v1.websocket.V1WebSocketClient;
SageMakerConfig smConfig = SageMakerConfig.builder()
        .endpointName("<your-endpoint-name>")
        .region("us-east-2")
        .build();
DeepgramClient client = DeepgramClient.builder()
        .apiKey("unused") // auth is AWS SigV4 via the transport, not a Deepgram API key
        .transportFactory(new SageMakerTransportFactory(smConfig))
        .build();
// Same SDK surface as the Deepgram-hosted API:
V1WebSocketClient ws = client.listen().v1().v1WebSocket();
ws.onResults(r -> { /* handle transcript */ });
ws.connect(connectOptions).get();
ws.sendMedia(ByteString.of(audioChunk));
// ... send a CloseStream message when finished

The remaining sections show the underlying AWS APIs directly, which apply to any language.

Streaming (real-time)

Use streaming for live, interactive transcription over a persistent bidirectional connection. You send audio chunks and receive transcription results as the audio is processed, up to 30 minutes per connection.

Streaming uses the HTTP/2 bidirectional streaming client (@aws-sdk/client-sagemaker-runtime-http2 in TypeScript, aws_sdk_sagemaker_runtime_http2 in Python) against the SageMaker bidirectional runtime endpoint (https://runtime.sagemaker.<region>.amazonaws.com:8443). The request Body is an async iterable of payload parts:

• Binary audio is sent as a Bytes payload with DataType: "BINARY".
• Control messages (for example, KeepAlive and CloseStream) are sent as UTF-8 encoded JSON with DataType: "UTF8".

import {
  SageMakerRuntimeHTTP2Client,
  InvokeEndpointWithBidirectionalStreamCommand,
} from "@aws-sdk/client-sagemaker-runtime-http2";
const region = "us-east-2";
const client = new SageMakerRuntimeHTTP2Client({
  region,
  endpoint: `https://runtime.sagemaker.${region}.amazonaws.com:8443`,
});
// Async generator yielding audio chunks (BINARY) and control messages (UTF8)
async function* requestStream() {
  // yield { PayloadPart: { Bytes: audioChunk, DataType: "BINARY" } };
  // yield { PayloadPart: { Bytes: new TextEncoder().encode(
  //           JSON.stringify({ type: "CloseStream" })), DataType: "UTF8" } };
}
const command = new InvokeEndpointWithBidirectionalStreamCommand({
  EndpointName: "<your-endpoint-name>",
  ModelInvocationPath: "v1/listen",
  ModelQueryString: "model=nova-3&language=en&smart_format=true",
  Body: requestStream(),
});
const response = await client.send(command);
for await (const event of response.Body) {
  if (event.PayloadPart?.Bytes) {
    const message = new TextDecoder().decode(event.PayloadPart.Bytes);
    // message is a Deepgram JSON transcript result
  }
}

For the complete examples — file and microphone capture, payload wrapping, keepalive handling, and stream processing — see:

• TypeScript: js-stt/stt.file.ts and stt.microphone.ts
• Python: python-stt/stt_wav_stress.py (stream subcommand)

Synchronous (real-time)

Use synchronous invocation to transcribe a single pre-recorded file and receive the full transcript in one immediate response. This is Deepgram’s “batch” transcription on a real-time endpoint — there is no streaming connection and no queue. The request body is capped at 25 MB; use streaming or asynchronous invocation for larger audio.

You send the audio as the request body to InvokeEndpoint, pass the Deepgram parameters via CustomAttributes, and parse the transcript from the JSON response.

import json
import boto3
runtime = boto3.client("sagemaker-runtime", region_name="us-east-2")
with open("audio.wav", "rb") as f:
    response = runtime.invoke_endpoint(
        EndpointName="<your-endpoint-name>",
        ContentType="audio/wav",
        Accept="application/json",
        CustomAttributes="v1/listen?model=nova-3&language=en&punctuate=true",
        Body=f.read(),
    )
result = json.loads(response["Body"].read())
transcript = result["results"]["channels"][0]["alternatives"][0]["transcript"]

For the complete example, see python-stt/stt_wav_stress.py (batch subcommand) in the repository.

Asynchronous

Use asynchronous invocation for large or long-form pre-recorded files — up to 1 GB, with up to one hour of processing time. Requests are queued and processed with near real-time latency, and the result is written back to Amazon S3.

The flow is:

1. Upload the audio file to an S3 bucket.
2. Call InvokeEndpointAsync with InputLocation pointing to the uploaded file and the Deepgram parameters in CustomAttributes.
3. SageMaker immediately returns an OutputLocation and a FailureLocation in S3, and processes the request from the queue.
4. Poll the OutputLocation (success) and FailureLocation (error) prefixes until one appears, then download and parse the result — or react to an Amazon SNS notification, if configured.

import boto3
runtime = boto3.client("sagemaker-runtime", region_name="us-east-2")
response = runtime.invoke_endpoint_async(
    EndpointName="<your-async-endpoint-name>",
    InputLocation="s3://<your-bucket>/input/audio.wav",
    ContentType="audio/wav",
    Accept="application/json",
    CustomAttributes="v1/listen?model=nova-3&language=en",
    InvocationTimeoutSeconds=3600,
)
output_location = response["OutputLocation"]    # S3 URI for the transcript on success
failure_location = response["FailureLocation"]  # S3 URI for error details on failure

For the complete example — S3 upload, invocation, and polling for results — see python-stt/stt_wav_async.py in the repository.

Related resources

• Validate a Deepgram SageMaker Endpoint
• Troubleshooting
• Auto-Scaling SageMaker Endpoints
• deepgram-devs/dg-sagemaker example repository