> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://developers.deepgram.com/docs/amazon-sagemaker/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://developers.deepgram.com/_mcp/server.

# Amazon SageMaker

> Overview of running Deepgram on Amazon SageMaker AI. Compare benefits and tradeoffs versus self-hosted Docker or Kubernetes, review deployment options, and understand SageMaker pricing for Deepgram models.

Amazon SageMaker is a managed cloud platform from Amazon Web Services (AWS) that enables deployment of Deepgram as a managed, container-based service. The endpoint is air-gapped and runs on compute inside your own AWS VPC. Once you deploy Deepgram as a SageMaker Model Endpoint, you can run inference against the service using the Amazon SageMaker AI Software Development Kit (SDK).

The [Deepgram SDKs](/home) can also target a SageMaker Endpoint through the SageMaker transport, so you can keep the same client-side request and response patterns whether you call the Deepgram-hosted API or your own SageMaker deployment.

## Benefits and Tradeoffs

Deepgram on SageMaker is the fastest path to running Deepgram inside your own AWS account. Compared to self-hosting Deepgram on Docker or Kubernetes, SageMaker trades some flexibility for a managed endpoint that AWS operates on your behalf.

### When SageMaker is the right fit

* **Ease of deployment.** A ready-to-use endpoint can be created in minutes from the AWS Console or with infrastructure-as-code. There are no container images to mirror, no GPU drivers to install, and no Helm charts to maintain.
* **Lower management overhead.** AWS manages the underlying instances, host OS, container runtime, and model package distribution. You do not need a dedicated platform team to keep the service patched and healthy.
* **Compliance for regulated workloads.** Deepgram runs entirely inside your AWS account and VPC. Audio never leaves your environment, and you inherit the compliance posture of SageMaker AI (HIPAA-eligible, SOC, ISO, PCI, FedRAMP, and others). This makes SageMaker a strong fit for regulated industries that need a private deployment without operating their own Kubernetes platform.
* **Native integration with AWS services.** SageMaker Endpoints integrate out of the box with Amazon CloudWatch (logs and metrics), AWS IAM (authentication and authorization), Amazon VPC (network isolation), AWS PrivateLink, AWS KMS, AWS CloudTrail (audit), and SageMaker auto-scaling. You get production-grade observability and access controls without building them yourself.
* **AWS Marketplace billing.** Deepgram license charges flow through your existing AWS bill, simplifying procurement for teams that already buy through AWS.

### When Docker or Kubernetes may be a better fit

* You need to run Deepgram outside AWS or on bare metal.
* You require features that the SageMaker isolation model does not currently support, such as user-defined [callback URLs](/docs/callback) or JSON payloads that reference audio in cloud storage.
* You want to run the [Deepgram Voice Agent](/docs/voice-agent). SageMaker Endpoints cannot invoke Large Language Model (LLM) services, which the Voice Agent requires, so the Voice Agent cannot run inside SageMaker.
* You need streaming connections that stay open for longer than 30 minutes. SageMaker Real-Time Inference supports [up to 30 minutes of connection time](https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints-test-endpoints.html#realtime-endpoints-test-endpoints-sdk:~:text=The%20connection%20remains%20open%20until%20you%20explicitly%20close%20the%20input%20stream%20or%20the%20endpoint%20closes%20the%20connection%2C%20supporting%20up%20to%2030%20minutes%20of%20connection%20time) per bidirectional streaming connection.
* You need to send more than 25 MB of input data per non-streaming invocation. SageMaker enforces a [25 MB maximum payload size](https://docs.aws.amazon.com/marketplace/latest/userguide/ml-service-restrictions-and-limits.html#:~:text=For%20an%20endpoint%2C%20limit%20the%20maximum%20size%20of%20the%20input%20data%20per%20invocation%20to%2025%20MB.%20This%20value%20can%27t%20be%20adjusted) for real-time endpoints. Asynchronous endpoints (up to 1 GB per request) are temporarily not available for Marketplace-hosted Deepgram. If your use case needs them, contact your [Deepgram representative](https://deepgram.com/contact-us).
* You need fine-grained control over the container runtime, networking, or process supervision beyond what SageMaker exposes.

## Limitations

When using Deepgram services in Amazon SageMaker, please be aware of the following limitations.

* The SageMaker network isolation model prevents the container from making outbound calls to external LLM providers. As a result, the [Deepgram Voice Agent](/docs/voice-agent) cannot run inside SageMaker.
* Deepgram cannot invoke user-defined [callback URLs](/docs/callback)
* Passing a JSON payload for transcription (e.g., referencing a file stored in cloud storage via URL) is unsupported, as the SageMaker isolation model prevents the container from reaching out to external cloud storage
* Deepgram [custom metrics](/docs/metrics-guide) are not currently available through Amazon SageMaker Endpoints
* For streaming invocations, the connection remains open until you explicitly close the input stream or the endpoint closes the connection, supporting [up to 30 minutes of connection time](https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints-test-endpoints.html#realtime-endpoints-test-endpoints-sdk:~:text=The%20connection%20remains%20open%20until%20you%20explicitly%20close%20the%20input%20stream%20or%20the%20endpoint%20closes%20the%20connection%2C%20supporting%20up%20to%2030%20minutes%20of%20connection%20time).
* For non-streaming invocations, the [maximum size of the input data is 25 MB](https://docs.aws.amazon.com/marketplace/latest/userguide/ml-service-restrictions-and-limits.html#:~:text=For%20an%20endpoint%2C%20limit%20the%20maximum%20size%20of%20the%20input%20data%20per%20invocation%20to%2025%20MB.%20This%20value%20can%27t%20be%20adjusted) for real-time endpoints. Asynchronous endpoints (up to 1 GB per request) are temporarily not available for Marketplace-hosted Deepgram. If your use case needs them, contact your [Deepgram representative](https://deepgram.com/contact-us).

## Deployment options

Most customers can stand up a ready-to-use endpoint in minutes:

1. **Subscribe.** Subscribe to a Deepgram product on the AWS Marketplace and note its Model Package ARN. See [Subscribe on AWS Marketplace](/docs/subscribe-aws-marketplace).
2. **Deploy** with one of three paths:
   * **AWS CLI or SDK.** Create the SageMaker Model, Endpoint Configuration, and Endpoint with the AWS CLI or Boto3. This is the recommended path because it lets you pin the inference AMI version that current Deepgram model packages require. See [Deploy Deepgram on Amazon SageMaker](/docs/deploy-amazon-sagemaker).
   * **Infrastructure-as-Code.** Deploy the same model package using Terraform for repeatable, version-controlled rollouts. See [Deploy with Terraform](/docs/terraform-deploy-sagemaker).
   * **Agent-assisted (AI coding assistant).** Install the Deepgram SageMaker skill and let your assistant run each step with confirmation. See [Agent-assisted setup](#agent-assisted-setup).

> **Warning**
>
> The SageMaker AI console cannot set the inference AMI version, so endpoints created through the console fail to start with current Deepgram model packages. Use the CLI, SDK, or Terraform paths above.

## Agent-assisted setup

The [deepgram-devs/dg-sagemaker](https://github.com/deepgram-devs/dg-sagemaker/tree/main/skills/deepgram-sagemaker) repository publishes an installable agent skill (open `SKILL.md` format, plus a Claude Code plugin manifest) that walks an AI coding assistant through the whole flow: preflight, listing products and subscription state, subscribing, resolving the Model Package ARN, checking and requesting quota, creating the execution role, deploying on an instance pool, diagnosing endpoint status, running an invoke test, configuring auto-scaling, in-place updates, and teardown. Each step runs through a script in the kit.

```bash
npx skills add deepgram-devs/dg-sagemaker
# Claude Code
/plugin marketplace add deepgram-devs/dg-sagemaker
/plugin install deepgram-sagemaker@deepgram
```

The skill needs AWS credentials for the target account and [`uv`](https://docs.astral.sh/uv/). The assistant:

* Confirms with you before subscribing, creating, or deleting anything.
* Recommends an ordered [instance pool](/docs/deploy-amazon-sagemaker#choose-instance-types) rather than a single instance type.
* Refuses to deploy asynchronous endpoints, which are temporarily unsupported.
* Does not quote per-instance capacity numbers; measure on your own endpoint or ask a Deepgram representative for a planning estimate.

## Pricing

Deepgram on SageMaker is billed per request, at the same rates shown on [deepgram.com/pricing](https://deepgram.com/pricing). The AWS pricing page may list the dimension `inference.count.m.i.c Inference Pricing` at a cost of `$0.001/request`. When the cost of a request exceeds \$0.001, Deepgram automatically emits a charge for multiple units for that single request.

### Private offers

For larger or longer-term deployments, AWS Marketplace Private Offers are available with negotiated unit economics and committed-use terms. Contact your [AWS account team](https://aws.amazon.com/contact-us/sales-support/) or [Deepgram representative](https://deepgram.com/contact-us) to start a Private Offer.

### Try before you buy

A 14-day free trial is available with unlimited product usage and zero Deepgram license charges during the trial window. Each trial is available once per AWS account per product. Contact a [Deepgram representative](https://deepgram.com/contact-us) if you need additional time for testing.

### AWS field employees

If you are an AWS field employee, you can access Deepgram models through the [AWS Marketplace Field Demonstration Program](https://docs.aws.amazon.com/marketplace/latest/userguide/field-demonstration-program.html). Deepgram is an eligible provider. Refer to your internal AWS documentation for enrollment details, and reach out to a [Deepgram representative](https://deepgram.com/contact-us) if you need assistance activating the program.

### Infrastructure charges

Infrastructure charges are set by AWS and billed separately from Deepgram license charges. Public pricing for SageMaker Real-Time Inference is available at [aws.amazon.com/sagemaker/ai/pricing](https://aws.amazon.com/sagemaker/ai/pricing/). Self-service savings may be available on 1-year or 3-year committed usage by purchasing a [Machine Learning Savings Plan from AWS](https://aws.amazon.com/savingsplans/ml-pricing/). For more information or to discuss additional discounts, [contact your AWS sales representative](https://aws.amazon.com/contact-us/sales-support/).

> Run Deepgram inside your AWS account as a managed SageMaker AI Endpoint — with native AWS integrations, hourly billing, and a 14-day free trial.

## Docs

- [Supported Products](https://developers.deepgram.com/docs/supported-products-sagemaker.md): Deepgram products available as SageMaker Model Packages on the AWS Marketplace, and the GPU instance types each product supports.
- [Requesting SageMaker Quota](https://developers.deepgram.com/docs/request-sagemaker-quota.md): How to check and increase AWS service quotas for SageMaker endpoint instance types (ml.g4dn.2xlarge, ml.g5.2xlarge, ml.g6.2xlarge, ml.g6e.2xlarge, ml.g7.2xlarge, ml.g7e.2xlarge) required by Deepgram deployments.
- [Subscribe on AWS Marketplace](https://developers.deepgram.com/docs/subscribe-aws-marketplace.md): Subscribe to a Deepgram SageMaker product through the AWS Marketplace console or API, find the Model Package ARN, and manage private offers.
- [Deploy Deepgram on Amazon SageMaker](https://developers.deepgram.com/docs/deploy-amazon-sagemaker.md): Create the SageMaker Model, Endpoint Configuration, and Endpoint with the AWS CLI or Boto3, pinning the inference AMI version that current Deepgram model packages require.
- [Deploy with Terraform](https://developers.deepgram.com/docs/terraform-deploy-sagemaker.md): Deploy a Deepgram SageMaker Endpoint with Terraform using an AWS Marketplace Model Package ARN. Includes IAM role, endpoint configuration, auto-scaling, and optional environment variable overrides.
- [Configure Amazon SageMaker Deployments](https://developers.deepgram.com/docs/configure-sagemaker-deployments.md): Customize Deepgram services on Amazon SageMaker through environment variables that map to TOML configuration settings.
- [Invoke a Deepgram SageMaker Endpoint](https://developers.deepgram.com/docs/invoke-sagemaker-endpoint.md): Send audio to a Deepgram SageMaker Endpoint using streaming or synchronous invocation, or through the Deepgram SDKs' SageMaker transport.
- [Validate a Deepgram SageMaker Endpoint](https://developers.deepgram.com/docs/test-amazon-sagemaker-endpoint.md): Test a Deepgram SageMaker endpoint using the open-source client scripts in the dg-sagemaker repository, with one script per product and language.
- [Update an Amazon SageMaker Endpoint](https://developers.deepgram.com/docs/update-amazon-sagemaker-endpoint.md): Roll out a newer Deepgram Model Package or a different model version on a SageMaker Endpoint that is already serving production traffic.
- [Auto-Scaling SageMaker Endpoints](https://developers.deepgram.com/docs/auto-scaling-sagemaker.md): Compare real-time and asynchronous SageMaker endpoint types and choose the right autoscaling strategy for your Deepgram deployment.
- [Auto-Scaling Real-Time Endpoints](https://developers.deepgram.com/docs/auto-scaling-sagemaker-streaming.md): Use the CloudWatch ConcurrentRequestsPerModel metric to automatically scale your Amazon SageMaker real-time endpoints based on concurrent in-flight requests.
- [Auto-Scaling Asynchronous SageMaker Endpoints (Temporarily Unavailable)](https://developers.deepgram.com/docs/auto-scaling-sagemaker-async.md): Configure autoscaling with scale-to-zero for asynchronous Deepgram endpoints on Amazon SageMaker using queue-depth metrics.
- [Observability for Amazon SageMaker](https://developers.deepgram.com/docs/observability-sagemaker.md): Use Amazon CloudWatch to monitor Deepgram SageMaker Endpoints. Covers key metrics like ConcurrentRequestsPerModel and FirstChunkLatency, enhanced per-instance metrics, CloudWatch Logs for Deepgram container output, and alarm configuration for proactive alerting.
- [Prometheus & OpenTelemetry Metrics](https://developers.deepgram.com/docs/prometheus-otel-sagemaker.md): Enable Amazon SageMaker detailed observability on Deepgram Endpoints to collect per-GPU, host, and container Prometheus metrics through an AWS-managed OpenTelemetry Collector, and query them with PromQL from CloudWatch or Prometheus-compatible observability tools.
- [Deepgram Enhanced Metrics](https://developers.deepgram.com/docs/enhanced-metrics-sagemaker.md): Reference for the CloudWatch metrics Deepgram SageMaker containers emit via Embedded Metric Format: billing metrics in the Deepgram/SageMakerInference namespace and per-feature usage metrics in the Deepgram/SelfHosted namespace, with dimensions, units, and example queries.
- [Health Checks & Automatic Recovery](https://developers.deepgram.com/docs/health-checks-sagemaker.md): Understand the health check contract between Deepgram containers and Amazon SageMaker: what the container reports while models load and while it serves traffic, how SageMaker replaces unhealthy instances, the WebSocket ping/pong requirement for streaming connections, and the health-related log lines to look for in CloudWatch.
- [Security and Compliance](https://developers.deepgram.com/docs/security-and-compliance-sagemaker.md): Security and compliance for Deepgram on Amazon SageMaker: TLS requirements for API access, FIPS 140-3 endpoints for the control plane and inference traffic, FedRAMP coverage, network isolation for AWS Marketplace containers, container vulnerability scanning with no Critical or High CVEs, and VPC endpoint options for restricting access to your endpoint.
- [Use FIPS Endpoints](https://developers.deepgram.com/docs/fips-endpoints-sagemaker.md): How to configure boto3, the AWS SDKs, and the AWS CLI to reach Deepgram on Amazon SageMaker over FIPS 140-3 endpoints, including bidirectional streaming on port 8443, asynchronous endpoints on s3-fips, the IAM Identity Center caveat, and how to confirm a run used FIPS endpoints.
- [Troubleshooting](https://developers.deepgram.com/docs/troubleshooting-sagemaker.md): Diagnose Deepgram SageMaker Endpoints that fail to start, stay stuck in Creating, or return errors, using CloudWatch logs and the checks below.