Microsoft Azure
Deploying Deepgram on Microsoft Azure requires some preparation. In this guide, you’ll provision an Azure Kubernetes Service (AKS) cluster with an NVIDIA GPU node pool, create shared storage for Deepgram model files, and install Deepgram with the official Helm chart. You’ll run every step in your local terminal with the Azure CLI.
Prerequisites
Make sure you’ve completed the requirements in the Self-Hosted Introduction.
Choose a GPU VM size
Deepgram Engine requires a dedicated NVIDIA GPU, and the GPU you choose determines which models you can run. Pick the VM size before you create the cluster.
- Model and GPU Compatibility is the source of truth for which Deepgram models (Nova-3, Nova-2, Aura-2, Flux STT, and Flux TTS) run on which NVIDIA GPUs.
- Hardware Specifications lists the CPU, memory, and storage each Engine needs.
- Every supported GPU requires NVIDIA driver
>=580with the open kernel modules. See Drivers and Containerization Platforms.
This table maps the Azure VM series that ship NVIDIA CUDA drivers to those compatibility rules.
VM size names and GPU counts change as Azure adds series. To confirm a size is available in your region, run az vm list-skus --location <your-region> --size Standard_NC --output table, and check Microsoft’s GPU-optimized VM sizes for current details.
Request GPU quota
New Azure subscriptions often have no quota for GPU VM families, and GPU capacity is limited in many regions. Request a vCPU quota increase for your chosen VM family in your target region before you create the cluster. Follow the steps in Requesting Quotas for NVIDIA GPUs.
Azure CLI
The Azure CLI provides programmatic access to manage your Azure services. Many of the same actions are available in the Azure portal.
-
Follow the installation guide to install the CLI locally. You need version
2.72.2or later, which adds the--gpu-driveroption this guide uses. -
Sign in and select the subscription to bill for this cluster.
Shell
Choosing a region
This guide provisions resources in the Azurewestus2 region. To deploy to a different region, change the LOCATION variable and confirm that your chosen GPU VM family is available there.kubectl
The Kubernetes command-line tool, kubectl, runs commands against Kubernetes clusters. You can use kubectl to deploy applications, inspect and manage cluster resources, and view logs.
Install it with the official Kubernetes guides, or run az aks install-cli.
Kubernetes packages with helm
Helm is the package manager for Kubernetes. A Helm chart defines a Kubernetes application so you can install and upgrade it. This guide uses Helm to install Deepgram and the NVIDIA GPU Operator. See the installation guide for details on how to install it locally.
Create a cluster
Azure Kubernetes Service (AKS) is a managed Kubernetes service. AKS manages the availability and scalability of the Kubernetes control plane that schedules containers, manages application availability, and stores cluster data.
-
Set the variables this guide reuses, then create a resource group and an AKS cluster. The cluster starts with a small system node pool. You’ll add node pools for each Deepgram component next.
Shell -
Create a separate node pool for each Deepgram component (API, License Proxy, Engine). Adjust the VM sizes and node counts to your needs. You may wish to consult your Deepgram Account Representative when you plan your cluster’s capacity.
ShellSkip the AKS GPU driver install
AKS can install an NVIDIA driver for you, but AKS selects the driver version. Deepgram requires driver>=580with the open kernel modules, so create the Engine node pool with--gpu-driver none. Thedeepgram-self-hostedHelm chart then installs a compliant driver with the NVIDIA GPU Operator. Don’t use--enable-managed-gpuon this node pool.You can only set
--gpu-driverwhen you create a node pool. To change it later, create a new node pool. -
Create a dedicated namespace for Deepgram resources.
Shell
Configure persistent storage
Deepgram Engine pods read model files from a shared volume. The Helm chart can download models automatically only for Amazon EFS, so on AKS you create an Azure Files NFS share, populate it from inside the cluster with a one-shot Kubernetes Job, and point the chart at the resulting PersistentVolumeClaim.
Azure Files NFS shares need a Premium (SSD) storage account and network access from your cluster’s subnet. Review the NFS prerequisites before you continue. If the AKS identity lacks the required roles, volume creation or mounting fails.
-
Create a
StorageClassfor NFS shares. TheRetainreclaim policy keeps your model files if you delete the claim later.Shell -
Create a
PersistentVolumeClaimfor the models. Premium file shares have a 100 GiB minimum size.ShellThe claim should reach the
Boundstatus within a few minutes. -
Run a one-shot
Jobthat mounts the claim and downloads the model files provided by your Deepgram Account Representative.Replace the
wgetlines below with the model URLs supplied by your Deepgram Account Representative.Shell -
Wait for the Job to complete, then delete it. The claim and the model files remain.
Shell
Configure Kubernetes Secrets
Deepgram strongly recommends following best practices for configuring Kubernetes Secrets. Refer to Securing Your Cluster for more details.
The deepgram-self-hosted Helm chart takes two Secret references. One is a set of distribution credentials that allow the cluster to pull images from Deepgram’s container image repository. The other is your self-hosted API key that licenses each Deepgram container that’s created.
-
Complete the Self Service Licensing & Credentials guide to generate distribution credentials and a self-hosted API key.
-
If you use an external Secret store provider, such as Azure Key Vault, configure cluster access to these two Secrets, and name them
dg-regcred(distribution credentials) anddg-self-hosted-api-key. -
If you don’t use an external Secret store provider, create the Secrets manually in your cluster.
-
Using the distribution credentials username and password generated in the Deepgram Console, create a Kubernetes Secret named
dg-regcred.ShellReplace the placeholders
QUAY_DG_USERandQUAY_DG_PASSWORDwith the distribution credentials you generated in the Self Service Licensing & Credentials guide. -
Create a Kubernetes Secret named
dg-self-hosted-api-keyto store your self-hosted API key.ShellReplace the placeholder
YOUR_API_KEY_HEREwith the Deepgram API key you generated in the Self Service Licensing & Credentials guide.
-
Deploy Deepgram
Deepgram maintains the official deepgram-self-hosted Helm chart. You can reference the source and Artifact Hub listing for more details. We’ll use this chart to deploy Deepgram services in your cluster.
-
Shell
-
Save the following as
my-values.yaml. It pins each component to the node pool you created, mounts the Azure Files claim as the model volume, and lets the chart install the NVIDIA GPU Operator. See the chart README for every available option.my-values.yaml -
Adjust the values to match your cluster.
- Set
scaling.replicas.apiandscaling.replicas.engineto the initial number of replicas. Keep the Engine replica count at or below the number of nodes inenginepool, because each Engine pod requests one GPU. - If you chose a VM with less memory than the example, lower the
requestsandlimitsso the pods fit on the node. - For Aura text-to-speech, each Engine needs two GPUs. Set
engine.resources.requests.gpuandengine.resources.limits.gputo2, and start from the Deploy TTS Services guide.
Node pool autoscaling, which you configured with the Azure CLI, is separate from pod autoscaling in the chart. To enable pod autoscaling, reach out to your Deepgram Account Representative to discuss which limits and values suit your traffic. See Autoscaling Best Practices.
- Set
-
Install the Helm chart with your values file.
ShellAllow time for the first install
The GPU Operator installs the NVIDIA driver on each GPU node, and AKS may need to provision a new node first. This can take several minutes. If a pod isn’t scheduled as you expect,kubectl describe pod <pod-name>lists the events that explain why.
Verify the GPU setup
Confirm that the cluster advertises your GPUs and that the installed driver meets Deepgram’s requirements.
-
List the GPUs Kubernetes can schedule on each Engine node.
Shell -
Check the driver version that the GPU Operator installed.
ShellThe
nvidia-smioutput should list your GPU and a driver version of580or later. The version string should readNVIDIA UNIX Open Kernel Module.
Test your Deepgram setup with a sample request
Test your Deepgram deployment on AKS with an audio file.
-
Launch an ephemeral pod to send your test request from.
Shell -
Inside the ephemeral pod, download a sample file from Deepgram, or supply your own file.
Shell -
Send your audio file to your Deepgram setup for transcription.
ShellIf needed, adjust pieces of the above command:
- the query parameters to match the directions from your Deepgram Account Representative
- the service name
deepgram-api-external - the namespace
dg-self-hosted
You should receive a JSON response with the transcript and associated metadata. You’re all set: your self-hosted setup is working.
Troubleshoot common AKS issues
For general Kubernetes issues, see Troubleshooting.
Next steps
Your Deepgram services are accessible within your cluster through the deepgram-api-external Service that the Helm chart created.
To accept requests from outside the cluster, expose the API with an Azure load balancer. In your my-values.yaml, set api.service.type to LoadBalancer. Add the service.beta.kubernetes.io/azure-load-balancer-internal: "true" annotation under api.service.annotations to keep the load balancer private to your virtual network. See Microsoft’s AKS load balancer documentation for details. Your installation automatically balances received requests across the cluster, so the load balancer serves as the ingress endpoint into the cluster.
To stop charges when you’re done testing, delete the resource group. This removes the cluster, node pools, and model storage.
What’s next
Now that you have a basic Deepgram setup working, take some time to learn about building up to a production-level environment, as well as helpful Deepgram add-on services.