Using Private Container Registries

Background

Deepgram's self-hosted offering is powered by a set of container images. Access to these images is provided through Quay, and you may choose to have your self-hosted deployment pull images directly from this source.

Another option is to use a private container registry. Most cloud providers offer a managed private container registry solution, such as AWS' ECR or GCP's Artifact Registry. You can also self-host using tools like Harbor or zot. The typical workflow for using a private container registry includes a one-time pull of relevant containers from Quay and subsequent push to your private registry. Your self-hosted deployment can then use the private registry to create new containers as needed.

Use Cases

Performance

Customers who use auto-scaling in their environment will frequently spin up new nodes and new containers in an automated fashion to meet demand. Each time a container is deployed, a container image must be pulled from either a local container image cache or a container registry to instantiate the workload.

When replicating pods on a single node, the container runtime on the node will generally handle this for the operator transparently by referencing the local container image cache. When deploying new nodes, however, there are no local copies of the container images in the Container Image Cache, so the container images must be pulled from a remote container registry.

The network connection to an external container registry, such as Quay, will typically have lower bandwidth than an internal, private container registry. Using a private container registry can reduce the startup time for containers on newly allocated nodes by up to several minutes. This can improve your environment's reaction time when scaling to meet increased demand.

Security

Best practices for self-hosting software include only deploying trusted container images. Deepgram runs regular security scans of our container images and promptly patches vulnerabilities as part of our Security Policy, but many environments can harden their security stance with the additional layers of security provided by a private container registry. Security benefits include:

Increased version control
Easier to run your own security scans on all container images you will use in your environment
Improved network security
- Your private registry should be accessible from within your environment's private network, and your container orchestrator will no longer need to request images directly from Quay
- Therefore, you can adjust your firewall to deny-list all egress traffic to the public internet (with the exception of license.deepgram.com)

Guide Structure

This guide provides commands for creating private container registries on certain cloud platforms and updating your Deepgram self-hosted configuration to utilize the private container registry.

The list of supported cloud platforms is not exhaustive; if your environment is not covered by these examples, you can use the general principles to extrapolate to your platform of choice.

This guide gives explicit commands for hosting the Deepgram API container image in a private registry. You will likely want to also complete this guide for then Engine and License Proxy (if applicable) container images. You will need to adjust the CLI parameters in a few steps, but the core commands will be the same.

Pulling Images from Quay

Before setting up your private registry, you need to pull the necessary Deepgram container images from Quay.

Generate credentials to authenticate with Quay by following this guide section.
Login to Quay:
```
docker login quay.io
```
Identify the latest self-hosted release in the Deepgram Changelog. Filter by "Self-Hosted", and select the latest release.
Pull the relevant container images. This guide will focus on the API container:
```
export RELEASE_TAG=<LATEST_RELEASE_TAG>
docker pull quay.io/deepgram/self-hosted-api:$RELEASE_TAG
```
💻
Replace <LATEST_RELEASE_TAG> with the tag of the latest release from the Changelog.

Platform-Specific Registry Setup

Complete one of the following subsections, depending on your cloud platform of choice and whether you prefer to use a Cloud Console or CLI tools.

AWS

Using the AWS Console

In the AWS Console, navigate to the Elastic Container Registry page. Select Repositories in the left navigation menu, then click Create repository.
Keep the visibility settings as Private. Choose a repository name, such as deepgram-self-hosted-api. Modify any additional settings as needed, then click Create repository.
On the main ECR page, copy the URI of your newly created registry, and export it in your local terminal:
```
export API_REPO_URI=<YOUR_API_REPO_URI>
```
💻
Replace <YOUR_API_REPO_URI> with the URI from the AWS Console.

Using the AWS CLI

Install the AWS CLI. See here for a step-by-step installation guide.

Once authenticated, get your AWS Account ID:

AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query 'Account' --output text)

Export the AWS region for your self-hosted environment:

export AWS_REGION="us-west-2" # Replace if using a different region

Authenticate to your default registry:

aws ecr get-login-password --region "$AWS_REGION" | \
docker login \
   --username AWS \
   --password-stdin \
   "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com"

Create a repository for each container image you pulled from Quay:

API_REPO_NAME="deepgram-self-hosted-api"
API_REPO_URI=$(aws ecr create-repository \
   --repository-name "$API_REPO_NAME" \
   --region "$AWS_REGION" \
   --output text \
   --query 'repository.repositoryUri')

GCP

The following steps are a summary of this official GCP documentation. See their guides for the most up-to-date instructions.

Install the gcloud CLI, then run gcloud init to configure the CLI with access to your GCP account and project.

Create a container image repository:

export GCP_REGION="us-west1"
export GCP_PROJECT="<YOUR_GCP_PROJECT_ID>"
gcloud artifacts repositories create deepgram \
    --repository-format=docker \
    --location="$GCP_REGION" \
    --description="Private repository for storing Deepgram container images" \
    --project="$GCP_PROJECT"

💻
Replace <YOUR_GCP_PROJECT_ID> with your Google Cloud project ID.

Authenticate your local Docker agent with your new GCP container registry:

gcloud auth configure-docker "$GCP_REGION-docker.pkg.dev"
# If you use `root` to access docker, i.e. `sudo docker...`, make sure
# to run this command with `sudo` as well
# sudo gcloud auth configure-docker "$GCP_REGION-docker.pkg.dev"

API_REPO_URI="$GCP_REGION-docker.pkg.dev/$GCP_PROJECT/deepgram/self-hosted-api"

Pushing Images to Private Registry

Push your local copy of the Deepgram container, previously pulled from Quay, to your private registry.

docker tag \
   "quay.io/deepgram/self-hosted-api:$RELEASE_TAG" \
   "$API_REPO_URI:$RELEASE_TAG"

docker push "$API_REPO_URI:$RELEASE_TAG"

Configuring Deployment to Use Private Registry

Complete one of the following subsections, depending on your cloud platform of choice and container orchestrator used in your deployment.

Docker/Podman

AWS

You will need an existing EC2 instance, which can be setup using the AWS with Docker/Podman, Drivers and Container Orchestration Tools, and Deploy Deepgram Services guides. We will be modifying this instance with a new IAM role to allow pulling images from your ECR registry.
Create a new IAM role for EC2 to access ECR:
1. In the AWS Console, go to EC2 -> Instances -> Your Instance -> Actions -> Security -> Modify IAM Role.
2. Create a new role with the AmazonEC2ContainerRegistryReadOnly policy.
3. Apply this new role to your EC2 instance.
SSH into your EC2 instance, and install the amazon-ecr-credential-helper on your EC2 instance.
Configure Docker to use the credential helper by setting the contents of ~/.docker/config.json:
```
{
  "credsStore": "ecr-login"
}
```
💻
You do not need to import additional AWS credentials, such as outlined in the AWS credentials section of the README. The credentials will be automatically pulled from the Role you configured on the EC2 instance.

GCP

You will need an existing Compute Engine instance, which can be setup using the GCP with Docker/Podman, Drivers and Container Orchestration Tools, and Deploy Deepgram Services guides.
1. Google Compute Engine instances with default IAM settings should have access to pull images from your Artifact Registry by default. You will just need to configure your Docker client to use the instance's default credentials.
Authenticate Docker with your Artifact Registry on your Google Compute Engine instance:
```
gcloud auth configure-docker "$GCP_REGION-docker.pkg.dev"
```

Kubernetes

AWS

EKS clusters created using the Deepgram AWS EKS guide should have access to pull images from your private container registry by default.

GCP

GKE clusters with default IAM settings should have access to pull images from your Artifact Registry by default.

Updating your Deepgram Deployment

Docker/Podman

Modify your Compose files to use images from your private container registry:
```
# docker-compose.yaml or podman-compose.yaml
services:
  api:
    image: YOUR_PRIVATE_CONTAINER_PATH
```
💻
Replace YOUR_PRIVATE_CONTAINER_PATH with the output of:
```
echo "$API_REPO_URI:$RELEASE_TAG"
```

Restart your containers to use the new container image:

# Docker
docker compose up -d
# Podman
podman-compose up -d

Kubernetes

Modify your values.yaml file for the deepgram-self-hosted Helm chart to use the new image path:
```
api:
  image:
    path: IMAGE_PATH
    tag: IMAGE_TAG
```
💻
Replace IMAGE_PATH and IMAGE_TAG with the output of:
```
echo "$API_REPO_URI"
echo "$RELEASE_TAG"
```

Upgrade your Helm installation to use the new container images:

helm upgrade -f my-values.yaml [RELEASE_NAME] deepgram/deepgram-self-hosted --atomic --timeout 60m