Using Private Container Registries
Background
Deepgram’s self-hosted offering is powered by a set of container images. Access to these images is provided through Quay, and you may choose to have your self-hosted deployment pull images directly from this source.
Another option is to use a private container registry. Most cloud providers offer a managed private container registry solution, such as AWS’ ECR or GCP’s Artifact Registry. You can also self-host using tools like Harbor or zot. The typical workflow for using a private container registry includes a one-time pull of relevant containers from Quay and subsequent push to your private registry. Your self-hosted deployment can then use the private registry to create new containers as needed.
Use Cases
Performance
Customers who use auto-scaling in their environment will frequently spin up new nodes and new containers in an automated fashion to meet demand. Each time a container is deployed, a container image must be pulled from either a local container image cache or a container registry to instantiate the workload.
When replicating pods on a single node, the container runtime on the node will generally handle this for the operator transparently by referencing the local container image cache. When deploying new nodes, however, there are no local copies of the container images in the Container Image Cache, so the container images must be pulled from a remote container registry.
The network connection to an external container registry, such as Quay, will typically have lower bandwidth than an internal, private container registry. Using a private container registry can reduce the startup time for containers on newly allocated nodes by up to several minutes. This can improve your environment’s reaction time when scaling to meet increased demand.
Security
Best practices for self-hosting software include only deploying trusted container images. Deepgram runs regular security scans of our container images and promptly patches vulnerabilities as part of our Security Policy, but many environments can harden their security stance with the additional layers of security provided by a private container registry. Security benefits include:
-
Increased version control
-
Easier to run your own security scans on all container images you will use in your environment
-
Improved network security
- Your private registry should be accessible from within your environment’s private network, and your container orchestrator will no longer need to request images directly from Quay
- Therefore, you can adjust your firewall to deny-list all egress traffic to the public internet (with the exception of
license.deepgram.com
)
Guide Structure
This guide provides commands for creating private container registries on certain cloud platforms and updating your Deepgram self-hosted configuration to utilize the private container registry.
The list of supported cloud platforms is not exhaustive; if your environment is not covered by these examples, you can use the general principles to extrapolate to your platform of choice.
This guide gives explicit commands for hosting the Deepgram API container image in a private registry. You will likely want to also complete this guide for then Engine and License Proxy (if applicable) container images. You will need to adjust the CLI parameters in a few steps, but the core commands will be the same.
Pulling Images from Quay
Before setting up your private registry, you need to pull the necessary Deepgram container images from Quay.
-
Generate credentials to authenticate with Quay by following this guide section.
-
Login to Quay:
Shell -
Identify the latest self-hosted release in the Deepgram Changelog. Filter by “Self-Hosted”, and select the latest release.
-
Pull the relevant container images. This guide will focus on the API container:
ShellReplace
<LATEST_RELEASE_TAG>
with the tag of the latest release from the Changelog.
Platform-Specific Registry Setup
Complete one of the following subsections, depending on your cloud platform of choice and whether you prefer to use a Cloud Console or CLI tools.
AWS
Using the AWS Console
-
In the AWS Console, navigate to the Elastic Container Registry page. Select
Repositories
in the left navigation menu, then clickCreate repository
. -
Keep the visibility settings as
Private
. Choose a repository name, such asdeepgram-self-hosted-api
. Modify any additional settings as needed, then clickCreate repository
. -
On the main ECR page, copy the URI of your newly created registry, and export it in your local terminal:
ShellReplace
<YOUR_API_REPO_URI>
with the URI from the AWS Console.
Using the AWS CLI
-
Install the AWS CLI. See here for a step-by-step installation guide.
-
Once authenticated, get your AWS Account ID:
Shell -
Export the AWS region for your self-hosted environment:
Shell -
Authenticate to your default registry:
Shell -
Create a repository for each container image you pulled from Quay:
Shell
GCP
The following steps are a summary of this official GCP documentation. See their guides for the most up-to-date instructions.
-
Install the gcloud CLI, then run
gcloud init
to configure the CLI with access to your GCP account and project. -
Create a container image repository:
ShellReplace
<YOUR_GCP_PROJECT_ID>
with your Google Cloud project ID. -
Authenticate your local Docker agent with your new GCP container registry:
Shell
Pushing Images to Private Registry
Push your local copy of the Deepgram container, previously pulled from Quay, to your private registry.
Configuring Deployment to Use Private Registry
Complete one of the following subsections, depending on your cloud platform of choice and container orchestrator used in your deployment.
Docker/Podman
AWS
-
You will need an existing EC2 instance, which can be setup using the AWS with Docker/Podman, Drivers and Container Orchestration Tools, and Deploy Deepgram Services guides. We will be modifying this instance with a new IAM role to allow pulling images from your ECR registry.
-
Create a new IAM role for EC2 to access ECR:
- In the AWS Console, go to
EC2 -> Instances -> Your Instance -> Actions -> Security -> Modify IAM Role
. - Create a new role with the
AmazonEC2ContainerRegistryReadOnly
policy. - Apply this new role to your EC2 instance.
- In the AWS Console, go to
-
SSH into your EC2 instance, and install the
amazon-ecr-credential-helper
on your EC2 instance. -
Configure Docker to use the credential helper by setting the contents of
~/.docker/config.json
:JSONYou do not need to import additional AWS credentials, such as outlined in the AWS credentials section of the README. The credentials will be automatically pulled from the Role you configured on the EC2 instance.
GCP
-
You will need an existing Compute Engine instance, which can be setup using the GCP with Docker/Podman, Drivers and Container Orchestration Tools, and Deploy Deepgram Services guides.
- Google Compute Engine instances with default IAM settings should have access to pull images from your Artifact Registry by default. You will just need to configure your Docker client to use the instance’s default credentials.
-
Authenticate Docker with your Artifact Registry on your Google Compute Engine instance:
Shell
Kubernetes
AWS
EKS clusters created using the Deepgram AWS EKS guide should have access to pull images from your private container registry by default.
GCP
GKE clusters with default IAM settings should have access to pull images from your Artifact Registry by default.
Updating your Deepgram Deployment
Docker/Podman
-
Modify your Compose files to use images from your private container registry:
yamlReplace
YOUR_PRIVATE_CONTAINER_PATH
with the output of:Shell -
Restart your containers to use the new container image:
Shell
Kubernetes
-
Modify your
values.yaml
file for thedeepgram-self-hosted
Helm chart to use the new image path:yamlReplace
IMAGE_PATH
andIMAGE_TAG
with the output of:Shell -
Upgrade your Helm installation to use the new container images:
Shell
What’s Next
Now that you’ve modified your self-hosted deployment to use images from a private container registry, you can explore other guides on scaling and maintaining your deployment.