Docker/PodmanPlatform Options

Amazon Web Services

With Docker/Podman

Deploying Deepgram on Amazon Web Services (AWS) requires some preparation. In this section, you will learn how to provision a Virtual Machine where you will deploy Deepgram products. You will need to perform some of these steps in the AWS Management Console and some in your local terminal.

Prerequisites

Make sure you have completed the requirements in the Self-Hosted Introduction.

Accessing Your Cloud Environment

AWS uses public-key cryptography to secure login information for your instance. A Linux instance has no password; you use a key pair to log in to your instance securely. For the following steps, be sure to use a non-root AWS account.

Create an Amazon EC2 Key Pair

If you don’t already have an Amazon EC2 key pair, you will need to create one in order to access the AWS EC2 Virtual Machine. To learn how, read Create a key pair using Amazon EC2 in Amazon’s documentation.

Key pairs must be created in each AWS region in which you will deploy Deepgram self-hosted products.

At the end of this process, your browser should download a private-key.pem file for your key pair. Move this file to a secure and memorable location.

Create an Amazon EC2 Instance

​To begin your self-hosted installation with AWS as your cloud service provider, you need to create an Amazon EC2 instance.

The steps in this section utilize the new AWS launch experience. You may need to adjust the ordering if you are using the old AWS launch experience.

  1. Navigate to the EC2 Dashboard and confirm that the proper AWS Region is configured, then choose Launch Instance to open the wizard.

  2. For the Name and tags wizard step, type Deepgram Self-hosted, or another appropriate description of this deployment.

  3. For the Application and OS Images (Amazon Machine Image) wizard step, choose a basic configuration to serve as a template for your instance:

    1. Pick an Amazon Machine Image (AMI) from our list of recommended Linux distributions.
    2. Search for your desired configuration, such as Ubuntu Server 22.04 LTS (HVM), SSD Volume Type
    3. Ensure the 64-bit (x86) option is selected.
    4. Choose Select.

  4. For the Choose an Instance Type wizard step, search and select g6.2xlarge, or another GPU instance type that meets our minimum required hardware specifications.

  5. For the Key pair (login) wizard step, search and select the key pair you created in the Create an Amazon EC2 Key Pair section. Alternatively, you can select Create new key pair; ensure you download the private key and configure it properly on your local machine in order to access your server.

  6. For the Network settings wizard step, select Create security group, check the box Allow SSH traffic from anywhere, and ensure Auto-assign public IP is enabled. This will allow you to remotely login to your server.

If you want to be able to receive requests from the public internet, make sure to also check the box Allow HTTPS traffic from the internet. If you are colocating your Deepgram self-hosted deployment with other services and do not need to expose your server to the public internet, you can find configuration options on the Amazon networking documentation.

  1. For the Configure storage wizard step, change the root volume size to meet our minimum required hardware specifications.
  2. Verify that the summary box contains all the right details for your instance, and click Launch Instance.
  3. Once the instance successfully launches, you need to find its Public IPv4 DNS entry. Either select the link to the instance and find the Public IPv4 DNS column, or click on Connect to instance and follow the instructions under the SSH client tab. This will be the AWS_HOSTNAME value that is referenced later.

Log in to the AWS EC2 instance

To complete the rest of the installation, including configuring your environment and transferring files between your local computer and your AWS instance, you must connect to the AWS EC2 instance that you launched.

  1. Open the terminal application for your computer.

  2. Connect to your AWS instance:

    Shell
    $ssh -i /path/to/private-key.pem ubuntu@AWS_HOSTNAME

Be sure to replace the AWS_HOSTNAME placeholder value with the hostname for your instance. Also check that the path to your private-key.pem file is correct.

For AWS Ubuntu instances, the default username is ubuntu, not user as their documentation would suggest. If you are using a different AMI, the default username will be different - see the Connect page on the instance for more details.

If you are on a Windows machine, alternatives to ssh include PuTTY and the AWS EC2 Instance Connect service.

  1. If you receive a message that indicates that the authenticity of the host can’t be established, type yes, then press the Enter key on your keyboard. This message should only appear the first time you are connected to the instance.

What’s Next

Now that we have provisioned a deployment environment, we need to start configuring it for inference.