Free API Key

License Proxy

Suggest Edits

For customers deploying Deepgram’s on-premises solution in production, Deepgram recommends the License Proxy Server, which is a caching proxy that communicates with the Deepgram-hosted license server to ensure uptime and simplify network security.

Use Cases

  • For production deployments, the License Proxy Server allows your deployed on-premises services to continue to run even if your deployment loses connectivity to the Deepgram license server.

  • If network security requirements dictate that traffic over the web is allowed from only certain hosts, the License Proxy Server can be deployed statically while ASR services scale elastically.

  • If your customers will deploy Deepgram software as part of your on-premises solution and their traffic must flow back to your environment, you may deploy the License Proxy Server to relay traffic on to the Deepgram license server.

Architecture Overview

An on-premises Deepgram implementation of the License Proxy Server functions in the following way:

  1. The on-premises Deepgram ASR services are configured to make their licensing requests against a hostname associated with a load balancer.

  2. The load balancer passes a licensing request to one of multiple instances of the License Proxy Server.

  3. In normal operation, the proxies will then pass on requests to the Deepgram-hosted license server. If the license server is unreachable for any reason, then the Deepgram License Proxy Server will allow the other Deepgram components to continue to run for a pre-configured amount of time.

📘

Deployment Recommendation

Deepgram recommends deploying onprem-license-proxy as a singleton for a given on-premises system.

System Requirements

For the License Proxy Server, we recommend at least 16 GB RAM. When monitoring, if usage is consistently approaching the 16 GB limit, then increase to 32 GB RAM.

Deploying the License Proxy Server

Before You Begin

If you aren't certain which products your contract includes or if you are interested in adding the License Proxy Server to your on-premises deployment, please consult a Deepgram Account Representative. To reach one, contact us!

You will also need an account on Docker Hub, so you can conveniently get the latest Deepgram software and updates.

Installing

Deepgram makes all of its products available through Docker Hub.

  1. Log in to your Docker Hub account from one of your servers.

  2. Run the following command:

    $ docker pull deepgram/onprem-license-proxy

Running the Proxy

  1. In your terminal, start an instance of the proxy:

    docker run deepgram/onprem-license-proxy:1.0.0 -v serve --license-key YOUR_LICENSE_KEY --host 0.0.0.0

    ℹ️

    You may also start the proxy with a container orchestrator of your choice, such as Docker Compose, Kubernetes, or Nomad.

  2. While your instance of the proxy is running, configure the on-prem ASR services to make their licensing requests through the proxy. To do this, in your api.toml and engine.toml files, add a line that specifies the URL to your deployed proxy immediately below your license key configuration:

    [license]
key = "YOUR_LICENSE_KEY"
server_url = "YOUR_PROXY_URL"

  3. Restart ASR services to begin directing licensing requests through the proxy. After restarting, test that an ASR request is processed as expected.

Monitoring

The License Proxy Server provides a status endpoint that indicates whether the proxy succeeded in relaying the most recent licensing request to the license server.

Querying the Status Endpoint

You can reach the status endpoint via port 8080 at the default /v1/status route. You can test this with a simple curl command (we use the command line utility jq to format the output):

curl http://localhost:8080/v1/status | jq

You should receive a response similar to:

{
    "state": "Connected",
    "last_successful_checkin": "2022-01-01T22:10:48.744573703Z",
    "trust_expiration": "2022-01-08T22:10:48.744573703Z"
}

Interpreting the Status Endpoint Response

Response fields include:

  • state: The proxy’s behavior based on the most recent licensing request. Possible values include:
    • Ready: Indicates that the proxy has started but has not yet relayed a licensing request.
    • Connected: Indicates that the most recent relay of a licensing request succeeded.
    • TrustBased: Indicates that the most recent relay of a licensing request failed, and the proxy returned a cached response.
    • Failed: The most recent relay of a licensing request failed, and the proxy is outside of the trust window.
  • last_successful_checkin: The timestamp of the last successful relay of a licensing request.
  • trust_expiration: The timestamp past which the proxy will no longer return a cached response for licensing requests that fail.

Setting Up Automated Alerts

If you use an automated metrics and alerting system, you should query the proxy’s status endpoint regularly (for example, once each minute) and set a warning when the state is TrustBased for a significant time period (for example, one hour). If you receive a warning, you can contact Deepgram support to troubleshoot the connection to the license server while knowing that deployed services will continue to run due to trust caching.

Updated 3 days ago