For AI agents: a documentation index is available at the root level at /llms.txt and /llms-full.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.
Ask AIPlaygroundLoginFree API Key
HomeAPI ReferenceVoice AgentSpeech-to-TextText-to-SpeechIntelligenceSelf-Hosted Deployments
HomeAPI ReferenceVoice AgentSpeech-to-TextText-to-SpeechIntelligenceSelf-Hosted Deployments
    • Getting Started with Speech to Text
  • Pre-Recorded Audio
    • Getting Started
    • Feature Overview
    • Template Apps
  • Streaming Audio
      • Getting Started
      • Feature Overview
      • Template Apps
      • End-of-Turn Configuration
      • Flux Multilingual & Language Prompting
      • Build a Flux-enabled Voice Agent
      • Why Flux's State Machine Matters
        • Optimize Voice Agent Latency with EagerEndOfTurn
    • Compare Flux to Nova-3
  • Models and Languages
    • Models & Languages Overview
    • Languages Support
    • Language Detection
    • Multilingual Codeswitching
    • Model Options
    • Version
  • Formatting
    • Speaker Diarization
    • Dictation
    • Filler Words
    • Measurements
    • Numerals
    • Paragraphs
    • Profanity Filtering
    • Punctuation
    • Redaction
    • Smart Formatting
    • Supported Entity Types
    • Utterances
    • Utterance Split
  • Custom Vocabulary
    • Find and Replace
    • Keyterm Prompting
    • Keywords
    • Search
  • Media Input Settings
    • Channels
    • Encoding
    • Multichannel
    • Sample Rate
  • Results Processing
    • Understanding Word Confidence Scores
    • STT Callback
    • STT Tagging
    • Extra Metadata
  • Migrating
    • Migrating From Amazon Web Services (AWS) Transcribe to Deepgram
    • Migrating From Google Speech-to-Text (STT) to Deepgram
    • Migrating From OpenAI Whisper to Deepgram
    • Migrating from AssemblyAI Speech-to-Text to Deepgram
LogoLogo
Ask AIPlaygroundLoginFree API Key
On this page
  • How Eager End of Turn Processing Works
  • Implementation Strategies
  • 1. Simple (Recommended Starting Point)
  • 2. Optimized (With Eager End of Turn Processing)
  • Tips & Tricks for Eager End of Turn Processing
  • Tune Confidence Thresholds
  • Handle TurnResumed Gracefully
  • Keep Responses Flexible
  • Optimize LLM Cost
  • Monitor and Log Events
  • Optimize Audio Input
  • Example Code
  • Summary
  • When to Use Eager End of Turn
  • When to use End of Turn only
Streaming AudioConversational STT for Voice Agents (Flux)Tips & Tricks

Optimize Voice Agent Latency with Eager End of Turn

Reduce end-to-end latency by preparing responses early with Eager End of Turn events.

Was this page helpful?
Previous

Getting Started

An introduction to getting transcription data from live streaming audio in real time.
Next
Built with

Eager end of turn processing is the practice of starting LLM processing on medium-confidence transcripts (EagerEndOfTurn events) before waiting for a high-confidence EndOfTurn. By overlapping LLM generation with user speech, you can cut hundreds of milliseconds from your agent’s response time.

Important Note

EagerEndOfTurn and TurnResumed events are ONLY triggered if you have configured the eager_eot_threshold in your connection string.

How Eager End of Turn Processing Works

  1. Receive EagerEndOfTurn

    • Flux is moderately confident the user has finished speaking.
    • Send the transcript downstream to your LLM and begin preparing a reply.

  2. If TurnResumed occurs

    • The user wasn’t finished after all.
    • Cancel the in-progress response and wait for the next EagerEndOfTurn or EndOfTurn.

  3. If EndOfTurn occurs

    • The user is done speaking with high confidence.
    • Finalize and deliver the response you’ve already started preparing.
    • EndOfTurn transcript will exactly match the EagerEndOfTurn transcript, ensuring consistent transcription throughout the turn lifecycle.

Implementation Strategies

1. Simple (Recommended Starting Point)

  • Use only EndOfTurn events.
  • Simplest implementation and minimal LLM calls
  • Ideal for majority of developers

2. Optimized (With Eager End of Turn Processing)

  • Use both EagerEndOfTurn and EndOfTurn events.
  • Reduce latency by preparing replies early with speculative response generation.
  • Expect more LLM calls and slightly more complexity.
  • Recommended once you’re confident in your pipeline and want production-grade performance.

Tips & Tricks for Eager End of Turn Processing

Tune Confidence Thresholds

  • eager_eot_threshold: Lower values → earlier triggers, but more false starts.
  • eot_threshold: Higher values → more reliable EndOfTurn, but may increase latency.
  • Experiment with values to balance speed vs. stability.

Handle TurnResumed Gracefully

  • Treat TurnResumed as a cancellation signal.
  • Be ready to discard or revise any LLM replies in progress.
  • Consider a retry strategy if this happens often in your use case.

Keep Responses Flexible

  • Avoid committing to a reply until EndOfTurn.
  • Use EagerEndOfTurn outputs to draft, not finalize.
  • Build resilience for handling TurnResumed events when the user continues speaking.

Optimize LLM Cost

  • Eager end of turn processing means more LLM requests.
  • To reduce spend:
    • Use smaller/faster models for EagerEndOfTurn drafts.
    • Only call the full LLM on EndOfTurn.
    • Cache prepared responses and reuse on EndOfTurn (transcript guaranteed to match).

Monitor and Log Events

  • Track how often EagerEndOfTurn → TurnResumed vs. EagerEndOfTurn → EndOfTurn.
  • Use this data to refine thresholds and tune your pipeline.

Optimize Audio Input

  • Echo and background noise can cause false StartOfTurn events, which disrupt the turn lifecycle. For guidance on echo cancellation, noise suppression, and barge-in strategies, see Audio Preprocessing & Barge-In.

Example Code

This code demonstrates how to handle Flux message events to implement eager end-of-turn processing. The examples show message parsing and event handling for the three critical events: -

  • EagerEndOfTurn (start preparing response)
  • TurnResumed (cancel draft response)
  • EndOfTurn (finalize and deliver response)
1ws.onmessage = (event) => {
2  const data = JSON.parse(event.data);
3  if (data.type !== 'TurnInfo') return;
4
5  switch (data.event) {
6    case 'EagerEndOfTurn':
7      console.log('EagerEndOfTurn:', data.transcript);
8      prepareDraftResponse(data.transcript);
9      break;
10
11    case 'TurnResumed':
12      console.log('User kept speaking, cancel draft');
13      cancelDraftResponse();
14      break;
15
16    case 'EndOfTurn':
17      console.log('Final:', data.transcript);
18      finalizeResponse(data.transcript);
19      break;
20  }
21};

Summary

When to Use Eager End of Turn

  • High-interruption environments (e.g., call centers, IVRs).
  • Conversational agents where natural back-and-forth timing matters.
  • Latency-sensitive apps where response speed is critical to user experience.
  • If your LLM configuration is complex and has high latency issues. e.g., Good for trimming that last 100-200ms of end-to-end latency at the cost of 50-70% more LLM calls.
  • If Your LLM configuration has complex RAG (Retrieval-Augmented Generation) or Function Calling involved.

When to use End of Turn only

  • Most developers will find EndOfTurn detection sufficiently fast enough to support natural conversation, but not all voice AI workflows are the same.
  • For more complex and expensive voice AI workflows, it might be worthwhile to use EagerEndOfTurnto call LLMs speculatively, i.e., in preparation for an upcoming turn end, in order to minimize response latency.