Flux Multilingual & Language Prompting

Flux Multilingual (flux-general-multi) is a single model supporting 10 languages with the same turn-aware, interruption-aware conversational intelligence as flux-general-en. The optional language_hint parameter biases the model toward specific languages, delivering accuracy on par with dedicated monolingual models. Without hints, the model auto-detects the spoken language.

Flux Multilingual uses the same production endpoint and API key you already use for Flux. Just set model=flux-general-multi — no new credentials or endpoints required. Pricing is the same as flux-general-en.

Supported Languages

The language_hint Parameter

language_hint string (optional, repeatable)

Pass one or more language_hint values to bias the model toward specific languages. This improves accuracy when you know the expected language(s) ahead of time.

Usage Scenarios

1. Known Single Language

When you know the caller’s language ahead of time (e.g., a Spanish-language call center), set a single language_hint for best accuracy.

wss://api.deepgram.com/v2/listen?model=flux-general-multi&language_hint=es&encoding=linear16&sample_rate=16000

2. Known Subset of Languages

When callers may speak one of several languages (e.g., a bilingual English/Spanish support line), pass multiple hints. The model biases toward the specified set while still producing accurate transcripts regardless of which language is spoken.

wss://api.deepgram.com/v2/listen?model=flux-general-multi&language_hint=en&language_hint=es&encoding=linear16&sample_rate=16000

3. Unknown Language

When you have no knowledge of what language the caller will speak, omit language_hint entirely. The model auto-detects the language from the audio.

wss://api.deepgram.com/v2/listen?model=flux-general-multi&encoding=linear16&sample_rate=16000

4. Code-Switching

When speakers switch between languages mid-conversation (e.g., a bilingual speaker mixing English and Spanish), set hints for the expected languages. Flux handles mid-sentence language switches natively.

wss://api.deepgram.com/v2/listen?model=flux-general-multi&language_hint=en&language_hint=es&language_hint=fr&encoding=linear16&sample_rate=16000

Language Detection in TurnInfo Events

When using flux-general-multi, all TurnInfo events include two additional fields:

Example TurnInfo Response

1
{
2
  "type": "TurnInfo",
3
  "request_id": "ad12514a-0d38-4f7e-8fba-cce10d8f174c",
4
  "sequence_id": 11,
5
  "event": "EndOfTurn",
6
  "turn_index": 0,
7
  "audio_window_start": 0,
8
  "audio_window_end": 1.3,
9
  "transcript": "Hello, how are you?",
10
  "languages_hinted": ["en", "es", "de"],
11
  "languages": ["en"],
12
  "words": [
13
    { "word": "Hello,", "confidence": 0.96 },
14
    { "word": "how", "confidence": 0.94 },
15
    { "word": "are", "confidence": 0.97 },
16
    { "word": "you?", "confidence": 0.92 }
17
  ],
18
  "end_of_turn_confidence": 0.86
19
}

Use the languages field to route downstream processing — for example, selecting the correct TTS voice or LLM prompt language based on what the user actually spoke.

Mid-Stream Reconfiguration

You can update language hints during a stream using the Configure control message without disconnecting. This is useful when conversational context changes — for example, after detecting the caller’s language, you can narrow the hints for better accuracy.

1
{
2
  "type": "Configure",
3
  "language_hints": ["en", "es"]
4
}

Pattern: Detect-then-Lock

A common voice agent pattern is to start a call with broad language detection, then lock in the detected language for the rest of the conversation. This gives you the best of both worlds: flexible auto-detection at the start and high-accuracy single-language transcription once the caller’s language is known.

How it works:

1. Connect with no hints (or a broad subset of expected languages) to let the model auto-detect.
2. Read the languages field from the first EndOfTurn event to identify the caller’s language.
3. Send a Configure message to lock in that language as a single hint for the remainder of the call.
4. Monitor for language changes — if a subsequent turn returns a different primary language in languages, send another Configure to update the hint.

Step 1 — Connect with broad detection:

wss://api.deepgram.com/v2/listen?model=flux-general-multi&encoding=linear16&sample_rate=16000

Or, if you know callers will speak one of a few languages, start with a subset:

wss://api.deepgram.com/v2/listen?model=flux-general-multi&language_hint=en&language_hint=es&language_hint=fr&encoding=linear16&sample_rate=16000

Step 2 — Read the detected language from the first EndOfTurn:

1
{
2
  "type": "TurnInfo",
3
  "event": "EndOfTurn",
4
  "transcript": "Hola, necesito ayuda con mi cuenta.",
5
  "languages": ["es"],
6
  "languages_hinted": [],
7
  ...
8
}

The first entry in languages is the primary language by word count.

Step 3 — Lock in the detected language:

1
{
2
  "type": "Configure",
3
  "language_hints": ["es"]
4
}

This biases all subsequent transcription toward Spanish, improving accuracy for the rest of the call.

Step 4 — Handle language switches (optional):

If a later turn returns a different primary language, update the hint:

1
{
2
  "type": "Configure",
3
  "language_hints": ["en"]
4
}

Error Handling

Example error response:

1
{
2
  "code": "INVALID_PARAMETER",
3
  "description": "language_hint is not supported for model flux-general-en"
4
}

Flux Multilingual vs. flux-general-en

flux-general-en remains available and recommended for English-only workloads. Use flux-general-multi when you need multilingual support or expect non-English audio. Both models share the same turn detection architecture, end-of-turn configuration, and control message interface.

Related Resources

• Getting Started with Flux — Quickstart guide
• End-of-Turn Configuration — Tune turn detection behavior
• Configure Control Message — Update settings mid-stream
• Multilingual Codeswitching — Codeswitching on Nova-2/Nova-3
• Models & Languages Overview — All models and supported languages