mistralai /
Voxtral-Mini-4B-Realtime-2602
Open speech-to-text model that transcribes 13 languages live, with under 500ms delay
- 1.0k likes
- 2.0M downloads a month
- 4.4B parameters
- Some experience
- Speech
This model listens to audio and writes out the words as they are spoken. It has two parts: a language model of about 3.4B parameters and an audio encoder of about 970M parameters. The encoder was trained from scratch to process audio in a streaming way. The card says it is among the first open-source models to reach accuracy comparable to offline systems with a delay under 500ms. It supports English, French, Spanish, German, Russian, Chinese, Japanese, Italian, Portuguese, Dutch, Arabic, Hindi and Korean.
You choose how much delay to accept. A shorter delay is faster but makes more mistakes, and a longer one is slower but more accurate. In the card's Fleurs table the average error rate is 12.60% at 160ms, 8.72% at 480ms and 6.73% at 2400ms. At 480ms it is close to, but slightly behind, Mistral's offline Voxtral Mini Transcribe 2.0 (8.72% vs 5.90% on the Fleurs average). Results vary a lot by language: Arabic (22.53%) and Korean (15.74%) are much higher than Spanish (3.31%) at 480ms. The setup is also not beginner-friendly. The card's main path uses vLLM and says a single GPU with 16GB or more memory is needed. The card also says the architecture is currently only supported in vLLM, apart from a Transformers route (version 5.2.0 or later) and some community ports that are marked untested.
What you need to run it
4.4B parameters. The weights take about 9 GB at full 16-bit precision (fits a 16 GB Mac or a GPU with 16 GB), or about 3 GB as a 4-bit download, which runs on a 8 GB Mac or a GPU with 8 GB. Long inputs and images need more memory on top.
What it does well
Live transcription with a small delay
It is built to stream, with a custom causal audio encoder. At 480ms delay the card says it matches leading offline open-source transcription models and realtime APIs.
You set the latency/accuracy trade-off
The delay is configurable. The card recommends 480ms as the sweet spot, and says you can change "transcription_delay_ms" in tekken.json to a multiple of 80ms between 80 and 1200, or to 2400.
13 languages in one model
English, French, Spanish, German, Russian, Chinese, Japanese, Italian, Portuguese, Dutch, Arabic, Hindi and Korean are all covered. Fleurs scores are given per language, so you can check your own before building.
Designed for long sessions and modest hardware
Sliding window attention allows "infinite" streaming in principle. In practice the default maximum model length of 131072 covers roughly 3 hours. The card describes it as optimized for on-device deployment, and the vLLM route needs a single GPU with at least 16GB.
Apache 2.0 weights in BF16
The weights are released under the Apache 2.0 licence, so research and commercial use are both allowed.
What you could build
- Build a live subtitle tool that shows captions while someone speaks
- Make a private meeting transcriber that runs on your own machine instead of a cloud service
- Add a speech front end to a voice assistant that needs to react while the user is still talking
- Compare transcription accuracy across delay settings (for example 240ms vs 480ms vs 2400ms) in a language you speak
- Stream an audio file to a locally served model through vLLM's realtime endpoint, using the card's example clients
Get started
Install vLLM and the audio libraries, then serve the model; it exposes a /v1/realtime endpoint. The card then points to example clients for streaming audio files and a Gradio live microphone demo, and there is also a hosted demo on Hugging Face Spaces.
uv pip install -U vllmuv pip install soxr librosa soundfileVLLM_DISABLE_COMPILE_CACHE=1 vllm serve mistralai/Voxtral-Mini-4B-Realtime-2602 --compilation_config '{"cudagraph_mode": "PIECEWISE"}'Commands are copied from the model card. Read it on Hugging Face for the full setup.
More speech models
Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice
A 1.7B text-to-speech model with 9 preset voices and style control, in 10 languages.
microsoft/Phi-4-multimodal-instruct
A small open model that reads text, images and speech, and answers in text.
nvidia/parakeet-tdt-0.6b-v2
A 600M-parameter English speech-to-text model with punctuation, capitals and word timestamps.
Qwen/Qwen3-ASR-1.7B
An open speech-to-text model that transcribes 52 languages and dialects, with optional timestamps.
CohereLabs/cohere-transcribe-03-2026
A 2B-parameter open model that turns speech in 14 languages into written text.
openai/whisper-small
OpenAI's 244M-parameter model that turns speech into text and translates it into English.