# whisper

> Speech recognition model that transcribes and translates audio in many languages

- Repository: https://github.com/openai/whisper
- Category: Image, video & audio
- Stars: 111,093 · Forks: 13,452 · Language: Python · License: MIT
- Experience: Beginner friendly
- Page: https://ir.adyog.com/repos/openai/whisper/

Whisper turns spoken audio into text. It is a single Transformer model trained on a large, varied set of audio. One model handles several jobs: multilingual speech recognition, speech translation, language identification and voice activity detection.

It splits audio into 30-second windows and predicts text token by token for each window. Special tokens tell the decoder which task to do, such as transcribe or translate. This lets one model replace several stages of a traditional speech-processing pipeline.

You pick a model size to fit your hardware. Tiny needs about 1 GB of VRAM and runs fast. Large needs about 10 GB and is more accurate. Turbo is an optimized large-v3 that is faster with a small loss in accuracy. Code and weights are under the MIT license.

## What it does

- **Multilingual transcription**: Transcribes speech in many languages. You can name the language or let the model detect it. Accuracy varies by language, and the README shows per-language error rates.
- **Translation into English**: With --task translate and a multilingual model such as medium or large, non-English speech comes out as English text. The turbo model is not trained for translation.
- **Six model sizes**: Models range from tiny (39M parameters) to large (1550M). Four sizes have English-only versions that often do better on English.
- **Command line and Python API**: The whisper command transcribes one or more audio files. In Python, load_model and transcribe return the text. detect_language and decode give lower-level control.

## What you could build

- Transcribe interviews, lectures or podcasts into text files.
- Translate recordings of non-English speech into English text.
- Detect which language is spoken in an audio clip.
- Build a Python app that adds speech-to-text to your own tool.
- Compare model sizes to find the best speed and accuracy for your hardware.

## Who it's for

Suits developers and students who want speech-to-text and can use a terminal or basic Python; you need ffmpeg installed, and a GPU helps for larger models.

## Get started

Install the package with pip and install ffmpeg with your system's package manager (the brew command is for macOS). Then run the whisper command on an audio file. You may need Rust if no prebuilt tiktoken package exists for your platform.

```
pip install -U openai-whisper
```
```
brew install ffmpeg
```
```
whisper audio.flac audio.mp3 audio.wav --model turbo
```

Tags: speech-recognition, transcription, translation, openai, pytorch