huggingface /
transformers
Shared model definitions for text, vision, audio and multimodal AI models
- 168k stars
- Python
- Updated today
- Some experience
- Models & local AI
Machine learning models come in many architectures, and research teams often publish their own code. Transformers collects those model definitions in one library with a common interface. You can load a model from the Hugging Face Hub and use it without learning a new codebase each time.
The quickest way in is the Pipeline class. You name a task, such as text generation or speech recognition, and a model. It downloads the model, prepares the input, runs it and returns the result. For more control, the library exposes model internals and includes a training API for PyTorch models.
The README says these model definitions are meant to be the agreed reference across the ecosystem. Training frameworks (Axolotl, Unsloth, DeepSpeed, FSDP, PyTorch-Lightning), inference engines (vLLM, SGLang, TGI) and libraries like llama.cpp and mlx reuse them. It says it is not a modular toolbox of neural net building blocks, and for generic training loops it points to Accelerate.
What it does
Pipeline API
A high-level class that handles preprocessing and output for text, audio, vision and multimodal tasks. The README shows text generation, chat, speech recognition, image classification and visual question answering.
Hub of pretrained checkpoints
Works with over a million checkpoints on the Hugging Face Hub, across hundreds of architectures. Models are downloaded and cached on first use.
Wide modality coverage
Example models include Whisper and Parakeet for speech, MusicGen for audio, SAM and DINO v2 for vision, and Qwen and Llama for text.
Chat from the command line
With transformers serve running, the transformers chat command lets you talk to a model directly from the terminal.
Readable, customizable model code
Model files are deliberately not wrapped in extra abstractions, so researchers can edit one model without digging through shared layers. Examples reproduce results published by the original authors.
What you could build
- Transcribe audio files with a speech recognition model such as Whisper.
- Run a chat model locally and build a simple assistant on top of it.
- Classify images or answer questions about them with a pretrained vision model.
- Fine-tune a pretrained model on your own data using the training API.
- Read a model's source code to learn how a modern architecture is put together.
Get started
Create a virtual environment with Python 3.11 or newer and install the library with the torch extra. Then build a pipeline for a task and a model name and pass it some input.
pip install "transformers[torch]"uv pip install "transformers[torch]"transformers chat Qwen/Qwen2.5-0.5B-InstructCommands are copied from the project's README. Read it on GitHub for the full setup.
Anything better than transformers?
Transformers is the default pick for running or fine-tuning pretrained models across text, vision, audio and multimodal tasks, and nothing here competes with it as a general library. Look elsewhere only for a narrow task, such as speech transcription.
More in Models & local AI
open-webui
Self-hosted chat interface for Ollama and OpenAI-compatible models, with RAG and plugins
odysseus
Self-hosted workspace for AI chat, agents, research, documents, email and calendar
minimind
Train a tiny 64M-parameter language model from scratch on one consumer GPU
Kronos
Pre-trained Transformer models that forecast candlestick (K-line) price data
airllm
Run very large open language models on a small GPU by loading one layer at a time
Vane
Self-hosted AI answer engine that searches the web and cites its sources
timesfm
Pretrained Google model that forecasts time series without training on your data
Unlimited-OCR
Open OCR model that turns images and multi-page PDFs into structured text in one pass
gallery
Android and iOS app for running open LLMs offline on your phone
lingbot-map
Feed-forward model that builds 3D scenes from video frames as they stream in