Suits neuroscience and machine learning researchers or students comfortable with Python and PyTorch, who know what fMRI and cortical surface meshes are.

What can you build with tribev2?

Predict the cortical response to a video clip and plot it on a brain surface. Compare predicted brain activity for the same content as speech, text or video. Run in-silico experiments on stimuli before designing a scanner study. Train a brain encoding model from scratch on public fMRI datasets such as Algonauts2025. Reproduce the results of the TRIBE v2 paper using the released weights and code.

facebookresearch /

tribev2

Predicts fMRI brain responses to video, audio and text, with pretrained weights

3.3k stars
55 stars since 3 Oct
Jupyter Notebook
Updated 4 months ago
Advanced
Models & local AI

Open on GitHub (opens in a new tab)

TRIBE v2 is a brain encoding model. You give it a video, an audio clip or a piece of text, and it predicts how a person's brain would respond, as fMRI activity over time. This lets neuroscientists test ideas in software before, or instead of, running a scanner.

It combines existing text, audio and video models into one Transformer. The Transformer maps their features onto the cortical surface, using the fsaverage5 mesh of about 20,000 vertices. Predictions are for an average subject and are shifted 5 seconds back to account for the delay in blood-flow response. Text input is turned into speech and transcribed, so words get timings.

The README does not compare it to other tools. It does ship pretrained weights on Hugging Face, a Colab demo, a paper, and the code to train from scratch on studies such as Algonauts2025 and Lahner2024. The license is CC BY-NC 4.0, so commercial use is not allowed.

What it does

Predicts brain responses
Takes video, audio or text and returns an array of shape (timesteps, vertices) on the fsaverage5 cortical surface.
Pretrained weights
Load facebook/tribev2 from Hugging Face with TribeModel.from_pretrained and run inference without training.
Text and audio input
Pass text_path or audio_path as well as video. Text is converted to speech and transcribed for word-level timings.
Brain visualization
An optional plotting extra draws predictions with PyVista or Nilearn, and the Colab notebook walks through it.
Training code included
PyTorch Lightning modules and grid scripts let you train cortical and subcortical models, locally or on Slurm.

What you could build

Predict the cortical response to a video clip and plot it on a brain surface.
Compare predicted brain activity for the same content as speech, text or video.
Run in-silico experiments on stimuli before designing a scanner study.
Train a brain encoding model from scratch on public fMRI datasets such as Algonauts2025.
Reproduce the results of the TRIBE v2 paper using the released weights and code.

Get started

Install the package, then load facebook/tribev2 with TribeModel.from_pretrained, build an events dataframe from a video, and call predict. The Colab notebook is the easiest way to see it with brain plots.

pip install -e .

pip install -e ".[plotting]"

Commands are copied from the project's README. Read it on GitHub for the full setup.