mutonby /
openshorts
Self-hosted tool that cuts long videos into 9:16 shorts with captions and dubbing
- 7k stars
- Python
- Updated 2 days ago
- Some experience
- Image, video & audio
Making short vertical clips from a long recording is slow work. You find the good moments, reframe the picture for a phone, add captions and post it. OpenShorts does those steps in one dashboard. You upload a video, and it transcribes it, picks 3 to 15 moments, crops each one to 9:16 and burns in subtitles.
Google Gemini reads the transcript and scene boundaries to choose moments. faster-whisper makes word-level subtitles, and MediaPipe and YOLOv8 track faces for the crop. There are layouts for one speaker, two speakers stacked with captions between them, and screen shares. If you don't want a Google key, you can point it at a local model such as Ollama through an OpenAI-compatible URL.
The README compares it to hosted clip tools like Opus Clip, Vizard and Klap. The main difference is that you can self-host it under the MIT license with no watermark or upload limit. It also bundles an AI-actor video maker and a YouTube toolkit. The same software is offered hosted at openshorts.app, with GPU speed, from $12 a month.
What it does
Moment detection
Gemini reads the transcript and scene boundaries to find 3 to 15 clip-worthy moments. A local LLM can do this step instead.
Layout-aware 9:16 cropping
Chooses face tracking, a blurred-background fit, a stacked two-speaker layout or a screencast layout per video. You can also force one from the dashboard.
Subtitles, dubbing and hooks
Word-level subtitles come from faster-whisper. ElevenLabs dubbing covers 30+ languages with voice cloning, and AI-written hook text can be overlaid on clips.
AI Shorts with AI actors
Builds marketing videos from a product description or URL: script, AI actor, voiceover, lip-sync, b-roll and subtitles, assembled with FFmpeg.
YouTube Studio and publishing
Generates titles, thumbnails and chaptered descriptions. Posts to TikTok, Instagram Reels and YouTube Shorts, with scheduling, through Upload-Post.
What you could build
- Cut a podcast or webinar into a batch of captioned vertical clips for TikTok, Reels and Shorts.
- Dub a clip into another language using a cloned voice.
- Generate a product marketing video with an AI actor from a website URL.
- Create titles, thumbnails and chapter timestamps for a YouTube upload.
- Run the clip pipeline on your own machine with a local LLM, keeping footage private.
Get started
Clone the repo, optionally copy .env.example to .env, then start it with Docker Compose. Open http://localhost:5175, enter your API keys under Settings, and upload a video to the Clip Generator.
git clone https://github.com/mutonby/openshorts.git
cd openshortscp .env.example .envdocker compose up --buildCommands are copied from the project's README. Read it on GitHub for the full setup.
More in Image, video & audio
ComfyUI
Node-graph app for generating images, video, audio and 3D with local AI models
MoneyPrinterTurbo
Turns a topic or keyword into a finished short video with script, voice and subtitles
whisper
Speech recognition model that transcribes and translates audio in many languages
VoiceStudio
Desktop app for cloning voices, dubbing video and dictating, run on your own machine
OpenMontage
Video production pipelines your AI coding assistant runs from a plain-language prompt
hyperframes
Turn HTML, CSS and animations into repeatable MP4 videos, built for coding agents
voicebox
Local voice studio for cloning voices, generating speech and dictating into any app
upscayl
Desktop app that enlarges low-resolution images using AI, on Linux, macOS and Windows
VoxCPM
Open text-to-speech model that makes, designs and clones voices in 30 languages
awesome-gpt-image-2
A library of 530+ GPT Image 2 prompts with templates and an agent skill