Osmantic /
ODS
Installer that sets up a private local AI server with chat, voice, RAG and workflows
- 7.7k stars
- 699 stars since 3 Oct
- Python
- Updated today
- Some experience
- Models & local AI
Running AI locally usually means stitching together separate projects such as Ollama or llama.cpp, Open WebUI, n8n and ComfyUI, then writing Docker configs so they talk to each other. ODS (Osmantic Deployment System) does that setup for you. One installer detects your hardware, picks a model, generates credentials and starts the services.
It is a Docker-based stack. A llama-server handles inference, Open WebUI gives you a chat page, and a dashboard lets you manage models, services and GPU status. Voice, agents, workflows, document search and image generation run alongside. Cloud API modes (OpenAI, Anthropic, Together) are optional if you have no GPU.
The README compares it to Ollama and llama.cpp, Open WebUI, AnythingLLM and n8n's self-hosted AI starter kit. Each of those covers one slice. ODS aims to be the whole server around them. It is labelled a V3 pre-release that installs the current main branch, which is not a signed release, so expect rough edges.
What it does
One-command install
A single curl command on Linux and macOS, or a guided PowerShell script on Windows that sets up WSL2, Docker Desktop and Ubuntu, asking before each change.
Hardware-aware model choice
The installer detects your GPU and picks a model to suit it. It supports NVIDIA and AMD Strix Halo on Linux and Windows, and Apple Silicon with Metal on macOS. Intel Arc is experimental.
Control dashboard
One place to manage models, services, setup, GPU status and extensions. On AMD under Windows it can download compatible GGUF models, including from Hugging Face.
Voice, agents, workflows and RAG
Bundled services let you build automations that listen, speak and call tools, and connect local documents and private search to your models.
Local-first privacy
Inference, chat history and files stay on your machine, and the README says there is no telemetry. By default it goes online only for downloads, release checks and agent web searches.
What you could build
- Run a private ChatGPT-style chat page on a home PC, Mac or Linux box.
- Build voice-enabled agents and automations that stay on your own hardware.
- Search your own documents with a local retrieval setup.
- Generate images locally with ComfyUI instead of a hosted API.
- Set up a homelab AI server that other devices on your network can use.
Get started
On Linux or macOS, make sure Docker is installed and running, then run the command in a normal terminal. When it finishes, open Portal at http://localhost:3001/pixel and send a message to check it works. Windows uses a separate PowerShell installer described in the README.
curl -fsSL https://install.osmantic.com/ods.sh | bashCommands are copied from the project's README. Read it on GitHub for the full setup.
More in Models & local AI
transformers
Shared model definitions for text, vision, audio and multimodal AI models
open-webui
Self-hosted chat interface for Ollama and OpenAI-compatible models, with RAG and plugins
odysseus
Self-hosted workspace for AI chat, agents, research, documents, email and calendar
minimind
Train a tiny 64M-parameter language model from scratch on one consumer GPU
Kronos
Pre-trained Transformer models that forecast candlestick (K-line) price data
airllm
Run very large open language models on a small GPU by loading one layer at a time
Vane
Self-hosted AI answer engine that searches the web and cites its sources
timesfm
Pretrained Google model that forecasts time series without training on your data
Unlimited-OCR
Open OCR model that turns images and multi-page PDFs into structured text in one pass
gallery
Android and iOS app for running open LLMs offline on your phone