# ODS

> Installer that sets up a private local AI server with chat, voice, RAG and workflows

- Repository: https://github.com/Osmantic/ODS
- Category: Models & local AI
- Stars: 7,658 · Forks: 1,082 · Language: Python · License: Apache-2.0
- Experience: Some experience
- Page: https://ir.adyog.com/repos/osmantic/ods/

Running AI locally usually means stitching together separate projects such as Ollama or llama.cpp, Open WebUI, n8n and ComfyUI, then writing Docker configs so they talk to each other. ODS (Osmantic Deployment System) does that setup for you. One installer detects your hardware, picks a model, generates credentials and starts the services.

It is a Docker-based stack. A llama-server handles inference, Open WebUI gives you a chat page, and a dashboard lets you manage models, services and GPU status. Voice, agents, workflows, document search and image generation run alongside. Cloud API modes (OpenAI, Anthropic, Together) are optional if you have no GPU.

The README compares it to Ollama and llama.cpp, Open WebUI, AnythingLLM and n8n's self-hosted AI starter kit. Each of those covers one slice. ODS aims to be the whole server around them. It is labelled a V3 pre-release that installs the current main branch, which is not a signed release, so expect rough edges.

## What it does

- **One-command install**: A single curl command on Linux and macOS, or a guided PowerShell script on Windows that sets up WSL2, Docker Desktop and Ubuntu, asking before each change.
- **Hardware-aware model choice**: The installer detects your GPU and picks a model to suit it. It supports NVIDIA and AMD Strix Halo on Linux and Windows, and Apple Silicon with Metal on macOS. Intel Arc is experimental.
- **Control dashboard**: One place to manage models, services, setup, GPU status and extensions. On AMD under Windows it can download compatible GGUF models, including from Hugging Face.
- **Voice, agents, workflows and RAG**: Bundled services let you build automations that listen, speak and call tools, and connect local documents and private search to your models.
- **Local-first privacy**: Inference, chat history and files stay on your machine, and the README says there is no telemetry. By default it goes online only for downloads, release checks and agent web searches.

## What you could build

- Run a private ChatGPT-style chat page on a home PC, Mac or Linux box.
- Build voice-enabled agents and automations that stay on your own hardware.
- Search your own documents with a local retrieval setup.
- Generate images locally with ComfyUI instead of a hosted API.
- Set up a homelab AI server that other devices on your network can use.

## Who it's for

Suits homelab tinkerers and developers who want a local AI server without assembling it, and who are comfortable with Docker and with running a pre-release installer.

## Get started

On Linux or macOS, make sure Docker is installed and running, then run the command in a normal terminal. When it finishes, open Portal at http://localhost:3001/pixel and send a message to check it works. Windows uses a separate PowerShell installer described in the README.

```
curl -fsSL https://install.osmantic.com/ods.sh | bash
```

Tags: local-ai, self-hosted, docker, llama-cpp, rag