StartLuxLabs /
StartLux-Decision
Open models that give a probability for every answer option, and think only when unsure
- 1.4k stars
- Python
- Updated today
- Some experience
- Models & local AI
Most LLMs answer by writing text, so you have to parse it and guess how sure the model was. StartLux-Decision takes a state (text, JSON or images) and a set of typed questions. It returns a probability for every option of every question.
There are two variants. Instant models answer in a single forward pass. AutoThink models also start with one pass, but if the top probability is below a gate (0.6 by default) they write a short reasoning trace and read the answer again. So they spend tokens only on hard cases.
This repository holds the inference engine and an HTTP server with a System One compatible API at /v1/systemone. It runs on vLLM, and the models also ship as GGUF and MLX files for llama.cpp and Apple hardware. The authors report the 27B Instant model ranks first on the Decision Index 0.3.2 leaderboard; that is their own account, and the technical report has the full evaluation.
What it does
Typed questions with probabilities
Questions can be a choice among listed options, a yes/no judgement, or a level on a scale. Each answer comes with probabilities, not just a label.
Thinks only when unsure
AutoThink reasons only when its top probability is under the gate. Requests can set think_mode (auto, never, always), gate and think_budget.
Many sizes and formats
Sizes run from 0.8B to 27B, each with GGUF and MLX 8-bit and 4-bit versions on Hugging Face. AutoThink is offered at 4B, 9B and 27B.
Fast single-pass answers
The README reports Instant models answer in 25 to 52 ms at the median on one H200 with vLLM. One serve command can launch vLLM across several GPUs.
Hosted option and demos
bamboonode.dev hosts the 27B AutoThink model with the same request format, so you can try it without a GPU. Demos include chess, Super Mario Bros. and StarCraft II.
What you could build
- Route support tickets to a team and rate their urgency and severity in one request.
- Add a calibrated decision step to an agent that picks its next action from a list of options.
- Classify images or video frames, one frame a second, with typed questions.
- Run a small 0.8B or 2B model locally on a Mac with MLX for fast classification.
- Set a confidence gate so only uncertain cases trigger slower reasoning.
Get started
Easiest is the hosted 27B AutoThink model at bamboonode.dev, which needs no GPU. To self-host, install the package next to vLLM, download a model from Hugging Face, start the server and POST a state with typed questions to /v1/systemone.
pip install -e '.[vllm]'hf download startlux-models/StartLux-Decision-v1.1-4B-AutoThink --local-dir models/StartLux-Decision-v1.1-4B-AutoThinkstartlux-decide serve --model models/StartLux-Decision-v1.1-4B-AutoThink --launch-vllm --gpus 0,1,2,3 --port 8090Commands are copied from the project's README. Read it on GitHub for the full setup.
More in Models & local AI
transformers
Shared model definitions for text, vision, audio and multimodal AI models
open-webui
Self-hosted chat interface for Ollama and OpenAI-compatible models, with RAG and plugins
odysseus
Self-hosted workspace for AI chat, agents, research, documents, email and calendar
minimind
Train a tiny 64M-parameter language model from scratch on one consumer GPU
Kronos
Pre-trained Transformer models that forecast candlestick (K-line) price data
airllm
Run very large open language models on a small GPU by loading one layer at a time
Vane
Self-hosted AI answer engine that searches the web and cites its sources
timesfm
Pretrained Google model that forecasts time series without training on your data
Unlimited-OCR
Open OCR model that turns images and multi-page PDFs into structured text in one pass
gallery
Android and iOS app for running open LLMs offline on your phone