AI repos for vLLM
Open-source AI projects built around vLLM or genuinely useful if you work with it, each with what it does for you. 5 repos, most relevant first.
-
HyperQwen
Patch series for a pinned vLLM that serves Qwen on one 24 GB GPU with speculative decoding.
-
Unlimited-OCR
Includes a vLLM recipe to serve the OCR model behind an API.
-
GLM-5
Includes a vLLM cookbook for serving GLM-5 models locally.
-
modular
Serve models through an OpenAI-compatible endpoint with MAX, an alternative inference framework.
-
StartLux-Decision
Self-host the models next to vLLM and POST typed questions to an HTTP endpoint.