# needle

> A tiny 8-29 MB model that calls tools and extracts fields on phones and microcontrollers

- Repository: https://github.com/cactus-compute/needle
- Category: Models & local AI
- Stars: 14,209 · Forks: 982 · Language: Python · License: Apache-2.0
- Experience: Some experience
- Page: https://ir.adyog.com/repos/cactus-compute/needle/

Needle is a foundation model for small devices. The whole model is one 8-29 MB file. It gives up general chat ability so it can do a narrow set of jobs well: choose which of your app's functions to call, fill in the arguments, and return typed data from unstructured text.

You declare your tools as Python functions or as a JSON file. The model reads the user's request and returns one JSON object with function calls, its reasoning and a confidence score. A grammar built from your schemas constrains each token, so the output always parses. If no tool fits the request, you get an empty list instead of a guess.

The README says it beats models 10x its size on mobile tool calls and matches models 2-3x larger on extraction. Those figures are the authors' own benchmarks. The package also includes Whistle, a small speech-to-text model that shares the same runtime, so audio can go in and tool calls come out. You can fine-tune Needle on your own tools, either locally or on the authors' hosted platform.

## What it does

- **Tool calling**: Given your functions, it selects the right ones, fills every argument and returns several calls in order. If no tool applies, it returns an empty list.
- **Grammar-constrained extraction**: You declare a shape and pass in messy text. A grammar compiled from your schema guarantees the output parses. It can also be used for classification.
- **Confidence and embeddings**: Every response carries a calibrated confidence score you can use to act, confirm or refuse. The same model also returns text embeddings for local search and routing.
- **Whistle speech-to-text**: A 16.9 MB model on the same engine that transcribes up to 30 seconds of audio in seven languages. Audio in, tool calls out, in one call.
- **Fine-tuning and deployment**: Fine-tune locally with LoRA, or on the hosted platform for every model depth. Prebuilt engines for each platform are fetched with needle build.

## What you could build

- Build a voice assistant that turns a spoken request into calls to your app's functions, fully on the device.
- Extract invoice, booking or notification fields from messy text on a phone without a network call.
- Control smart-home devices or robots by mapping short commands to a fixed set of actions.
- Search and route user queries locally using the model's text embeddings.
- Fine-tune a small 2-to-8-layer model on one product's tools to fit a very constrained device.

## Who it's for

Developers building apps for phones, wearables, robots or embedded hardware who want local tool calling; basic Python and the idea of function calling are enough to start.

## Get started

Install the package, then decorate a Python function with @needle.tool and create a needle.Needle(tools=[...]) agent. Call agent.run with a request and it picks the function, runs it and returns the results. Note that telemetry is on by default; set NEEDLE_TELEMETRY=0 and DO_NOT_TRACK=1 to turn it off.

```
pip install cactus-needle
```

Tags: on-device-ai, tool-calling, edge-ai, function-calling, tinyml