needle-2-esp32 overview
A 45M-parameter LLM doing tool calls fully offline on ESP32-S3. Repo ↗
What it is
An independent C99 implementation of the .cact inference format, about 3,700 lines with no dependencies. Weights memory-map from flash, requests stream in over a terminal, and schema-valid tool calls drive the onboard RGB LED with no network involved.
Each response takes around forty seconds on the board, reasoning included.
Hardware
One specific configuration is supported: ESP32-S3 with 16 MB flash and 8 MB octal PSRAM, WS2812 LED on GPIO48, console over the UART USB port.
Smaller flash or quad PSRAM cannot fit the model and KV cache, so match the board before assuming broken code.