Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m…
Henry from Cactus introduced Needle 2, a 14MB agentic LLM designed for phones, wearables, smart homes, and robots. This compact model runs a full session in just 28MB of RAM and boasts a decode speed of 500 tokens per second on a Raspberry Pi 5. The model's performance drops to 300-700 tokens per second on VR devices like Meta Quest 3S and Apple Vision Pro, and 300-700 tokens per second on sub-$200 phones such as the Samsung A-Series.
In comparison to other small models, Needle 2 offers performance close to LFM2.5 230M and Apple Foundation Model at significantly smaller sizes—five times to 70 times smaller, both in f16 format as opposed to Needle 2's 2-bit compression.
Rooted in Simple Attention Networks, Needle 2 is engineered for edge AI, making it accessible to the vast majority of connected IoT devices worldwide. In fact, conventional transformers expend 164 MFLOPs per token, while Needle itself consumes 70 MFLOPs. On a high-end phone, this edge AI model consumes 7x to 85x fewer power per token compared to the smallest performant LLMs.
Needle's architecture is designed around structured intelligence functions with typed parameters, eliminating the need for world knowledge and open-ended prose. This allows the model to achieve frontier-level performance in just 45 million parameters. Needle 2 can also be used for structured extraction, where a schema can be passed in place of tools, resulting in structured output.
As a text-classification model or summarization model, Needle 2 can be fine-tuned with a Python package provided by Cactus Compute, enabling users to fine-tune the model on a Mac/PC in just minutes to a few hours using an automated data-generation pipeline. The Python package allows users to pass a few samples to fine-tune Needle.
Additionally, each response from Needle 2 comes with a learned confidence score based on Cactus Hybrid technique. If the confidence score exceeds a set threshold, the model will act; otherwise, it will escalate to the cloud or a larger model. Combining Needle 2 with a private DeepSeek-v4-Flash deployment is particularly effective for enterprise-level tasks at minimal cost.
Cactus Compute is available to help users set up this combination. The Cactus Compute team encourages users to test Needle 2 via a playground provided in the link and share feedback.
Written by urgent.news from Hacker News Best's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
