Show HN: Cactus Needle 3: 8-29MB automation models can match DeepSeek V4 Flash

The Cactus Needle 3 model, capable of producing automation models ranging from 2 to 20 layers, has achieved performance comparable to DeepSeek V4 Flash. Each layer functions as a sub-network with increasing capacity, allowing developers to select the appropriate size from 2-layer to 20-layer models. Finetuning is possible for each sub-network, with a 4-layer subnetwork able to match DeepSeek V4 Flash after one epoch of fine-tuning on downstream tasks.

The resulting models are compact, ranging from 9 to 29 MB in size and are suitable for deployment on a variety of devices.

The Needle 3 model utilizes a Python package for installation and inference is fetched from Hugging Face and cached, eliminating the need for additional building. Developers can define tools by decorating functions, providing argument types in the signature and the tool description in the docstring. The model then executes the function, feeds the result back into the model, and returns the final response with the executed tool results attached.

Pattern matching can be utilized for routing requests to specific tools, with a catch-all excluding unrelated terms. Extraction of structured data from text can be performed by declaring the desired shape and calling extract(), returning a typed object. The model's confidence score, which already has a floor of 0.1, can be used to gate responses and route them appropriately. Tools with triggers always produce a call for a matching request, informing the system whether to run the tool or confirm otherwise.

When writing tools, it's recommended to have one tool per action, with a narrow tool having a plain description being more effective than a broad one. Each action should be clearly defined and named after the user's possible inputs, with synonyms included in the description. Required arguments should have defaults to accommodate optional inputs.

Fine-tuning of the model can be achieved through the Python package, with LoRA on the frozen base at full 20 layers followed by a 4-bit .cact file for any subnetwork. Guides are available on designing tools, confidence gating, extraction, fine-tuning, and more on the GitHub repository. The Cactus Platform provides additional tools for dataset management, full-depth fine-tuning, and deployment on various platforms, all under 1 MB for devices.