Accelerating vision-language models with LFM2.5-VL-DSpark

Liquid AI has unveiled an experimental DSpark draft model for their vision-language model (VLM) LFM2.5-VL-3B. This new model incorporates a speculative decoding path that enhances speed without altering the output quality. The vision drafter architecture mirrors that of their text LFM2.5-DSpark drafters, capturing hidden states at predetermined layers and conditioning them to generate candidate tokens.

Image patches and text tokens are projected into a shared representation before reaching these layers, ensuring identical dimensional vectors for both modalities.

The inference algorithm remains consistent with text models. To optimize the model, a mixture of vision-language SFT data is employed, emphasizing workloads anticipated for the model. Based on different layer variants (3, 4, and 5), the draft model was simplified with 4 layers and a block size of 9. Following 10 epochs of training, the acceptance rate improved, eventually plateauing. The recommended block size at inference time is 8 or 9, contingent on the hardware used.

The DSpark draft model, featuring around 280 million parameters, increases the deployed model's parameter count by merely 8.9%. It is compatible with llama.cpp, MLX-VLM, and SGLang, with day-one support. The DSpark model ships with pre-installed support for these frameworks. Both on-device and GPU inference were tested, with the models evaluated on six diverse vision-based tasks such as general VQA, text VQA, image captioning, chart VQA, complex reasoning, and multi-turn conversations, following the MMSpec benchmark.

Performance-wise, on-device inference with MLX on an M5 Max showcased a decoding speedup ranging from 2.30x to 3.13x, while end-to-end latency improved by 1.56x to 2.62x. With llama.cpp on an M3 Ultra, decoding speedup and end-to-end latency improvements were recorded at 1.57x to 2.14x and 1.30x to 1.77x, respectively. In GPU inference, the same draft model delivered 20.4x to 2.66x faster decoding, with end-to-end latency improvements of 1.64x to 2.27x.

Speculative decoding only accelerates the decoding process, not the vision encoding or prefill stages. Consequently, the overall speedup is limited by the portion of the workload not accelerated. This aligns with Amdahl's law, where the overall speedup is capped by the non-accelerated part of the workload. To utilize the DSpark draft models with SGLang, a build with DSpark support for LFM2 targets is required (PR #40651).

The target should be launched with the draft attached, and queries should be made at http://localhost:30000/v1. The block size is obtained from the draft's config.json, while the baseline is the same command without the --speculative-* flags. Running the models with llama.cpp necessitates a respective llama.cpp build (PR#29339), and with MLX-VLM, a build from PR#2280 is required. The block size is read from the sidecar metadata, with n-max clamped to it.

The vision DSpark draft model is accessible on Hugging Face in Safetensors and GGUF formats. Liquid AI's vision of AI that runs anywhere continues with the release of these models. For citations, refer to the provided reference or BibTeX: Liquid AI, "LFM2.5-VL-DSpark: Accelerating vision-language models on edge and beyond," Liquid AI Blog, September 2026.