The official RKLLM vision demo doesn't run on the transformers it targets — a five-version bisection
I'm converting Qwen2-VL-2B-Instruct to run on a Rockchip board with Rockchip's official rknn-llm toolkit. Step one is exporting the vision tower to ONNX with the repo's own multimodal demo script. It did not run. Not on my code — on the repo's own script, with the transformers generation the repo's newer scripts are written for. Here is the failure trail, version by version: transformers result…
A reporter is detailing the process of converting the Qwen2-VL-2B-Instruct model to run on a Rockchip board using the Rockchip's official rknn-llm toolkit. The first step is exporting the vision tower to ONNX using the repo's own multimodal demo script, but this script is incompatible with the newer transformers generation. The reporter identifies version-specific issues with the transformers library, specifically AttributeError and TypeError, which occur when trying to access certain attributes and unpack position_embeddings.
The vision export script is written against a pre-4.54 contract, while the quantization script already uses the 5.x module layout. The reporter decides to run two environments side by side, one with transformers 4.51.3 and another with transformers 5.8.0, to avoid altering the original script. By verifying the conditions against the transformers source code and ensuring compatibility, the vision tower exports cleanly as a 53 MB ONNX graph with 2,661,085,184 bytes of weights.
Next steps involve quantization, RKLLM export, and ultimately, performance evaluation on the device. The full process and results can be found in the GitHub repository linked in the story.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
