Is your RKNN model secretly running on the CPU? I built a tool that tells you
Converting a model to RKNN and watching it "run" on a Rockchip board proves exactly one thing: it runs. It says nothing about where it runs. RKNN will happily execute unsupported or badly-quantized layers on the CPU, silently, while your benchmarks just look "a bit slower than expected." In my own testing I have seen a SAM-variant conversion whose decoder carried a dozen CPU ops — the model…
RKNN models don't always run on the NPU by default, and a new tool called RK Edge Doctor reveals this hidden behavior. RKNN will execute unsupported or poorly quantized layers on the CPU without any warning. This issue was discovered during testing with a Sam-variant conversion that had a dozen CPU operations. To check for these hidden CPU operations, the tool adds a model command to RK Edge Doctor, which provides a one-run solution to find out if a model is NPU_ONLY or has fallback operations on the CPU.
The tool uses the runtime library that comes with every RKNN board and measures the runtime's own accounting to determine the fallback share of time. It's important to run the tool on custom architectures or models with exotic operations, as they are likely to have CPU fallback operations. The tool records the model file name and hash only, ensuring no path information is exposed.
The repo for RK Edge Doctor is available on GitHub, with support for multiple RKNN boards and future plans for RTSP camera testing.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
