I got Qwen3.8-27B running on dual RTX 3090s (no NVLink) under WSL2 — every pitfall I hit

I successfully installed Qwen3.8-27B on a dual RTX 3090 system running Windows 10 and WSL2. Initially, I attempted to use vLLM, but it failed to provide any speedup or benefit from MTP speculation without NVLink support, resulting in slower token generation rates. Next, I tried SGLang, which led to the model freezing during weight loading due to differences in weight packing between Qwen3.6 and Qwen3.8 checkpoints.

After discovering that the model checkpoint is half-quantized, I changed the CUDA toolchain to version 13.0, which resolved the load issue. Additionally, I had to disable custom all-reduce communication and the SymmMem feature to avoid crashes under WSL2. Once these configuration hurdles were overcome, I found that using the DSpark speculative decoding model achieved the best performance of 170-210 tokens per second for code and JSON inputs.

However, several important lessons were learned during this process, including the critical importance of matching CUDA toolchain versions, avoiding SM86 limitations, and being cautious with memory allocation settings, as these issues can significantly impact performance.