I Benchmarked Two Local LLMs on Real Dev Work — Qwopus 27B vs Muse Glimmer 30B

Two open-weight language models, Qwopus 27B and Meta's Muse Glimmer 30B, were benchmarked on real development tasks by a reporter. The models were run locally on an AMD Radeon RX 7900 XT GPU using llama.cpp multi-model router. A third model, Codex, reviewed both outputs and provided a verdict.

In the first task, both models fixed a broken Progressive Web App (PWA) with a single regression. Qwopus completed the task in 8.5 minutes and Muse took 21 minutes. Both produced a byte-identical diff and fixed the manifest issue. However, Codex rejected both models' solutions due to a latent conflict that neither model noticed. The contradiction was that the tests hard-coded the root path, while the deployment lived under a subpath.

In the second task, both models added a single-player mode against an AI opponent to the game. Qwopus completed the task in 32 minutes and Muse in 26 minutes. Qwopus wrote 18 tests, while Muse wrote 7 tests, one of which had a contradiction in its comment. Both models had their strengths and weaknesses, with Qwopus demonstrating adaptive thresholding and better test coverage, while Muse had a more straightforward approach with fixed thresholds.

The reviewer model, Codex, caught the latent manifest/base-path conflict missed by both models and identified real bugs in the second task.