CoTA++: Understanding the Repeat Curse in dMLLMs from an Information Flow Perspective
Diffusion-based multimodal large language models (dMLLMs) decode by iterative denoising with bidirectional attention, which precludes the standard autoregressive key--value cache. Practical deployments therefore rely on approximate caching that reuses intermediate states across denoising steps. Such caching, however, often makes a response repeat one token at adjacent positions in a run that…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
