Designing Transformers for Continuous Diffusion Language Models
Continuous diffusion language models (CDLMs) offer an alternative approach to parallel language generation; however, most existing studies directly use transformer architectures from either causal large language models (LLMs) or Diffusion Transformers (DiTs) for image generation, which were not optimized for CDLMs. In this paper, we conduct a systematic study on Transformer architectures for…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
