ER-JEPA: Experience Replay Improves Joint-Embedding Predictive Learning in Language Models
Large language models (LLMs) excel at token-level generation but may learn undesirable abstract semantics and lack comprehensive perception. LLM-JEPA mitigates this by aligning different views of the same underlying knowledge via a joint-embedding predictive architecture (JEPA). However, strong alignment does not necessarily lead to accurate, stable predictions. To address this, we propose…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
