Keeping JEPA World Models Plannable When Little of the Frame Moves
Specifying a goal in language rather than as a goal frame is a natural interface for planning with a latent world model, but testing it needs scenes in which language must discriminate between several objects. We build SLIM, a pushing benchmark with several small objects and paired visual and language goals on identical scenes. On SLIM a LeWM world model that solves PushT succeeds on under 1% of…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.