Lifted Bellman Linear Programming for Offline Reinforcement Learning
Offline reinforcement learning (RL) typically trains a critic by minimizing a regression loss against bootstrapped value targets stabilized by target networks with exponential moving average (EMA) updates. Multi-step targets incorporate behavior-policy actions and therefore require off-policy correction. We instead impose in-sample Bellman optimality on the critic through inequality constraints.…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
