Rare Event Estimation via Iterative Unalignment
As agents are deployed with increased autonomy, even extremely rare events along their stochastic output trajectories can occur and prove catastrophic. Safe deployment therefore does not depend on whether these events can occur, but on how often they might. We study the problem of estimating the probability of rare events that arise from stochastic variation in the agent's own actions. Estimating…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
