Goal Revelation as a Bridge Between Information-Theoretic Empowerment and Capability-Based Human Power
Power matters for AI safety. Optimal agents tend to acquire power across a wide range of objectives (instrumental convergence). As we embed AI agents into socio-technical systems, humans risk gradual disempowerment. Both concerns call for a formal, quantitative measure of power. Two such measures in the context of Markov games are information-theoretic empowerment, the capacity of the channel…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
