Reading up on Victoria Krakovna

Optimization of measurable proxies diverges from true goals, causing harms from Soviet whaling to Facebook’s engagement-driven decline and AI reward hacking, revealing a universal failure mode.