Have you heard of Goodhart’s Law? If not, I have come to believe it is a crucial piece in understanding why complex systems in which intelligent agents adapt their behavior can easily deteriorate over time: If agents are rewarded for optimizing a proxy, and they have limited resources to spend on improving their condition across… Continue reading The Goodhart-Sinclair Trap
Exploring how the meaning of things that matter to me comes about