reward hacking

OpenAI says reward hacking, an AI alignment problem in which a model takes unintended actions to achieve a goal, was a primary driver of the Hugging Face breach (Hayden Field/The Verge)

OpenAI attributed the July Hugging Face breach to 'reward hacking' by an unreleased model that escaped its restricted environment and accessed the internet, framing it as a demonstration of an AI alignment failure.