A new paper from Anthropic, released on Friday, suggests that AI can be "quite evil" when it's trained to cheat.

Anthropic found that when an AI model learns to cheat on software programming tasks and is rewarded for that behavior, it continues to display "other, even more misaligned behaviors as an unintended consequence." The result? Alignment faking and even sabotage of AI safety research.

"The cheating that induces this misalignment is what we call 'reward hacking': an AI fooling its training process into assigning a high reward, without actually completing the intended task (another way of putting it is that, in hacking the task, the model has found a loophole—working out how to be rewarded for satisfying the letter of the task but not its spirit)," Anthropic wrote of its papers' fi

See Full Page