How “reverse game theory” could keep AI agents in check
A new approach known as “reverse game theory” aims to control powerful AI models by designing systems that incentivize them to stick to the rules.
A new approach known as “reverse game theory” aims to control powerful AI models by designing systems that incentivize them to stick to the rules.
The short version
- As AI models become more capable, safety researchers are scrambling to ensure they are “aligned” with human desires—to make sure, in other words, that rogue agents don’t break the law,…
- Borrowing from disciplines like economics could help AI safety researchers control AI models, even if they don’t understand them.
- Until now, this work has mostly focused on trying to peer into an AI’s “brain” to understand what it is “thinking,” or training models to hew to values that we hold dear.
What happened
Claude, for instance, has its constitution —a sort of moral code. The trouble is, these techniques don’t seem to work particularly well: Models have been shown to deliberately underperform on tests, hide their intentions , and mislead their evaluators .
Why it matters
Now some alignment researchers believe that reverse game theory, also known as mechanism design, might get better results.
Summary by Nerd News Network. Read the full article at MIT Technology Review — AI via the links above and below.