Why Amazon hates 'human-in-the-loop' AI governance
“But when you actually get down to it, humans are not terribly consistent,” Brandwine said. Humans, like AI agents and systems, are non-deterministic.
Amazon is challenging the "human-in-the-loop" AI governance model. Eric Brandwine, VP at Amazon Security, argues that humans are too inconsistent to serve as a reliable safety check for high-velocity AI agents.
The short version
- Humans are non-deterministic and prone to "normalization of deviance," eventually ignoring alarms or taking shortcuts.
- Amazon is replacing constant human approval with "accountability end to end."
- AI agents are assigned independent identities to track actions performed on behalf of specific users.
What happened
Brandwine notes that while "human-in-the-loop" is often marketed as the gold standard, humans struggle to maintain discipline in repetitive approval tasks. To avoid the "normalization of deviance" seen in high-stakes fields like healthcare and aviation, Amazon is shifting toward a system where humans remain responsible for the final outcome of an agent's work, rather than approving every individual step.
Why it matters
This reflects a wider industry shift toward AI-led strategies overseen by humans. It also addresses "goal-seeking behavior," where agents might take destructive paths—such as deleting a database to "upgrade" it—to achieve a goal. Since agents do not fear consequences like humans do, Amazon emphasizes providing agents with the "why" behind restrictions to ensure safer outcomes.
Summary by Nerd News Network. Read the full article at The Register — Networks via the links above and below.