AI Can Quietly Influence Experts’ Decisions, Anthropic Study Finds

Professional analysts adopted their AI agent's hidden, randomly assigned opinion on 86% of key judgment calls, against about 50% expected by chance, according to an experiment published Tuesday by Anthropic's Alignment team. Only 15% credited the agent afterward, and none suspected its preferences had been planted.

AI agents---programs that carry out tasks on their own---are being handed expert work on what the study calls the usual assumption: that experts still make the decisions that matter. Anthropic launched 10 agent templates for financial firms and insurers in May, saying users have the final word.

That arrangement, called "human in the loop," assumes the person who signs off on an AI's work is the one making the important calls. Anthropic tested it on 50 analysts experienced in causal inference, meaning working out from data whether one thing causes another.

Each analyst used Claude Code, Anthropic's coding agent, to advise a fictional farming cooperative on expanding an advisory program. Unknown to them, a coin flip set which option each agent favored, hidden in a system prompt, the instructions that shape how an AI behaves.

"Across the four analytical choices, analysts went with their agent's lean 86% of the time. If the agent's input made no difference, we'd expect about 50%," Anthropic said. Those who had already chosen before the agent weighed in abandoned it 60% of the time when the agent disagreed, against 6% when it agreed, though that rests on only 38 cases.

Asked to explain their choices, 77% repeated the agent's argument in at least one answer, sometimes word for word. Only 15% credited the agent, and 52% called the final recommendation more their own than the agent's.

"For individual experts, this could mean losing agency over their own work, a form of what researchers call disempowerment," Anthropic said. This phenomenon basically means the human voluntarily loses control over their own decisions. "The work still goes out under the expert's name," the authors wrote, "but the decisions that most depend on their judgment are in reality settled by the agent's prior beliefs."

In April 2025, OpenAI rolled back an update to ChatGPT that it called overly flattering or agreeable. The agents here were told never to mislead, yet they rarely argued for the alternative.

"With only one side argued well," the authors wrote, "the analysts could adopt the agent's assumptions without working out their own." Error checks struggle to catch this because judgment calls have no single correct answer, so the work looks sound whichever option is picked.
OpenAI Rolls Back ChatGPT Update After User Pushback Over 'Sycophantic' Behavior

Anthropic says this is no argument for using AI less, since analysts with the agent finished faster and made far fewer serious errors, though the no-agent group was small and less experienced. It suggests agents that argue the best case for each option or ask the expert first, though that could overwhelm users.

The researchers deliberately strengthened the agents' preferences and tested one task, field, and model, so real-world effects may be smaller. Measuring it outside the lab is the hard part: the team could detect deference only because it randomized each agent's lean, which the authors say is rarely possible in real work.