A recent study by AI company Anthropic found that even participants with experience in causal inference may have their decisions significantly influenced by AI agents, with most failing to recognize this influence.
The study recruited 50 testers who were asked to use Claude Code to complete statistical analyses. Participants acted as board members of a fictional agricultural cooperative, needing to decide whether to expand or terminate a paid consulting program. The final conclusion was based on four smaller decisions, including whether to assign equal weight to all members' farms or to consider larger farms more heavily. All participants had experience in causal inference and were able to identify correlations from raw data.
Significant Impact of AI on Human Decision-Making
During the test, researchers preemptively instructed Claude to favor a particular option in five decisions, with each inclination determined randomly by a coin toss. If AI suggestions had no effect, the expected proportion of participants choosing in line with the suggestions would be about 50%. The actual results showed that participants adopted Claude's recommendations 86% of the time. When participants had an initial judgment and Claude offered a contrary suggestion, they changed their decision 60% of the time.
Researchers also established a statistical model based on the randomly generated AI suggestions, which accurately predicted 82% of the final outcomes. Anthropic stated that this means researchers could roughly predict what recommendations analysts would make based solely on the coin toss results before the task began.
Follow-up surveys revealed that 77% of participants reiterated the logic provided by Claude when explaining their decision-making basis, with some statements nearly identical, yet only 15% admitted that AI influenced their judgment. A total of 6 individuals recognized that the experiment aimed to measure AI's impact on human decision-making, but no one questioned that the AI outputs were determined by the coin toss results.
Anthropic warned that when AI agents are applied in healthcare, finance, defense, and AI safety research, any arbitrary or misaligned default biases within the system could enter expert judgment and influence scientific and policy decisions. Researchers also noted that the experiment was conducted in a highly controlled environment, and while participants were familiar with causal inference, they lacked agricultural experience, meaning the applicability of the findings in real-world scenarios requires further validation.