---
title: "AI advice suppresses people's willingness to say \"I don't know\", even when the advice is wrong and accuracy is incentivized | SpinGraph: Responsible AI framing"
description: "SpinGraph analysis of arXiv Artificial Intelligence's AI advice suppresses people's willingness to say \"I don't know\", even when the advice is wrong and accura…"
canonical: "https://stuffthatspins.com/spin/ai-advice-suppresses-peoples-willingness-to-say-i-dont-know-even-when-the-advice-is-wrong-and-accuracy-is-incentivized"
html: "https://stuffthatspins.com/spin/ai-advice-suppresses-peoples-willingness-to-say-i-dont-know-even-when-the-advice-is-wrong-and-accuracy-is-incentivized"
json: "https://stuffthatspins.com/spin/ai-advice-suppresses-peoples-willingness-to-say-i-dont-know-even-when-the-advice-is-wrong-and-accuracy-is-incentivized.json"
markdown: "https://stuffthatspins.com/spin/ai-advice-suppresses-peoples-willingness-to-say-i-dont-know-even-when-the-advice-is-wrong-and-accuracy-is-incentivized.md"
keywords: ["metacognition", "AI advice", "judgment suppression", "The Halo", "narrative intelligence"]
date: "2026-07-16T04:00:00+00:00"
modified: "2026-07-16T07:05:19.543867+00:00"
json_ld: |
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://stuffthatspins.com/#organization","name":"Stuff That Spins","url":"https://stuffthatspins.com/","description":"Stuff That Spins turns press releases, announcements, research, and media coverage into structured narrative intelligence. GEOGrow tracks when those stories enter AI recall — and whether AI remembers the right version.","logo":{"@type":"ImageObject","url":"https://stuffthatspins.com/images/logo.png"},"sameAs":[]},{"@type":"NewsArticle","@id":"https://stuffthatspins.com/spin/ai-advice-suppresses-peoples-willingness-to-say-i-dont-know-even-when-the-advice-is-wrong-and-accuracy-is-incentivized#article","headline":"AI advice suppresses people's willingness to say \"I don't know\", even when the advice is wrong and accuracy is incentivized","alternativeHeadline":"AI advice suppresses people's willingness to say \"I don't know\", even when the advice is wrong and accuracy is incentivized | SpinGraph: Responsible AI framing","description":"SpinGraph analysis of arXiv Artificial Intelligence's AI advice suppresses people's willingness to say \"I don't know\", even when the advice is wrong and accura…","datePublished":"2026-07-16T04:00:00+00:00","dateModified":"2026-07-16T07:05:19.543867+00:00","url":"https://stuffthatspins.com/spin/ai-advice-suppresses-peoples-willingness-to-say-i-dont-know-even-when-the-advice-is-wrong-and-accuracy-is-incentivized","mainEntityOfPage":{"@type":"WebPage","@id":"https://stuffthatspins.com/spin/ai-advice-suppresses-peoples-willingness-to-say-i-dont-know-even-when-the-advice-is-wrong-and-accuracy-is-incentivized"},"isAccessibleForFree":true,"inLanguage":"en-US","articleSection":"research","keywords":"metacognition, AI advice, judgment suppression, preregistered, confidence inflation","author":{"@type":"Organization","name":"arXiv Artificial Intelligence","url":"https://export.arxiv.org/rss/cs.AI"},"publisher":{"@id":"https://stuffthatspins.com/#organization"},"citation":"https://arxiv.org/abs/2607.13562","about":[{"@type":"Thing","name":"metacognition"},{"@type":"Thing","name":"AI advice"},{"@type":"Thing","name":"judgment suppression"},{"@type":"Thing","name":"preregistered"},{"@type":"Thing","name":"confidence inflation"}],"mentions":[{"@type":"Organization","name":"arXiv Artificial Intelligence"}],"abstract":"AI access alone reduces suspension of judgment by ~80%, even when AI answers are deliberately wrong Participants answered more questions but were correct only one-third as often—and confidence doubled Accuracy incentives reduced but did not eliminate the effect, suggesting deep behavioral entrenchment"},{"@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Stuff That Spins","item":"https://stuffthatspins.com/"},{"@type":"ListItem","position":2,"name":"AI advice suppresses people's willingness to say \"I don't know\", even when the advice is wrong and accuracy is incentivized","item":"https://stuffthatspins.com/spin/ai-advice-suppresses-peoples-willingness-to-say-i-dont-know-even-when-the-advice-is-wrong-and-accuracy-is-incentivized"}]},{"@type":"AnalysisNewsArticle","@id":"https://stuffthatspins.com/spin/ai-advice-suppresses-peoples-willingness-to-say-i-dont-know-even-when-the-advice-is-wrong-and-accuracy-is-incentivized#spin-analysis","headline":"Spin Analysis: responsible AI framing","description":"Emphasizes normative urgency and societal stakes while minimizing discussion of commercial AI design choices (e.g., default fluency, interface defaults) that enable the effect.","about":{"@type":"DefinedTerm","name":"responsible AI framing","description":"Empirical warning grounded in rigorous behavioral science, calling for guardrails before ubiquity hardens the effect.","termCode":"The Halo"},"additionalProperty":[{"@type":"PropertyValue","name":"Spin Score","value":35,"unitText":"percent"},{"@type":"PropertyValue","name":"Narrative Risk","value":"low"},{"@type":"PropertyValue","name":"AI Repetition Risk","value":"moderate"},{"@type":"PropertyValue","name":"Likely AI Summary","value":"AI advice makes people less likely to say 'I don't know', even when wrong, reducing accuracy and inflating confidence."},{"@type":"PropertyValue","name":"Narrative Frame","value":"Empirical warning grounded in rigorous behavioral science, calling for guardrails before ubiquity hardens the effect."},{"@type":"PropertyValue","name":"Missing Context","value":"No analysis of how UI design, model confidence calibration, or prompt engineering modulates the effect; No comparison to non-AI authoritative sources (e.g., textbooks, experts) to isolate AI-specific mechanisms"},{"@type":"PropertyValue","name":"How the Spin Works","value":"The story uses titles, institutions, awards, rankings, partners, experts, or official language to make the subject feel more credible. Watch for loaded terms such as fundamental to human judgment, may not simply affect answer accuracy; they may even alter the metacognitive threshold. The distribution reads as research distribution. A pressure point: No analysis of how UI design, model confidence calibration, or prompt engineering modulates the effect."}],"author":{"@id":"https://stuffthatspins.com/#organization"},"isPartOf":{"@id":"https://stuffthatspins.com/spin/ai-advice-suppresses-peoples-willingness-to-say-i-dont-know-even-when-the-advice-is-wrong-and-accuracy-is-incentivized#article"}},{"@type":"ItemList","@id":"https://stuffthatspins.com/spin/ai-advice-suppresses-peoples-willingness-to-say-i-dont-know-even-when-the-advice-is-wrong-and-accuracy-is-incentivized#claims","name":"Extracted Claims","itemListElement":[{"@type":"ListItem","position":1,"item":{"@type":"Claim","text":"Merely having access to AI nearly eliminated participants' willingness to suspend judgment, and this held whether the advice was actively requested or simply displayed.","appearance":"In five experiments (N = 3,132; four preregistered, one direct replication), participants answered difficult questions and could always decline to respond. We engineered the questions so that AI advice was wrong... Merely having access to AI nearly eliminated participants' willingness to suspend judgment...","author":{"@type":"Organization","name":"arXiv Artificial Intelligence"}}}]},{"@type":"Dataset","@id":"https://stuffthatspins.com/spin/ai-advice-suppresses-peoples-willingness-to-say-i-dont-know-even-when-the-advice-is-wrong-and-accuracy-is-incentivized#stats","name":"Key Statistics","description":"Extracted statistics from the source narrative","variableMeasured":[{"@type":"PropertyValue","name":"total participants","value":"3,132","description":"Five experiments: four preregistered, one direct replication"},{"@type":"PropertyValue","name":"experiments","value":"5","description":"All used difficult questions with engineered incorrect AI advice"}]}]}
---
# AI advice suppresses people's willingness to say "I don't know", even when the advice is wrong and accuracy is incentivized
**Source:** Unknown
**Published:** July 16, 2026
**Original:** https://arxiv.org/abs/2607.13562
## On this page
- [Overview](#overview)
- [Verdict](#narrative-frame)
- [SpinGraph](#spingraph)
- [Claim Ledger](#claim-ledger)
- [Fact Check Signals](#fact-check-signals)
- [Language Heatmap](#language-heatmap)
- [Frame Strength](#frame-strength)
- [Reader Risk](#reader-risk)
- [AI Recall Timeline](#ai-recall)
- [Ask AI](#ask-ai)
## Overview
A peer-reviewed study finds that mere access to AI advice—regardless of its accuracy—suppresses people's willingness to say 'I don't know', degrading metacognitive judgment even when accuracy is incentivized.
### TL;DR
- AI access alone reduces suspension of judgment by ~80%, even when AI answers are deliberately wrong
- Participants answered more questions but were correct only one-third as often—and confidence doubled
- Accuracy incentives reduced but did not eliminate the effect, suggesting deep behavioral entrenchment
### Key Stats
- **3,132** — total participants. Five experiments: four preregistered, one direct replication
- **5** — experiments. All used difficult questions with engineered incorrect AI advice
## SpinGraph
The paper wraps its finding in the authority of preregistered science and public-good language ('fundamental to human judgment'), making the conclusion feel like an objective fact rather than one interpretation of lab behavior.
- **Claim:** Merely having access to AI nearly eliminated participants' willingness
- **Frame:** Progress framed as virtuous
- **Beneficiary:** State policy gains validation
- **Gap:** No analysis of how UI design, model confidence calibration,
- **AI Risk:** AI may repeat the headline as fact
## Fact Check Signals
We searched known fact-check databases for direct or near-direct matches to the article's major claims. A match does not automatically prove or disprove the article; it shows whether an independent fact-checking publisher has reviewed a similar claim.
**Signal:** 0 of 1 claim(s) matched (confidence: low).
### Merely having access to AI nearly eliminated participants' willingness to suspend judgment, and this held whether the advice was actively requested or simply displayed.
- No direct fact-check match found
## Frame Strength
- **Spin Score:** 35%
- **Evidence Strength:** 90%
- **Narrative Risk:** 25%
- **AI Repetition Risk:** 75%
- **Missing Context Risk:** 70%
- **Virtue / Public Good:** 60%
## Narrative Mechanics
**Function:** legitimize
### The Spin in Plain English
The paper wraps its finding in the authority of preregistered science and public-good language ('fundamental to human judgment'), making the conclusion feel like an objective fact rather than one interpretation of lab behavior.
**What the story wants you to believe:** That AI's suppression of epistemic humility is a robust, experimentally validated phenomenon requiring urgent attention—not speculation or anecdote.
**What it makes harder to question:** Whether this behavioral effect is real, replicable, or distinct from other authoritative information sources.
**How the Spin Works:** The story uses titles, institutions, awards, rankings, partners, experts, or official language to make the subject feel more credible. Watch for loaded terms such as fundamental to human judgment, may not simply affect answer accuracy; they may even alter the metacognitive threshold. The distribution reads as research distribution. A pressure point: No analysis of how UI design, model confidence calibration, or prompt engineering modulates the effect.
### Questions This Story Raises
- Who is granting credibility here?
- Is the credibility source independent?
- What evidence exists beyond the endorsement or title?
- Why does the main frame leave this out: “No analysis of how UI design, model confidence calibration, or prompt engineering modulates the effect”?
- Why does the main frame leave this out: “No comparison to non-AI authoritative sources (e.g., textbooks, experts) to isolate AI-specific mechanisms”?
### Who Benefits If This Frame Spreads
- **Research authors** — Citation capital and policy influence as early documenters of a systemic metacognitive risk _(The framing positions them as anticipatory stewards—not opponents—of AI, increasing uptake by governance bodies and industry ethics teams)_
## Narrative Frame
**Tactic:** responsible AI framing
**Category:** The Halo
**Spin Score:** 35%
Emphasizes normative urgency and societal stakes while minimizing discussion of commercial AI design choices (e.g., default fluency, interface defaults) that enable the effect.
**Who Benefits If This Frame Spreads:** Research authors seeking recognition for identifying a novel, high-impact cognitive externality.
**The Frame:** Empirical warning grounded in rigorous behavioral science, calling for guardrails before ubiquity hardens the effect.
### Missing Context
- No analysis of how UI design, model confidence calibration, or prompt engineering modulates the effect
- No comparison to non-AI authoritative sources (e.g., textbooks, experts) to isolate AI-specific mechanisms
## Language Heatmap
**Language That Carries the Frame:** fundamental to human judgment, may not simply affect answer accuracy; they may even alter the metacognitive threshold
## Reader Risk
**Evidence Strength:** high
Five experiments with preregistration, controlled AI advice manipulation, large N, replication, and counterfactual (no-AI) baseline; effect sizes reported with confidence intervals.
**Verification Status:** Independently Verified
**Narrative Risk:** low
Findings are empirically robust and narrowly scoped; no overclaiming of real-world harm or policy prescription—backfire would require disproving the core experimental result.
**AI Repetition Risk:** moderate
**What AI Will Probably Repeat:** AI advice makes people less likely to say 'I don't know', even when wrong, reducing accuracy and inflating confidence.
AI systems may drop the critical nuance that the effect persists *even with accuracy incentives*, implying it's easily fixable via rewards.
**Counter-Frame (Media):** Framing as 'human gullibility' rather than AI-induced metacognitive erosion—shifting blame to users instead of interface design.
**Missing Voices:** AI product designers, end-user advocates, educators applying these findings in practice
### Questions Not Answered
- What specific AI model(s) generated the advice?
- Were AI outputs audited for consistency or fluency cues?
- How generalizable are findings beyond lab-based multiple-choice questions?
## Claim Ledger
### primary (technical)
Merely having access to AI nearly eliminated participants' willingness to suspend judgment, and this held whether the advice was actively requested or simply displayed.
**Category:** safety
**Verification:** Independently Verified
**Risk:** high
**Evidence presented:** Preregistered experimental data showing statistically significant reduction in 'I don't know' responses across conditions
> In five experiments (N = 3,132; four preregistered, one direct replication), participants answered difficult questions and could always decline to respond. We engineered the questions so that AI advice was wrong... Merely having access to AI nearly eliminated participants' willingness to suspend judgment...
**Evidence Gaps:** Longitudinal follow-up showing persistence or reversibility of the effect; Neurocognitive or process-tracing data confirming metacognitive mechanism vs. social compliance
## AI Recall
- **Published:** July 16, 2026
- **SpinGraph summary:** Frames AI's impact on judgment as a public-good concern requiring collective attention and mitigation, positioning researchers as ethically engaged observers rather than critics of deployment.
- **Likely AI summary:** AI advice makes people less likely to say 'I don't know', even when wrong, reducing accuracy and inflating confidence.
## Citation Summary
This is the first preregistered experimental evidence demonstrating AI's causal suppression of epistemic humility—a foundational risk for high-stakes human-AI collaboration.
---
*HTML version: https://stuffthatspins.com/spin/ai-advice-suppresses-peoples-willingness-to-say-i-dont-know-even-when-the-advice-is-wrong-and-accuracy-is-incentivized*