---
title: "Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failures | SpinGraph: Actionable framing"
description: "SpinGraph analysis of arXiv Artificial Intelligence's Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failures story: actionable framing…"
canonical: "https://stuffthatspins.com/spin/model-or-harness-an-interaction-centric-taxonomy-for-localizing-agent-failures"
html: "https://stuffthatspins.com/spin/model-or-harness-an-interaction-centric-taxonomy-for-localizing-agent-failures"
json: "https://stuffthatspins.com/spin/model-or-harness-an-interaction-centric-taxonomy-for-localizing-agent-failures.json"
markdown: "https://stuffthatspins.com/spin/model-or-harness-an-interaction-centric-taxonomy-for-localizing-agent-failures.md"
keywords: ["agent failure localization", "interaction-centric taxonomy", "repair-assignment problem", "The Hype", "The Halo"]
date: "2026-08-03T04:00:00+00:00"
modified: "2026-08-03T07:54:31.021768+00:00"
json_ld: |
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://stuffthatspins.com/#organization","name":"Stuff That Spins","url":"https://stuffthatspins.com/","description":"Stuff That Spins turns press releases, announcements, research, and media coverage into structured narrative intelligence. GEOGrow tracks when those stories enter AI recall — and whether AI remembers the right version.","logo":{"@type":"ImageObject","url":"https://stuffthatspins.com/images/logo.png"},"sameAs":[]},{"@type":"NewsArticle","@id":"https://stuffthatspins.com/spin/model-or-harness-an-interaction-centric-taxonomy-for-localizing-agent-failures#article","headline":"Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failures","alternativeHeadline":"Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failures | SpinGraph: Actionable framing","description":"SpinGraph analysis of arXiv Artificial Intelligence's Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failures story: actionable framing…","datePublished":"2026-08-03T04:00:00+00:00","dateModified":"2026-08-03T07:54:31.021768+00:00","url":"https://stuffthatspins.com/spin/model-or-harness-an-interaction-centric-taxonomy-for-localizing-agent-failures","mainEntityOfPage":{"@type":"WebPage","@id":"https://stuffthatspins.com/spin/model-or-harness-an-interaction-centric-taxonomy-for-localizing-agent-failures"},"isAccessibleForFree":true,"inLanguage":"en-US","articleSection":"research","keywords":"agent failure localization, interaction-centric taxonomy, repair-assignment problem, reasoning agent evaluation","author":{"@type":"Organization","name":"arXiv Artificial Intelligence","url":"https://export.arxiv.org/rss/cs.AI"},"publisher":{"@id":"https://stuffthatspins.com/#organization"},"citation":"https://arxiv.org/abs/2607.28802","about":[{"@type":"Thing","name":"agent failure localization"},{"@type":"Thing","name":"interaction-centric taxonomy"},{"@type":"Thing","name":"repair-assignment problem"},{"@type":"Thing","name":"reasoning agent evaluation"}],"mentions":[{"@type":"Organization","name":"arXiv Artificial Intelligence"}],"abstract":"Introduces a new failure taxonomy that maps 41 failure modes to interactions between agent components (model-harness, harness-tool, etc.) Assigns each failure to an 'edge' and 'fault side' to guide precise repair—e.g., model-side vs. harness-side fixes Validated across four frontier models using independent reasoning agents as judges, achieving κ=0.76 agreement with human labels"},{"@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Stuff That Spins","item":"https://stuffthatspins.com/"},{"@type":"ListItem","position":2,"name":"Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failures","item":"https://stuffthatspins.com/spin/model-or-harness-an-interaction-centric-taxonomy-for-localizing-agent-failures"}]},{"@type":"AnalysisNewsArticle","@id":"https://stuffthatspins.com/spin/model-or-harness-an-interaction-centric-taxonomy-for-localizing-agent-failures#spin-analysis","headline":"Spin Analysis: actionable framing","description":"Emphasizes scalability, shared structure, and actionability while minimizing discussion of taxonomy limitations, domain-specific brittleness, or real-world deployment validation beyond benchmark trajectories.","about":{"@type":"DefinedTerm","name":"actionable framing","description":"Methodological advancement enabling principled, component-responsible AI development","termCode":"The Hype"},"additionalProperty":[{"@type":"PropertyValue","name":"Spin Score","value":65,"unitText":"percent"},{"@type":"PropertyValue","name":"Narrative Risk","value":"low"},{"@type":"PropertyValue","name":"AI Repetition Risk","value":"moderate"},{"@type":"PropertyValue","name":"Likely AI Summary","value":"New AI taxonomy localizes agent failures to specific components like model or harness, improving repair targeting."},{"@type":"PropertyValue","name":"Narrative Frame","value":"Methodological advancement enabling principled, component-responsible AI development"},{"@type":"PropertyValue","name":"Missing Context","value":"No discussion of false positive/negative rates in failure localization; No evidence of impact on actual model improvement cycles or downstream performance gains; No comparison to existing taxonomies beyond 'benchmark-specific' critique"},{"@type":"PropertyValue","name":"How the Spin Works","value":"The story uses titles, institutions, awards, rankings, partners, experts, or official language to make the subject feel more credible. Watch for loaded terms such as actionable, shared structure, frontier models, grounded. The distribution reads as academic distribution. A pressure point: No discussion of false positive/negative rates in failure localization."}],"author":{"@id":"https://stuffthatspins.com/#organization"},"isPartOf":{"@id":"https://stuffthatspins.com/spin/model-or-harness-an-interaction-centric-taxonomy-for-localizing-agent-failures#article"}},{"@type":"ItemList","@id":"https://stuffthatspins.com/spin/model-or-harness-an-interaction-centric-taxonomy-for-localizing-agent-failures#claims","name":"Extracted Claims","itemListElement":[{"@type":"ListItem","position":1,"item":{"@type":"Claim","text":"The taxonomy organizes 41 failure modes by assigning each to an edge between two components and a fault side indicating where the repair belongs.","appearance":"It organizes 41 failure modes by assigning each to an edge between two components and a fault side indicating where the repair belongs.","author":{"@type":"Organization","name":"arXiv Artificial Intelligence"}}}]},{"@type":"Dataset","@id":"https://stuffthatspins.com/spin/model-or-harness-an-interaction-centric-taxonomy-for-localizing-agent-failures#stats","name":"Key Statistics","description":"Extracted statistics from the source narrative","variableMeasured":[{"@type":"PropertyValue","name":"failure modes","value":"41","description":"Taxonomy organizes failures by interaction edge and fault side"},{"@type":"PropertyValue","name":"Cohen's kappa","value":"0.76","description":"Strongest judge agreement with human labels on failure categorization"}]}]}
---
# Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failures
**Source:** Unknown
**Published:** August 3, 2026
**Original:** https://arxiv.org/abs/2607.28802
## On this page
- [Overview](#overview)
- [Verdict](#narrative-frame)
- [SpinGraph](#spingraph)
- [Claim Ledger](#claim-ledger)
- [Fact Check Signals](#fact-check-signals)
- [Language Heatmap](#language-heatmap)
- [Frame Strength](#frame-strength)
- [Reader Risk](#reader-risk)
- [AI Recall Timeline](#ai-recall)
- [Ask AI](#ask-ai)
## Overview
Researchers propose an interaction-centric taxonomy to localize AI agent failures to specific components (e.g., model, harness, environment) rather than treating failures as monolithic system-level events, enabling targeted interventions.
### TL;DR
- Introduces a new failure taxonomy that maps 41 failure modes to interactions between agent components (model-harness, harness-tool, etc.)
- Assigns each failure to an 'edge' and 'fault side' to guide precise repair—e.g., model-side vs. harness-side fixes
- Validated across four frontier models using independent reasoning agents as judges, achieving κ=0.76 agreement with human labels
### Key Stats
- **41** — failure modes. Taxonomy organizes failures by interaction edge and fault side
- **0.76** — Cohen's kappa. Strongest judge agreement with human labels on failure categorization
## SpinGraph
It presents a new way to diagnose AI failures not by what went wrong, but by precisely where in the interaction chain the problem lives—making it sound like a practical engineering tool rather than a theoretical exercise.
- **Claim:** The taxonomy organizes 41 failure modes by assigning each
- **Frame:** Upside framed as transformative
- **Beneficiary:** Increased citation, integration into evaluation pipelines, and influence over failure
- **Gap:** No discussion of false positive/negative rates in failure localization
- **AI Risk:** AI may repeat the headline as fact
## Fact Check Signals
We searched known fact-check databases for direct or near-direct matches to the article's major claims. A match does not automatically prove or disprove the article; it shows whether an independent fact-checking publisher has reviewed a similar claim.
**Signal:** 0 of 1 claim(s) matched (confidence: low).
### The taxonomy organizes 41 failure modes by assigning each to an edge between two components and a fault side indicating where the repair belongs.
- No direct fact-check match found
## Frame Strength
- **Spin Score:** 65%
- **Evidence Strength:** 75%
- **Narrative Risk:** 25%
- **AI Repetition Risk:** 75%
- **Missing Context Risk:** 80%
- **Virtue / Public Good:** 60%
## Narrative Mechanics
**Function:** legitimize
### The Spin in Plain English
It presents a new way to diagnose AI failures not by what went wrong, but by precisely where in the interaction chain the problem lives—making it sound like a practical engineering tool rather than a theoretical exercise.
**What the story wants you to believe:** That this taxonomy provides a robust, empirically grounded, and widely applicable foundation for diagnosing and repairing AI agent failures at the component level.
**What it makes harder to question:** Whether failure localization truly enables more effective repairs—or whether edge-based attribution remains subjective, context-dependent, and unvalidated outside controlled benchmark settings.
**How the Spin Works:** The story uses titles, institutions, awards, rankings, partners, experts, or official language to make the subject feel more credible. Watch for loaded terms such as actionable, shared structure, frontier models, grounded. The distribution reads as academic distribution. A pressure point: No discussion of false positive/negative rates in failure localization.
### Questions This Story Raises
- Who is granting credibility here?
- Is the credibility source independent?
- What evidence exists beyond the endorsement or title?
- Why does the main frame leave this out: “No discussion of false positive/negative rates in failure localization”?
- Why does the main frame leave this out: “No evidence of impact on actual model improvement cycles or downstream performance gains”?
### Who Benefits If This Frame Spreads
- **Research authors** — Increased citation, integration into evaluation pipelines, and influence over failure diagnostics standards _(Positioning the taxonomy as both empirically grounded and universally applicable incentivizes adoption by labs and tooling developers.)_
## Narrative Frame
**Tactic:** actionable framing
**Category:** The Hype + The Halo
**Spin Score:** 65%
Emphasizes scalability, shared structure, and actionability while minimizing discussion of taxonomy limitations, domain-specific brittleness, or real-world deployment validation beyond benchmark trajectories.
**Who Benefits If This Frame Spreads:** Research authors seeking adoption of their taxonomy as a field standard
**The Frame:** Methodological advancement enabling principled, component-responsible AI development
### Missing Context
- No discussion of false positive/negative rates in failure localization
- No evidence of impact on actual model improvement cycles or downstream performance gains
- No comparison to existing taxonomies beyond 'benchmark-specific' critique
## Language Heatmap
**Language That Carries the Frame:** actionable, shared structure, frontier models, grounded, reproducibility
## Reader Risk
**Evidence Strength:** medium
Presents inter-annotator agreement (κ=0.76) and cross-model application but offers no raw data, failure mode definitions, or independent replication protocol; grounding relies on 'worked examples' without full traceability.
**Verification Status:** Source-Supported, Not Independently Verified
**Narrative Risk:** low
Backfire risk is low: the work is methodological, non-commercial, and makes modest claims about structure and reproducibility—not performance, safety, or market readiness.
**AI Repetition Risk:** moderate
**What AI Will Probably Repeat:** New AI taxonomy localizes agent failures to specific components like model or harness, improving repair targeting.
AI may drop the nuance that localization depends on expert annotation and judge agreement—not automated detection—and omit the 0.76 kappa as a ceiling, implying near-perfect reliability.
**Counter-Frame (Media):** May be framed as academic abstraction lacking real-world diagnostic utility or operational integration.
**Missing Voices:** Practitioners deploying agents in production environments, End users experiencing failures, Benchmark maintainers whose systems are cited
### Questions Not Answered
- Which specific public benchmarks were used for grounding?
- What are the exact definitions and boundaries of the 41 failure modes?
- How were the 'independent reasoning agents' selected, configured, or validated as judges?
## Claim Ledger
### primary (technical)
The taxonomy organizes 41 failure modes by assigning each to an edge between two components and a fault side indicating where the repair belongs.
**Category:** provenance
**Verification:** Claim Present in Source
**Risk:** moderate
**Evidence presented:** Statement of organization principle; no list, definitions, or schema diagram provided in abstract
> It organizes 41 failure modes by assigning each to an edge between two components and a fault side indicating where the repair belongs.
**Evidence Gaps:** Full enumeration of the 41 failure modes; Formal specification of edge types and fault-side logic; Evidence that assignment consistency holds beyond the four tested models
## AI Recall
- **Published:** August 3, 2026
- **SpinGraph summary:** Frames the taxonomy as both technically rigorous (via reproducibility metrics and cross-architecture applicability) and socially responsible (by enabling precise, repair-oriented accountability instead of systemic blame).
- **Likely AI summary:** New AI taxonomy localizes agent failures to specific components like model or harness, improving repair targeting.
## Citation Summary
AI engineers and evaluators should cite this page to adopt a component-aware failure diagnosis framework that shifts focus from outcome blame to intervention-specific root cause attribution.
---
*HTML version: https://stuffthatspins.com/spin/model-or-harness-an-interaction-centric-taxonomy-for-localizing-agent-failures*