LLMs Know the Constraint But Do Not Use It: Activation Bottlenecks in Pragmatic Constraint Reasoning | SpinGraph: Technical precision framing
SpinGraph analysis of arXiv Computation and Language's LLMs Know the Constraint But Do Not Use It: Activation Bottlenecks in Pragmatic Constraint Reasoning sto…
Know the moment AI knows your story. Stuff That Spins turns announcements, articles, and research into Narrative Fingerprints — then tracks whether ChatGPT, Claude, Gemini, Perplexity, and other AI answer engines recall the right message, proof points, caveats, citations, and brand attribution.
LLMs Know the Constraint But Do Not Use It: Activation Bottlenecks in Pragmatic Constraint Reasoning
pragmatic reasoning, activation patching, constraint routing, LLM failure modes
LLMs encode feasibility constraints but fail to consistently activate them during reasoning The paper introduces a 'quartet diagnostic' and activation patching to isolate routing failures from knowledge gaps Two distinct failure modes are identified; one is repairable via donor activation, the other is not
Spin Analysis: technical precision framing
Emphasizes theoretical elegance and intervention potential while minimizing limitations: no evaluation on real-world applications, no user-facing impact assessment, and no discussion of scalability or generalizability beyond the diagnostic setup.
Foundational cognitive architecture insight — positioning the work as revealing a core mechanistic bottleneck rather than a domain-specific artifact.
LLMs know constraints but don’t always use them — it’s a routing problem, not a knowledge problem.
No discussion of training data origins or model provenance affecting constraint encoding; No comparison to human pragmatic reasoning baselines or cognitive plausibility
The story uses titles, institutions, awards, rankings, partners, experts, or official language to make the subject feel more credible. Watch for loaded terms such as repairable, mitigation frontier, diagnostic, routing problem. The distribution reads as academic distribution. A pressure point: No discussion of training data origins or model provenance affecting constraint encoding.
Hidden-constraint failure is a routing problem, not a knowledge problem.
We formalize the distinction as conditional constraint activation: the constraint is internally encoded (Knowledge) symmetrically across constraint-present and -absent prompts (Symmetry), yet only sometimes routed into the decision (Routing) and repairable by a donor activation (Repair).
Extracted statistics from the source narrative
Across open-weight and proprietary-architecture LLMs
Probes on two open-weight models successfully decode hidden constraints