IsoSci: A Benchmark of Isomorphic Cross-Domain Science Problems for Evaluating Reasoning versus Knowledge Retrieval in LLMs

Summary

arXiv:2607.01431v1 Announce Type: new Abstract: We introduce ISOSCI, a benchmark of isomorphic cross-domain science problem pairs that separates reasoning ability from domain knowledge retrieval in LLM evaluation. Each pair shares identical logical structure but requires different domain-specific knowledge, enabling controlled attribution of reasoning-mode gains. Across five model pairs spanning four model families, we find that 91.3% of reasoning-mode gains are knowledge-dependent rather than s

SpinGraph analysis pending — check back after processing.

• Can Language Models Actually Retrieve In-Context? Drowning in Documents at Million Token Scale
• Parameter Golf: What Really Works?
• From Monolingual to Multilingual: Evaluating Mamba for ASR in South African Languages
• Comparing Architectures for Supervised Political Scaling
• Grounded Optimization: A Layered Engineering Framework for Reducing LLM Hallucination in Automated Personal Document Rewriting
• FaithMed: Training LLMs For Faithful Evidence-Based Medical Reasoning