SPIN Unprocessed July 8, 2026 ai_technology research
Where to cut, how deep: BPE and Unigram-LM on chemistry SMILES
View original on arxiv.orgOverview
arXiv:2607.05691v1 Announce Type: new Abstract: Every chemical language model reading SMILES begins with a tokenizer, yet the field has inherited byte-pair encoding (BPE) from natural language with little scrutiny. In natural language, BPE's principal alternative, Unigram-LM, is known to build structurally different vocabularies. Whether that contrast survives in chemistry was open. We report a controlled comparison of BPE and Unigram-LM over a fixed 165-token chemistry base, at the small vocabu
SpinGraph analysis pending — check back after processing.
Ask AI about this story
Opens with the SpinGraph .md URL and structured context — one click, prompt included.
More from arXiv Computation and Language
View all →- CoPiT: Cognitive Pivot Translation for Digraphic Low-Resource Mongolian in the Traditional Script
- Inject or Navigate? Token-Efficient Retrieval for LLM Analysis of Transactional Legal Documents
- When Should LLMs Search? Counterfactual Supervision for Search Routing
- Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decoding
- SpanUQ: Span-Level Uncertainty Quantification for Large Language Model Generation
- UCSC NLP at SemEval-2026 Task 10: Boundary-Aware Span Extraction and RoBERTa Classification for Conspiracy Detection
Markdown (.md) · JSON-LD schema (.json) · Machine-readable for AI & GEO