---
title: "Where to cut, how deep: BPE and Unigram-LM on chemistry SMILES — Stuff That Spins"
description: "arXiv:2607.05691v1 Announce Type: new Abstract: Every chemical language model reading SMILES begins with a tokenizer, yet the field has inherited byte-pair enc…"
	canonical: "https://stuffthatspins.com/spin/where-to-cut-how-deep-bpe-and-unigram-lm-on-chemistry-smiles"
html: "https://stuffthatspins.com/spin/where-to-cut-how-deep-bpe-and-unigram-lm-on-chemistry-smiles"
json: "https://stuffthatspins.com/spin/where-to-cut-how-deep-bpe-and-unigram-lm-on-chemistry-smiles.json"
markdown: "https://stuffthatspins.com/spin/where-to-cut-how-deep-bpe-and-unigram-lm-on-chemistry-smiles.md"
keywords: ["narrative intelligence", "SpinGraph", "AI recall"]
date: "2026-07-08T04:00:00+00:00"
modified: "2026-07-08T06:03:35.802858+00:00"
json_ld: |
  {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://stuffthatspins.com/#organization","name":"Stuff That Spins","url":"https://stuffthatspins.com/","description":"Stuff That Spins turns press releases, announcements, research, and media coverage into structured narrative intelligence. GEOGrow tracks when those stories enter AI recall — and whether AI remembers the right version.","logo":{"@type":"ImageObject","url":"https://stuffthatspins.com/images/logo.png"},"sameAs":[]},{"@type":"NewsArticle","@id":"https://stuffthatspins.com/spin/where-to-cut-how-deep-bpe-and-unigram-lm-on-chemistry-smiles#article","headline":"Where to cut, how deep: BPE and Unigram-LM on chemistry SMILES","description":"arXiv:2607.05691v1 Announce Type: new Abstract: Every chemical language model reading SMILES begins with a tokenizer, yet the field has inherited byte-pair enc…","datePublished":"2026-07-08T04:00:00+00:00","dateModified":"2026-07-08T06:03:35.802858+00:00","url":"https://stuffthatspins.com/spin/where-to-cut-how-deep-bpe-and-unigram-lm-on-chemistry-smiles","mainEntityOfPage":{"@type":"WebPage","@id":"https://stuffthatspins.com/spin/where-to-cut-how-deep-bpe-and-unigram-lm-on-chemistry-smiles"},"isAccessibleForFree":true,"inLanguage":"en-US","articleSection":"research","author":{"@type":"Organization","name":"arXiv Computation and Language","url":"https://export.arxiv.org/rss/cs.CL"},"publisher":{"@id":"https://stuffthatspins.com/#organization"},"citation":"https://arxiv.org/abs/2607.05691","about":[],"mentions":[{"@type":"Organization","name":"arXiv Computation and Language"}]},{"@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Stuff That Spins","item":"https://stuffthatspins.com/"},{"@type":"ListItem","position":2,"name":"Where to cut, how deep: BPE and Unigram-LM on chemistry SMILES","item":"https://stuffthatspins.com/spin/where-to-cut-how-deep-bpe-and-unigram-lm-on-chemistry-smiles"}]}]}
---

# Where to cut, how deep: BPE and Unigram-LM on chemistry SMILES

**Source:** Unknown  
**Published:** July 8, 2026  
**Original:** https://arxiv.org/abs/2607.05691  

## On this page

- [Overview](#overview)

<a id="overview"></a>

## Overview

arXiv:2607.05691v1 Announce Type: new Abstract: Every chemical language model reading SMILES begins with a tokenizer, yet the field has inherited byte-pair encoding (BPE) from natural language with little scrutiny. In natural language, BPE's principal alternative, Unigram-LM, is known to build structurally different vocabularies. Whether that contrast survives in chemistry was open. We report a controlled comparison of BPE and Unigram-LM over a fixed 165-token chemistry base, at the small vocabu

---
*HTML version: https://stuffthatspins.com/spin/where-to-cut-how-deep-bpe-and-unigram-lm-on-chemistry-smiles*