SPIN Unprocessed October 28, 2026 ai_technology research
Beyond IceBERT: Open-Data ModernBERT Encoders for Icelandic from 12M to 1B Parameters
View original on arxiv.orgOverview
arXiv:2610.36230v1 Announce Type: new Abstract: IceBERT, the reference encoder for Icelandic, found that a cleaned web crawl is enough to train a strong model and that starting from a multilingual model can help on some tasks. We revisit both findings with the ModernBERT architecture. We train Icelandic encoders from 12M to 1B parameters on 9B tokens of open data, and rerun IceBERT's benchmarks and five newer tasks for 30 encoders under one protocol with confidence intervals. Our 1B model traine
SpinGraph analysis pending — check back after processing.
Ask AI about this story
Opens with the SpinGraph .md URL and structured context — one click, prompt included.
More from arXiv Computation and Language
View all →- Japanese Discourse Structure Analysis Based on Segmented Discourse Representation Theory Using LLMs
- Structured Maps for Landmark-Based Route Instructions with LLMs
- Agentic Data Sampling Framework for Multilingual ASR via Linguistic-Aware Error-Driven Optimization
- ETC-Bench: A Multilingual Benchmark for Evaluation-Side Transcript Canonicalization
- Causal Localization of Evidence Effects in Transformers
- Benchmarking Task Takeover Across Agent Handoffs
Markdown (.md) · JSON-LD schema (.json) · Machine-readable for AI & GEO