[2601.09566] Hot-Start from Pixels: Low-Resolution Visual Tokens for Chinese Language Modeling

Computer Science > Computer Vision and Pattern Recognition arXiv:2601.09566 (cs) [Submitted on 14 Jan 2026 (v1), last revised 3 Mar 2026 (this version, v3)] Title:Hot-Start from Pixels: Low-Resolution Visual Tokens for Chinese Language Modeling Authors:Shuyang Xiang, Hao Guan View a PDF of the paper titled Hot-Start from Pixels: Low-Resolution Visual Tokens for Chinese Language Modeling, by Shuyang Xiang and Hao Guan View PDF HTML (experimental) Abstract:Large language models typically represent Chinese characters as discrete index-based tokens, largely ignoring their visual form. For logographic scripts, visual structure carries semantic and phonetic information, which may aid prediction. We investigate whether low-resolution visual inputs can serve as an alternative for character-level modeling. Instead of token IDs, our decoder receives grayscale images of individual characters, with resolutions as low as 8 x 8 pixels. Remarkably, these inputs achieve 39.2% accuracy, comparable to the index-based baseline of 39.1%. Such low-resource settings also exhibit a pronounced hot-start effect: by 0.4% of total training, accuracy reaches above 12%, while index-based models lag at below 6%. Overall, our results demonstrate that minimal visual structure can provide a robust and efficient signal for Chinese language modeling, offering an alternative perspective on character representation that complements traditional index-based approaches. Comments: Subjects: Computer Vision and Pa...