[2604.03476] Fine-tuning DeepSeek-OCR-2 for Molecular Structure Recognition

Computer Science > Computer Vision and Pattern Recognition arXiv:2604.03476 (cs) [Submitted on 3 Apr 2026] Title:Fine-tuning DeepSeek-OCR-2 for Molecular Structure Recognition Authors:Haocheng Tang, Xingyu Dang, Junmei Wang View a PDF of the paper titled Fine-tuning DeepSeek-OCR-2 for Molecular Structure Recognition, by Haocheng Tang and Xingyu Dang and Junmei Wang View PDF HTML (experimental) Abstract:Optical Chemical Structure Recognition (OCSR) is critical for converting 2D molecular diagrams from printed literature into machine-readable formats. While Vision-Language Models have shown promise in end-to-end OCR tasks, their direct application to OCSR remains challenging, and direct full-parameter supervised fine-tuning often fails. In this work, we adapt DeepSeek-OCR-2 for molecular optical recognition by formulating the task as image-conditioned SMILES generation. To overcome training instabilities, we propose a two-stage progressive supervised fine-tuning strategy: starting with parameter-efficient LoRA and transitioning to selective full-parameter fine-tuning with split learning rates. We train our model on a large-scale corpus combining synthetic renderings from PubChem and realistic patent images from USPTO-MOL to improve coverage and robustness. Our fine-tuned model, MolSeek-OCR, demonstrates competitive capabilities, achieving exact matching accuracies comparable to the best-performing image-to-sequence model. However, it remains inferior to state-of-the-art imag...