![[2502.11684] MathFimer: Enhancing Mathematical Reasoning by Expanding Reasoning Steps through Fill-in-the-Middle Task](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2502.11684] MathFimer: Enhancing Mathematical Reasoning by Expanding Reasoning Steps through Fill-in-the-Middle Task
Summary
The paper introduces MathFimer, a framework designed to enhance mathematical reasoning in large language models by expanding reasoning steps through a novel 'Fill-in-the-Middle' task.
Why It Matters
As mathematical reasoning is crucial for advancing AI capabilities, MathFimer addresses the limitations of existing models by providing a scalable solution that enhances the quality of reasoning steps without requiring expensive computational resources. This innovation could significantly improve the performance of AI in solving complex mathematical problems.
Key Takeaways
- MathFimer enhances mathematical reasoning by expanding reasoning steps.
- The framework utilizes a 'Fill-in-the-Middle' task to improve model training.
- Models trained on MathFimer-expanded datasets outperform those on original data.
- The approach is scalable and does not require powerful external models.
- Comprehensive experiments validate the effectiveness across multiple benchmarks.
Computer Science > Computation and Language arXiv:2502.11684 (cs) [Submitted on 17 Feb 2025 (v1), last revised 25 Feb 2026 (this version, v3)] Title:MathFimer: Enhancing Mathematical Reasoning by Expanding Reasoning Steps through Fill-in-the-Middle Task Authors:Yuchen Yan, Yongliang Shen, Yang Liu, Jin Jiang, Xin Xu, Mengdi Zhang, Jian Shao, Yueting Zhuang View a PDF of the paper titled MathFimer: Enhancing Mathematical Reasoning by Expanding Reasoning Steps through Fill-in-the-Middle Task, by Yuchen Yan and 7 other authors View PDF Abstract:Mathematical reasoning represents a critical frontier in advancing large language models (LLMs). While step-by-step approaches have emerged as the dominant paradigm for mathematical problem-solving in LLMs, the quality of reasoning steps in training data fundamentally constrains the performance of the models. Recent studies have demonstrated that more detailed intermediate steps can enhance model performance, yet existing methods for step expansion either require more powerful external models or incur substantial computational costs. In this paper, we introduce MathFimer, a novel framework for mathematical reasoning step expansion inspired by the ''Fill-in-the-middle'' task from code reasoning. By decomposing solution chains into prefix-suffix pairs and training models to reconstruct missing intermediate steps, we develop a specialized model, MathFimer-7B, on our carefully curated NuminaMath-FIM dataset. We then apply these models to e...
