[2603.26830] A Regression Framework for Understanding Prompt Component Impact on LLM Performance

Computer Science > Machine Learning arXiv:2603.26830 (cs) [Submitted on 27 Mar 2026] Title:A Regression Framework for Understanding Prompt Component Impact on LLM Performance Authors:Andrew Lauziere, Jonathan Daugherty, Taisa Kushner View a PDF of the paper titled A Regression Framework for Understanding Prompt Component Impact on LLM Performance, by Andrew Lauziere and 2 other authors View PDF HTML (experimental) Abstract:As large language models (LLMs) continue to improve and see further integration into software systems, so does the need to understand the conditions in which they will perform. We contribute a statistical framework for understanding the impact of specific prompt features on LLM performance. The approach extends previous explainable artificial intelligence (XAI) methods specifically to inspect LLMs by fitting regression models relating portions of the prompt to LLM evaluation. We apply our method to compare how two open-source models, Mistral-7B and GPT-OSS-20B, leverage the prompt to perform a simple arithmetic problem. Regression models of individual prompt portions explain 72% and 77% of variation in model performances, respectively. We find misinformation in the form of incorrect example query-answer pairs impedes both models from solving the arithmetic query, though positive examples do not find significant variability in the impact of positive and negative instructions - these prompts have contradictory effects on model performance. The framework se...