[2407.16893] The Price of Prompting: Profiling Energy Use in Large Language Models Inference

Computer Science > Computers and Society arXiv:2407.16893 (cs) [Submitted on 4 Jul 2024 (v1), last revised 3 Mar 2026 (this version, v2)] Title:The Price of Prompting: Profiling Energy Use in Large Language Models Inference Authors:Erik Johannes Husom, Arda Goknil, Lwin Khin Shar, Sagar Sen View a PDF of the paper titled The Price of Prompting: Profiling Energy Use in Large Language Models Inference, by Erik Johannes Husom and 3 other authors View PDF HTML (experimental) Abstract:In the rapidly evolving realm of artificial intelligence, deploying large language models (LLMs) poses increasingly pressing computational and environmental challenges. This paper introduces MELODI - Monitoring Energy Levels and Optimization for Data-driven Inference - a multifaceted framework crafted to monitor and analyze the energy consumed during LLM inference processes. MELODI enables detailed observations of power consumption dynamics and facilitates the creation of a comprehensive dataset reflective of energy efficiency across varied deployment scenarios. The dataset, generated using MELODI, encompasses a broad spectrum of LLM deployment frameworks, multiple language models, and extensive prompt datasets, enabling a comparative analysis of energy use. Using the dataset, we investigate how prompt attributes, including length and complexity, correlate with energy expenditure. Our findings indicate substantial disparities in energy efficiency, suggesting ample scope for optimization and adopti...