![[2510.26510] LLMs as In-Context Meta-Learners for Model and Hyperparameter Selection](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2510.26510] LLMs as In-Context Meta-Learners for Model and Hyperparameter Selection
Summary
This article explores the use of large language models (LLMs) as in-context meta-learners for model and hyperparameter selection in machine learning, demonstrating their potential to recommend effective models without extensive search.
Why It Matters
The ability to efficiently select models and hyperparameters is crucial in machine learning, often requiring expert knowledge. This study highlights how LLMs can simplify this process, making advanced machine learning techniques more accessible and efficient for practitioners.
Key Takeaways
- LLMs can act as effective meta-learners for model selection.
- Two prompting strategies were tested: zero-shot and meta-informed.
- Meta-informed prompting significantly improves model recommendations.
- LLMs can leverage dataset metadata for better hyperparameter tuning.
- This approach reduces the need for expert intuition and costly searches.
Computer Science > Machine Learning arXiv:2510.26510 (cs) [Submitted on 30 Oct 2025 (v1), last revised 13 Feb 2026 (this version, v3)] Title:LLMs as In-Context Meta-Learners for Model and Hyperparameter Selection Authors:Youssef Attia El Hili, Albert Thomas, Malik Tiomoko, Abdelhakim Benechehab, Corentin Léger, Corinne Ancourt, Balázs Kégl View a PDF of the paper titled LLMs as In-Context Meta-Learners for Model and Hyperparameter Selection, by Youssef Attia El Hili and 6 other authors View PDF HTML (experimental) Abstract:Model and hyperparameter selection are critical but challenging in machine learning, typically requiring expert intuition or expensive automated search. We investigate whether large language models (LLMs) can act as in-context meta-learners for this task. By converting each dataset into interpretable metadata, we prompt an LLM to recommend both model families and hyperparameters. We study two prompting strategies: (1) a zero-shot mode relying solely on pretrained knowledge, and (2) a meta-informed mode augmented with examples of models and their performance on past tasks. Across synthetic and real-world benchmarks, we show that LLMs can exploit dataset metadata to recommend competitive models and hyperparameters without search, and that improvements from meta-informed prompting demonstrate their capacity for in-context meta-learning. These results highlight a promising new role for LLMs as lightweight, general-purpose assistants for model selection and h...
