[2603.24138] Efficient Controller Learning from Human Preferences and Numerical Data Via Multi-Modal Surrogate Models

Computer Science > Machine Learning arXiv:2603.24138 (cs) [Submitted on 25 Mar 2026] Title:Efficient Controller Learning from Human Preferences and Numerical Data Via Multi-Modal Surrogate Models Authors:Lukas Theiner, Maik Pfefferkorn, Yongpeng Zhao, Sebastian Hirt, Rolf Findeisen View a PDF of the paper titled Efficient Controller Learning from Human Preferences and Numerical Data Via Multi-Modal Surrogate Models, by Lukas Theiner and 4 other authors View PDF HTML (experimental) Abstract:Tuning control policies manually to meet high-level objectives is often time-consuming. Bayesian optimization provides a data-efficient framework for automating this process using numerical evaluations of an objective function. However, many systems, particularly those involving humans, require optimization based on subjective criteria. Preferential Bayesian optimization addresses this by learning from pairwise comparisons instead of quantitative measurements, but relying solely on preference data can be inefficient. We propose a multi-fidelity, multi-modal Bayesian optimization framework that integrates low-fidelity numerical data with high-fidelity human preferences. Our approach employs Gaussian process surrogate models with both hierarchical, autoregressive and non-hierarchical, coregionalization-based structures, enabling efficient learning from mixed-modality data. We illustrate the framework by tuning an autonomous vehicle's trajectory planner, showing that combining numerical and...