[2603.10252] Bayesian Hierarchical Models and the Maximum Entropy Principle

Statistics > Machine Learning arXiv:2603.10252 (stat) [Submitted on 10 Mar 2026 (v1), last revised 30 Apr 2026 (this version, v2)] Title:Bayesian Hierarchical Models and the Maximum Entropy Principle Authors:Brendon J. Brewer View a PDF of the paper titled Bayesian Hierarchical Models and the Maximum Entropy Principle, by Brendon J. Brewer View PDF HTML (experimental) Abstract:Bayesian hierarchical models are frequently used in practical data analysis contexts. One interpretation of these models is that they provide an indirect way of assigning a prior for unknown parameters, through the introduction of hyperparameters. The resulting marginal prior for the parameters (integrating over the hyperparameters) is usually dependent, so that learning one parameter provides some information about the others. In this contribution, I will demonstrate that, when the prior given the hyperparameters is a canonical distribution (a maximum entropy distribution with moment constraints), the dependent marginal prior also has a maximum entropy property, with a different constraint. This constraint is on the marginal distribution of some function of the unknown quantities. The results shed light on what information is actually being assumed when we assign a hierarchical model. Comments: Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Data Analysis, Statistics and Probability (physics.data-an); Methodology (stat.ME) Cite as: arXiv:2603.10252 [stat.ML]   (or arXiv:2603.10252v2 ...