![[2411.02137] Finite-sample performance of the maximum likelihood estimator in logistic regression](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2411.02137] Finite-sample performance of the maximum likelihood estimator in logistic regression
Summary
This article examines the finite-sample performance of the maximum likelihood estimator (MLE) in logistic regression, focusing on its existence and accuracy under various conditions.
Why It Matters
Understanding the performance of MLE in logistic regression is crucial for statisticians and data scientists, as it impacts the reliability of predictive models in real-world applications. This research provides non-asymptotic guarantees that enhance the theoretical foundation of logistic regression, particularly in scenarios with non-Gaussian covariates and model misspecification.
Key Takeaways
- The existence of MLE depends on the dataset not being linearly separated.
- Non-asymptotic guarantees are provided for MLE's accuracy under Gaussian covariates.
- Results are generalized to non-Gaussian covariates and misspecified models.
- The sensitivity of MLE behavior is highlighted in Bernoulli design contexts.
- Understanding these properties aids in improving logistic regression applications.
Mathematics > Statistics Theory arXiv:2411.02137 (math) [Submitted on 4 Nov 2024 (v1), last revised 19 Feb 2026 (this version, v3)] Title:Finite-sample performance of the maximum likelihood estimator in logistic regression Authors:Hugo Chardon, Matthieu Lerasle, Jaouad Mourtada View a PDF of the paper titled Finite-sample performance of the maximum likelihood estimator in logistic regression, by Hugo Chardon and 2 other authors View PDF HTML (experimental) Abstract:Logistic regression is a classical model for describing the probabilistic dependence of binary responses to multivariate covariates. We consider the predictive performance of the maximum likelihood estimator (MLE) for logistic regression, assessed in terms of logistic risk. We consider two questions: first, that of the existence of the MLE (which occurs when the dataset is not linearly separated), and second, that of its accuracy when it exists. These properties depend on both the dimension of covariates and the signal strength. In the case of Gaussian covariates and a well-specified logistic model, we obtain sharp non-asymptotic guarantees for the existence and excess logistic risk of the MLE. We then generalize these results in two ways: first, to non-Gaussian covariates satisfying a certain two-dimensional margin condition, and second to the general case of statistical learning with a possibly misspecified logistic model. Finally, we consider the case of a Bernoulli design, where the behavior of the MLE is hi...
