[2210.09709] Importance Weighting Correction of Regularized Least-Squares for Target Shift

Statistics > Machine Learning arXiv:2210.09709 (stat) [Submitted on 18 Oct 2022 (v1), last revised 3 Mar 2026 (this version, v3)] Title:Importance Weighting Correction of Regularized Least-Squares for Target Shift Authors:Davit Gogolashvili View a PDF of the paper titled Importance Weighting Correction of Regularized Least-Squares for Target Shift, by Davit Gogolashvili View PDF HTML (experimental) Abstract:Importance weighting is a standard tool for correcting distribution shift, but its statistical behavior under target shift -- where the label distribution changes between training and testing while the conditional distribution of inputs given the label remains stable -- remains under-explored. We analyze importance-weighted kernel ridge regression under target shift and show that, because the weights depend only on the output variable, reweighting corrects the train-test mismatch without altering the input-space complexity that governs kernel generalization. Under standard RKHS regularity and capacity conditions and a mild Bernstein-type moment condition on the label weights, we obtain finite-sample guarantees showing that the estimator achieves the same convergence behavior as in the no-shift case, with shift severity affecting only the constants through weight moments. We complement these results with matching minimax lower bounds, establishing rate optimality and quantifying the unavoidable dependence on shift severity. We further study more general weighting schemes...