![[2509.22794] Differentially Private Two-Stage Gradient Descent for Instrumental Variable Regression](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2509.22794] Differentially Private Two-Stage Gradient Descent for Instrumental Variable Regression
Summary
This paper presents a novel algorithm for instrumental variable regression that ensures differential privacy while maintaining statistical efficiency, marking a significant advancement in privacy-preserving machine learning techniques.
Why It Matters
As data privacy concerns grow, this research addresses the critical need for methods that protect sensitive information in statistical analyses. The proposed algorithm balances privacy and accuracy, making it relevant for fields like economics and machine learning where sensitive data is prevalent.
Key Takeaways
- Introduces a differentially private two-stage gradient descent algorithm for instrumental variable regression.
- Establishes finite-sample convergence rates, ensuring consistency while preserving privacy.
- Quantifies the trade-off between optimization, privacy, and sampling error.
- First work to provide privacy guarantees and provable convergence rates for this regression method.
- Validates theoretical findings with experiments on both synthetic and real datasets.
Statistics > Machine Learning arXiv:2509.22794 (stat) [Submitted on 26 Sep 2025 (v1), last revised 15 Feb 2026 (this version, v3)] Title:Differentially Private Two-Stage Gradient Descent for Instrumental Variable Regression Authors:Haodong Liang, Yanhao Jin, Krishnakumar Balasubramanian, Lifeng Lai View a PDF of the paper titled Differentially Private Two-Stage Gradient Descent for Instrumental Variable Regression, by Haodong Liang and 3 other authors View PDF HTML (experimental) Abstract:We study instrumental variable regression (IVaR) under differential privacy constraints. Classical IVaR methods (like two-stage least squares regression) rely on solving moment equations that directly use sensitive covariates and instruments, creating significant risks of privacy leakage and posing challenges in designing algorithms that are both statistically efficient and differentially private. We propose a noisy two-stage gradient descent algorithm that ensures $\rho$-zero-concentrated differential privacy by injecting carefully calibrated noise into the gradient updates. Our analysis establishes finite-sample convergence rates for the proposed method, showing that the algorithm achieves consistency while preserving privacy. In particular, we derive precise bounds quantifying the trade-off among optimization, privacy, and sampling error. To the best of our knowledge, this is the first work to provide both privacy guarantees and provable convergence rates for instrumental variable regr...
