![[2506.06060] Simple Yet Effective: Extracting Private Data Across Clients in Federated Fine-Tuning of Large Language Models](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2506.06060] Simple Yet Effective: Extracting Private Data Across Clients in Federated Fine-Tuning of Large Language Models
Summary
This article discusses the privacy risks associated with federated fine-tuning of large language models, highlighting methods for extracting personally identifiable information (PII) from clients' data.
Why It Matters
As federated learning becomes more prevalent in privacy-sensitive fields like healthcare and finance, understanding the vulnerabilities of large language models to data leakage is crucial. This research provides insights into potential threats and establishes a framework for future privacy-preserving efforts.
Key Takeaways
- Federated large language models (FedLLMs) can leak sensitive data across clients.
- The study introduces effective strategies for extracting PII using contextual prefixes.
- Experimental results indicate a significant recovery rate of victim-exclusive PII, raising privacy concerns.
- A new benchmark and evaluation framework for privacy in federated learning is established.
- The findings are relevant for institutions handling sensitive data, emphasizing the need for robust privacy measures.
Computer Science > Computation and Language arXiv:2506.06060 (cs) [Submitted on 6 Jun 2025 (v1), last revised 25 Feb 2026 (this version, v2)] Title:Simple Yet Effective: Extracting Private Data Across Clients in Federated Fine-Tuning of Large Language Models Authors:Yingqi Hu, Zhuo Zhang, Jingyuan Zhang, Jinghua Wang, Qifan Wang, Lizhen Qu, Zenglin Xu View a PDF of the paper titled Simple Yet Effective: Extracting Private Data Across Clients in Federated Fine-Tuning of Large Language Models, by Yingqi Hu and Zhuo Zhang and Jingyuan Zhang and Jinghua Wang and Qifan Wang and Lizhen Qu and Zenglin Xu View PDF HTML (experimental) Abstract:Federated large language models (FedLLMs) enable cross-silo collaborative training among institutions while preserving data locality, making them appealing for privacy-sensitive domains such as law, finance, and healthcare. However, the memorization behavior of LLMs can lead to privacy risks that may cause cross-client data leakage. In this work, we study the threat of cross-client data extraction, where a semi-honest participant attempts to recover personally identifiable information (PII) memorized from other clients' data. We propose three simple yet effective extraction strategies that leverage contextual prefixes from the attacker's local data, including frequency-based prefix sampling and local fine-tuning to amplify memorization. To evaluate these attacks, we construct a Chinese legal-domain dataset with fine-grained PII annotations co...
