![[2510.09717] Provable Training Data Identification for Large Language Models](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2510.09717] Provable Training Data Identification for Large Language Models
Summary
This paper presents a novel approach for identifying training data in large language models, addressing issues of copyright and privacy through a statistically reliable method.
Why It Matters
As large language models become integral to various applications, ensuring the integrity and provenance of their training data is crucial for legal compliance and ethical AI practices. This research provides a framework that enhances the reliability of training data identification, which is vital for addressing copyright concerns and privacy audits.
Key Takeaways
- Introduces Provable Training Data Identification (PTDI) for reliable data identification.
- PTDI controls false identification rates using a novel statistical approach.
- Demonstrates improved performance over existing methods across various datasets.
Computer Science > Machine Learning arXiv:2510.09717 (cs) [Submitted on 10 Oct 2025 (v1), last revised 13 Feb 2026 (this version, v2)] Title:Provable Training Data Identification for Large Language Models Authors:Zhenlong Liu, Hao Zeng, Weiran Huang, Hongxin Wei View a PDF of the paper titled Provable Training Data Identification for Large Language Models, by Zhenlong Liu and 3 other authors View PDF HTML (experimental) Abstract:Identifying training data of large-scale models is critical for copyright litigation, privacy auditing, and ensuring fair evaluation. However, existing works typically treat this task as an instance-wise identification without controlling the error rate of the identified set, which cannot provide statistically reliable evidence. In this work, we formalize training data identification as a set-level inference problem and propose Provable Training Data Identification (PTDI), a distribution-free approach that enables provable and strict false identification rate control. Specifically, our method computes conformal p-values for each data point using a set of known unseen data and then develops a novel Jackknife-corrected Beta boundary (JKBB) estimator to estimate the training-data proportion of the test set, which allows us to scale these p-values. By applying the Benjamini-Hochberg (BH) procedure to the scaled p-values, we select a subset of data points with provable and strict false identification control. Extensive experiments across various models ...
