[2412.19496] Multi-PA: A Multi-perspective Benchmark on Privacy Assessment for Large Vision-Language Models

Computer Science > Cryptography and Security arXiv:2412.19496 (cs) [Submitted on 27 Dec 2024 (v1), last revised 2 Mar 2026 (this version, v4)] Title:Multi-PA: A Multi-perspective Benchmark on Privacy Assessment for Large Vision-Language Models Authors:Jie Zhang, Xiangkui Cao, Zhouyu Han, Shiguang Shan, Xilin Chen View a PDF of the paper titled Multi-PA: A Multi-perspective Benchmark on Privacy Assessment for Large Vision-Language Models, by Jie Zhang and 4 other authors View PDF HTML (experimental) Abstract:Large Vision-Language Models (LVLMs) exhibit impressive potential across various tasks but also face significant privacy risks, limiting their practical applications. Current researches on privacy assessment for LVLMs is limited in scope, with gaps in both assessment dimensions and privacy categories. To bridge this gap, we propose Multi-PA, a comprehensive benchmark for evaluating the privacy preservation capabilities of LVLMs in terms of privacy awareness and leakage. Privacy awareness measures the model's ability to recognize the privacy sensitivity of input data, while privacy leakage assesses the risk of the model unintentionally disclosing privacy information in its output. We design a range of sub-tasks to thoroughly evaluate the model's privacy protection offered by LVLMs. Multi-PA covers 26 categories of personal privacy, 15 categories of trade secrets, and 18 categories of state secrets, totaling 31,962 samples. Based on Multi-PA, we evaluate the privacy prese...