[2502.19463] Hedging and Non-Affirmation: Quantifying LLM Alignment on Questions of Human Rights

Computer Science > Computers and Society arXiv:2502.19463 (cs) [Submitted on 26 Feb 2025 (v1), last revised 6 Apr 2026 (this version, v2)] Title:Hedging and Non-Affirmation: Quantifying LLM Alignment on Questions of Human Rights Authors:Rafiya Javed, Cassandra Parent, Jackie Kay, David Yanni, Abdullah Zaini, Anushe Sheikh, Maribeth Rauh, Walter Gerych, Ramona Comanescu, Iason Gabriel, Marzyeh Ghassemi, Laura Weidinger View a PDF of the paper titled Hedging and Non-Affirmation: Quantifying LLM Alignment on Questions of Human Rights, by Rafiya Javed and Cassandra Parent and Jackie Kay and David Yanni and Abdullah Zaini and Anushe Sheikh and Maribeth Rauh and Walter Gerych and Ramona Comanescu and Iason Gabriel and Marzyeh Ghassemi and Laura Weidinger View PDF HTML (experimental) Abstract:Hedging and non-affirmation are behaviors exhibited by large language models (LLMs) that limit the clear endorsement of specific statements. While these behaviors are desirable in subjective contexts, they are undesirable in the context of human rights - which apply unambiguously to all groups. We present a systematic framework to measure these behaviors in unconstrained LLM responses regarding various identity groups. We evaluate six large proprietary models as well as one open-weight LLM on 4738 prompts across 205 national and stateless ethnic identities and find that 4 out of 7 display hedging and non-affirmation that is significantly dependent on the identity of the group. While factors ...