[2604.03447] Measuring LLM Trust Allocation Across Conflicting Software Artifacts

Computer Science > Software Engineering arXiv:2604.03447 (cs) [Submitted on 3 Apr 2026] Title:Measuring LLM Trust Allocation Across Conflicting Software Artifacts Authors:Noshin Ulfat, Ahsanul Ameen Sabit, Soneya Binta Hossain View a PDF of the paper titled Measuring LLM Trust Allocation Across Conflicting Software Artifacts, by Noshin Ulfat and 1 other authors View PDF HTML (experimental) Abstract:LLM-based software engineering assistants fail not only by producing incorrect outputs, but also by allocating trust to the wrong artifact when code, documentation, and tests disagree. Existing evaluations focus mainly on downstream outcomes and therefore cannot reveal whether a model recognized degraded evidence, identified the unreliable source, or calibrated its trust across artifacts. We present TRACE (Trust Reasoning over Artifacts for Calibrated Evaluation), a framework that elicits structured artifact-level trust traces over Javadoc, method signatures, implementations, and test prefixes under blind perturbations. Using 22,339 valid traces from seven models on 456 curated Java method bundles, we evaluate per-artifact quality assessment, inconsistency detection, affected artifact attribution, and source prioritization. Across all models, quality penalties are largely localized to the perturbed artifact and increase with severity, but sensitivity is asymmetric across artifact types: documentation bugs induce a substantially larger heavy-to-subtle gap than implementation faul...