[2603.19742] Dual Path Attribution: Efficient Attribution for SwiGLU-Transformers through Layer-Wise Target Propagation

Computer Science > Machine Learning arXiv:2603.19742 (cs) [Submitted on 20 Mar 2026] Title:Dual Path Attribution: Efficient Attribution for SwiGLU-Transformers through Layer-Wise Target Propagation Authors:Lasse Marten Jantsch, Dong-Jae Koh, Seonghyeon Lee, Young-Kyoon Suh View a PDF of the paper titled Dual Path Attribution: Efficient Attribution for SwiGLU-Transformers through Layer-Wise Target Propagation, by Lasse Marten Jantsch and 3 other authors View PDF HTML (experimental) Abstract:Understanding the internal mechanisms of transformer-based large language models (LLMs) is crucial for their reliable deployment and effective operation. While recent efforts have yielded a plethora of attribution methods attempting to balance faithfulness and computational efficiency, dense component attribution remains prohibitively expensive. In this work, we introduce Dual Path Attribution (DPA), a novel framework that faithfully traces information flow on the frozen transformer in one forward and one backward pass without requiring counterfactual examples. DPA analytically decomposes and linearizes the computational structure of the SwiGLU Transformers into distinct pathways along which it propagates a targeted unembedding vector to receive the effective representation at each residual position. This target-centric propagation achieves O(1) time complexity with respect to the number of model components, scaling to long input sequences and dense component attribution. Extensive exper...