![[2602.19169] Virtual Parameter Sharpening: Dynamic Low-Rank Perturbations for Inference-Time Reasoning Enhancement](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.19169] Virtual Parameter Sharpening: Dynamic Low-Rank Perturbations for Inference-Time Reasoning Enhancement
Summary
The paper introduces Virtual Parameter Sharpening (VPS), a novel technique for enhancing inference-time reasoning in transformer models through dynamic low-rank perturbations based on activation statistics.
Why It Matters
VPS offers a new approach to improve the adaptability and reasoning capabilities of large language models without the need for persistent parameter updates. This is particularly relevant as the demand for efficient and effective AI models continues to grow, especially in applications requiring real-time reasoning.
Key Takeaways
- VPS enhances frozen transformer layers with dynamic low-rank perturbations.
- The technique allows for test-time adaptation without persistent updates.
- It utilizes activation statistics for constructing perturbation factors.
- The paper includes a theoretical analysis of the perturbation's properties.
- An adaptive policy system modulates perturbation based on activation energy.
Computer Science > Machine Learning arXiv:2602.19169 (cs) [Submitted on 2 Dec 2025] Title:Virtual Parameter Sharpening: Dynamic Low-Rank Perturbations for Inference-Time Reasoning Enhancement Authors:Saba Kublashvili View a PDF of the paper titled Virtual Parameter Sharpening: Dynamic Low-Rank Perturbations for Inference-Time Reasoning Enhancement, by Saba Kublashvili View PDF HTML (experimental) Abstract:I introduce Virtual Parameter Sharpening (VPS), an inference-time technique that augments frozen transformer linear layers with dynamic, activation-conditioned low-rank perturbations. Unlike parameter-efficient fine-tuning methods such as LoRA, which learn static low-rank adapters, VPS constructs its perturbation factors on the fly from batch activation statistics and optional gradient signals, enabling test-time adaptation without persistent parameter updates. The perturbation takes the form Delta W = gamma * W^T V U^T W, where selector matrices U and V are constructed via sparse activation-guided selection or Sylvester-coupled regression. We provide a theoretical analysis of the perturbation's spectral properties and describe an adaptive policy system that modulates perturbation magnitude based on activation energy and token-level entropy. This system incorporates multi-objective verification with iterative refinement for tasks with ground-truth supervision. We present the complete algorithmic framework, analyze its mathematical foundations, and discuss the mechanisms b...