![[2602.15037] CircuChain: Disentangling Competence and Compliance in LLM Circuit Analysis](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.15037] CircuChain: Disentangling Competence and Compliance in LLM Circuit Analysis
Summary
The paper introduces CircuChain, a benchmark for evaluating large language models (LLMs) in electrical circuit analysis, focusing on their ability to comply with user instructions while maintaining physical reasoning accuracy.
Why It Matters
As LLMs are increasingly used in technical fields, ensuring their outputs align with established conventions is crucial for safety and reliability. CircuChain addresses the gap in evaluating LLMs' adherence to both competence and compliance, which is vital for engineering applications.
Key Takeaways
- CircuChain benchmarks LLMs on their ability to follow instructions in circuit analysis.
- The study reveals a Compliance-Competence Divergence in LLM performance.
- Higher model capability does not guarantee better adherence to methodological conventions.
- The framework provides insights for improving AI alignment in engineering contexts.
- New evaluation methods are necessary for rigorous instruction-following in technical domains.
Computer Science > Software Engineering arXiv:2602.15037 (cs) [Submitted on 29 Jan 2026] Title:CircuChain: Disentangling Competence and Compliance in LLM Circuit Analysis Authors:Mayank Ravishankara View a PDF of the paper titled CircuChain: Disentangling Competence and Compliance in LLM Circuit Analysis, by Mayank Ravishankara View PDF HTML (experimental) Abstract:As large language models (LLMs) advance toward expert-level performance in engineering domains, reliable reasoning under user-specified constraints becomes critical. In circuit analysis, for example, a numerically correct solution is insufficient if it violates established methodological conventions such as mesh directionality or polarity assignments, errors that can propagate in safety-critical systems. Yet it remains unclear whether frontier models truly apply first-principles reasoning or rely on entrenched training priors that conflict with explicit instructions. We introduce CircuChain, a diagnostic benchmark designed to disentangle instruction compliance from physical reasoning competence in electrical circuit analysis. CircuChain consists of counterbalanced Control/Trap problem pairs across five canonical circuit topologies, augmented with systematic variations in sign conventions, current orientations, and polarity definitions. A multi-stage verification pipeline, combining symbolic solvers, SPICE simulation, and an LLM-based error taxonomy, enables fine-grained attribution of failures to convention erro...