[2510.08580] LadderSym: A Multimodal Interleaved Transformer for Music Practice Error Detection

Computer Science > Sound arXiv:2510.08580 (cs) [Submitted on 16 Sep 2025 (v1), last revised 4 Mar 2026 (this version, v2)] Title:LadderSym: A Multimodal Interleaved Transformer for Music Practice Error Detection Authors:Benjamin Shiue-Hal Chou, Purvish Jajal, Nick John Eliopoulos, James C. Davis, George K. Thiruvathukal, Kristen Yeon-Ji Yun, Yung-Hsiang Lu View a PDF of the paper titled LadderSym: A Multimodal Interleaved Transformer for Music Practice Error Detection, by Benjamin Shiue-Hal Chou and 6 other authors View PDF HTML (experimental) Abstract:Music learners can greatly benefit from tools that accurately detect errors in their practice. Existing approaches typically compare audio recordings to music scores using heuristics or learnable models. This paper introduces LadderSym, a novel Transformer-based method for music error detection. LadderSym is guided by two key observations about the state-of-the-art approaches: (1) late fusion limits inter-stream alignment and cross-modality comparison capability; and (2) reliance on score audio introduces ambiguity in the frequency spectrum, degrading performance in music with concurrent notes. To address these limitations, LadderSym introduces (1) a two-stream encoder with inter-stream alignment modules to improve audio comparison capabilities and error detection F1 scores, and (2) a multimodal strategy that leverages both audio and symbolic scores by incorporating symbolic representations as decoder prompts, reducing ambig...