[2503.01804] $\texttt{SEM-CTRL}$: Semantically Controlled Decoding

Computer Science > Computation and Language arXiv:2503.01804 (cs) [Submitted on 3 Mar 2025 (v1), last revised 3 Mar 2026 (this version, v3)] Title:$\texttt{SEM-CTRL}$: Semantically Controlled Decoding Authors:Mohammad Albinhassan, Pranava Madhyastha, Alessandra Russo View a PDF of the paper titled $\texttt{SEM-CTRL}$: Semantically Controlled Decoding, by Mohammad Albinhassan and 1 other authors View PDF Abstract:Ensuring both syntactic and semantic correctness in Large Language Model (LLM) outputs remains a significant challenge, despite being critical for real-world deployment. In this paper, we introduce \texttt{SEM-CTRL}, a unified approach that allows for enforcing rich context-sensitive constraints, and task and instance specific semantics directly on the LLM decoder. Our approach integrates token-level MCTS which is guided by specific syntactic and semantic constraints. The constraints over desired outputs are expressed using Answer Set Grammars, which is a logic-based formalism that generalizes context sensitive grammars while incorporating background knowledge to represent task-specific semantics. We show that our approach helps guarantee valid completions for any off-the-shelf LLM without the need for fine-tuning. We evaluate \texttt{SEM-CTRL} on a range of tasks, including synthetic grammar synthesis, combinatorial reasoning, JSON parsing, and planning. Our experimental results demonstrate that \texttt{SEM-CTRL} allows even small pre-trained LLMs to efficiently o...