[2509.21465] Talking Trees: Reasoning-Assisted Induction of Decision Trees for Tabular Data

Computer Science > Machine Learning arXiv:2509.21465 (cs) [Submitted on 25 Sep 2025 (v1), last revised 4 Mar 2026 (this version, v2)] Title:Talking Trees: Reasoning-Assisted Induction of Decision Trees for Tabular Data Authors:George Yakushev, Alina Shutova, Ivan Rubachev, Natalia Bereberdina, Renat Sergazinov, Artem Babenko View a PDF of the paper titled Talking Trees: Reasoning-Assisted Induction of Decision Trees for Tabular Data, by George Yakushev and 5 other authors View PDF HTML (experimental) Abstract:Tabular foundation models are becoming increasingly popular for low-resource tabular problems. These models compensate for small training datasets by pretraining on large volumes of data. The prior knowledge obtained via pretraining provides exceptional performance, but the resulting model becomes a black box that is difficult to interpret and costly to run inference on. In this work, we explore an alternative strategy that is both more lightweight and controllable: using reasoning-capable LLMs to induce decision trees for small tabular datasets in an agentic setup. We design a minimal set of tools for constructing, analyzing, and manipulating decision trees. Using these tools, an LLM agent combines its prior knowledge with the user-specified constraints and learning from data to create lightweight decision trees. We show that a single decision tree constructed via the agentic loop can be competitive with state-of-the-art black-box models on tabular benchmarks, while ...