[2603.03805] Relational In-Context Learning via Synthetic Pre-training with Structural Prior

Computer Science > Machine Learning arXiv:2603.03805 (cs) [Submitted on 4 Mar 2026] Title:Relational In-Context Learning via Synthetic Pre-training with Structural Prior Authors:Yanbo Wang, Jiaxuan You, Chuan Shi, Muhan Zhang View a PDF of the paper titled Relational In-Context Learning via Synthetic Pre-training with Structural Prior, by Yanbo Wang and 3 other authors View PDF HTML (experimental) Abstract:Relational Databases (RDBs) are the backbone of modern business, yet they lack foundation models comparable to those in text or vision. A key obstacle is that high-quality RDBs are private, scarce and structurally heterogeneous, making internet-scale pre-training infeasible. To overcome this data scarcity, We introduce $\textbf{RDB-PFN}$, the first relational foundation model trained purely via $\textbf{synthetic data}$. Inspired by Prior-Data Fitted Networks (PFNs) where synthetic data generated from Structural Causal Models (SCMs) enables reasoning on single tables, we design a $\textbf{Relational Prior Generator}$ to create an infinite stream of diverse RDBs from scratch. Pre-training on $\textbf{over 2 million}$ synthetic single-table and relational tasks, RDB-PFN learns to adapt to any new database instantly via genuine $\textbf{in-context learning}$. Experiments verify RDB-PFN achieves strong few-shot performance on 19 real-world relational prediction tasks, outperforming graph-based and single-table foundation-model baselines (given the same DFS-linearized inputs)...