[2507.04754] Intervening to Learn and Compose Causally Disentangled Representations

Statistics > Machine Learning arXiv:2507.04754 (stat) [Submitted on 7 Jul 2025 (v1), last revised 2 Apr 2026 (this version, v2)] Title:Intervening to Learn and Compose Causally Disentangled Representations Authors:Alex Markham, Isaac Hirsch, Jeri A. Chang, Liam Solus, Bryon Aragam View a PDF of the paper titled Intervening to Learn and Compose Causally Disentangled Representations, by Alex Markham and 4 other authors View PDF HTML (experimental) Abstract:In designing generative models, it is commonly believed that in order to learn useful latent structure, we face a fundamental tension between expressivity and structure. In this paper we challenge this view by proposing a new approach to training arbitrarily expressive generative models that simultaneously learn causally disentangled concepts. This is accomplished by adding a simple context module to an arbitrarily complex black-box model, which learns to process concept information by implicitly inverting linear representations from the model's encoder. Inspired by the notion of intervention in a causal model, our module selectively modifies its architecture during training, allowing it to learn a compact joint model over different contexts. We show how adding this module leads to causally disentangled representations that can be composed for out-of-distribution generation on both real and simulated data. The resulting models can be trained end-to-end or fine-tuned from pre-trained models. To further validate our proposed...