![[2408.07016] Rethinking Disentanglement under Dependent Factors of Variation](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2408.07016] Rethinking Disentanglement under Dependent Factors of Variation
Summary
This paper proposes a new definition of disentanglement in representation learning that accounts for dependent factors of variation, offering a method to measure disentanglement effectively in real-world scenarios.
Why It Matters
Traditional definitions of disentanglement assume independence among factors of variation, which limits their applicability. This research addresses this gap, providing a more realistic framework for representation learning that can enhance machine learning models' performance in complex environments.
Key Takeaways
- Introduces a new definition of disentanglement based on information theory.
- Addresses the limitations of existing metrics that assume independence among factors.
- Proposes a novel method for measuring disentanglement with dependent factors.
- Demonstrates the effectiveness of the new method through experimental validation.
- Highlights the importance of realistic assumptions in representation learning.
Computer Science > Machine Learning arXiv:2408.07016 (cs) [Submitted on 13 Aug 2024 (v1), last revised 24 Feb 2026 (this version, v3)] Title:Rethinking Disentanglement under Dependent Factors of Variation Authors:Antonio Almudévar, Alfonso Ortega View a PDF of the paper titled Rethinking Disentanglement under Dependent Factors of Variation, by Antonio Almud\'evar and Alfonso Ortega View PDF HTML (experimental) Abstract:Representation learning is an approach that allows to discover and extract the factors of variation from the data. Intuitively, a representation is said to be disentangled if it separates the different factors of variation in a way that is understandable to humans. Definitions of disentanglement and metrics to measure it usually assume that the factors of variation are independent of each other. However, this is generally false in the real world, which limits the use of these definitions and metrics to very specific and unrealistic scenarios. In this paper we give a definition of disentanglement based on information theory that is also valid when the factors of variation are not independent. Furthermore, we relate this definition to the Information Bottleneck Method. Finally, we propose a method to measure the degree of disentanglement from the given definition that works when the factors of variation are not independent. We show through different experiments that the method proposed in this paper correctly measures disentanglement with non-independent facto...
