[2410.09355] On Divergence Measures for Training GFlowNets

Computer Science > Machine Learning arXiv:2410.09355 (cs) [Submitted on 12 Oct 2024 (v1), last revised 21 Oct 2024 (this version, v2)] Title:On Divergence Measures for Training GFlowNets Authors:Tiago da Silva, Eliezer de Souza da Silva, Diego Mesquita View a PDF of the paper titled On Divergence Measures for Training GFlowNets, by Tiago da Silva and 2 other authors View PDF HTML (experimental) Abstract:Generative Flow Networks (GFlowNets) are amortized inference models designed to sample from unnormalized distributions over composable objects, with applications in generative modeling for tasks in fields such as causal discovery, NLP, and drug discovery. Traditionally, the training procedure for GFlowNets seeks to minimize the expected log-squared difference between a proposal (forward policy) and a target (backward policy) distribution, which enforces certain flow-matching conditions. While this training procedure is closely related to variational inference (VI), directly attempting standard Kullback-Leibler (KL) divergence minimization can lead to proven biased and potentially high-variance estimators. Therefore, we first review four divergence measures, namely, Renyi-$\alpha$'s, Tsallis-$\alpha$'s, reverse and forward KL's, and design statistically efficient estimators for their stochastic gradients in the context of training GFlowNets. Then, we verify that properly minimizing these divergences yields a provably correct and empirically effective training scheme, often l...