[2504.02018] Geometric Reasoning in the Embedding Space

Computer Science > Machine Learning arXiv:2504.02018 (cs) [Submitted on 2 Apr 2025 (v1), last revised 1 Mar 2026 (this version, v3)] Title:Geometric Reasoning in the Embedding Space Authors:Jan Hůla, David Mojžíšek, Jiří Janeček, David Herel, Mikoláš Janota View a PDF of the paper titled Geometric Reasoning in the Embedding Space, by Jan H\r{u}la and 4 other authors View PDF HTML (experimental) Abstract:In this contribution, we demonstrate that Graph Neural Networks and Transformers can learn to reason about geometric constraints. We train them to predict spatial position of points in a discrete 2D grid from a set of constraints that uniquely describe hidden figures containing these points. Both models are able to predict the position of points and interestingly, they form the hidden figures described by the input constraints in the embedding space during the reasoning process. Our analysis shows that both models recover the grid structure during training so that the embeddings corresponding to the points within the grid organize themselves in a 2D subspace and reflect the neighborhood structure of the grid. We also show that the Graph Neural Network we design for the task performs significantly better than the Transformer and is also easier to scale. Comments: Subjects: Machine Learning (cs.LG) Cite as: arXiv:2504.02018 [cs.LG]   (or arXiv:2504.02018v3 [cs.LG] for this version)   https://doi.org/10.48550/arXiv.2504.02018 Focus to learn more arXiv-issued DOI via DataCite J...