[2603.24150] A visual observation on the geometry of UMAP projections of the difference vectors of antonym and synonym word pair embeddings

Computer Science > Computation and Language arXiv:2603.24150 (cs) [Submitted on 25 Mar 2026] Title:A visual observation on the geometry of UMAP projections of the difference vectors of antonym and synonym word pair embeddings Authors:Rami Luisto View a PDF of the paper titled A visual observation on the geometry of UMAP projections of the difference vectors of antonym and synonym word pair embeddings, by Rami Luisto View PDF HTML (experimental) Abstract:Antonyms, or opposites, are sometimes defined as \emph{word pairs that have all of the same contextually relevant properties but one}. Seeing how transformer models seem to encode concepts as directions, this begs the question if one can detect ``antonymity'' in the geometry of the embedding vectors of word pairs, especially based on their difference vectors. Such geometrical studies are then naturally contrasted by comparing antonymic pairs to their opposites; synonyms. This paper started as an exploratory project on the complexity of the systems needed to detect the geometry of the embedding vectors of antonymic word pairs. What we now report is a curious ``swirl'' that appears across embedding models in a somewhat specific projection configuration. Comments: Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG) MSC classes: 68T50 (Primary) 62H30, 68T09 (Secondary) Cite as: arXiv:2603.24150 [cs.CL]   (or arXiv:2603.24150v1 [cs.CL] for this version)   https://doi.org/10.48550/arXiv.2603.24150 Focus to learn ...