![[2407.11907] GraphFM: A generalist graph transformer that learns transferable representations across diverse domains](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2407.11907] GraphFM: A generalist graph transformer that learns transferable representations across diverse domains
Summary
GraphFM introduces a scalable graph transformer that learns transferable representations across diverse domains, enhancing generalization and performance in node classification tasks.
Why It Matters
This research addresses the limitations of traditional graph neural networks (GNNs) that require extensive tuning for specific datasets. By developing a generalist model, GraphFM promotes scalability and adaptability, which is crucial for advancing machine learning applications across various fields such as molecular biology and social networks.
Key Takeaways
- GraphFM enables training on 152 distinct graph datasets, improving generalization.
- The model uses a Perceiver-based encoder to compress features into a shared latent space.
- Pretraining on diverse datasets enhances performance compared to single-source models.
- Combining synthetic and real graphs improves adaptability and stability.
- GraphFM reduces the need for dataset-specific training, streamlining model deployment.
Computer Science > Machine Learning arXiv:2407.11907 (cs) [Submitted on 16 Jul 2024 (v1), last revised 14 Feb 2026 (this version, v2)] Title:GraphFM: A generalist graph transformer that learns transferable representations across diverse domains Authors:Divyansha Lachi, Mehdi Azabou, Vinam Arora, Eva Dyer View a PDF of the paper titled GraphFM: A generalist graph transformer that learns transferable representations across diverse domains, by Divyansha Lachi and 3 other authors View PDF HTML (experimental) Abstract:Graph neural networks (GNNs) are often trained on individual datasets, requiring specialized models and significant hyperparameter tuning due to the unique structures and features of each dataset. This approach limits the scalability and generalizability of GNNs, as models must be tailored for each specific graph type. To address these challenges, we introduce GraphFM, a scalable multi-graph pretraining approach designed for learning across diverse graph datasets. GraphFM uses a Perceiver-based encoder with learned latent tokens to compress domain-specific features into a shared latent space, enabling generalization across graph domains. We propose new techniques for scaling up graph training on datasets of different sizes, allowing us to train GraphFM on 152 distinct graph datasets, containing a total of 7.4 million nodes and 189 million edges. This allows us to study the effect of scale on pretraining across domains such as molecules, citation networks, and prod...
