Liheng Ma

Graph Inductive Biases in Transformers without Message Passing

2023 · arXiv (Cornell University)

Transformers for graph data are increasingly widely studied and successful in numerous learning tasks. Graph inductive biases are crucial for Graph Transformers, and previous works incorporate them using message-passing modules and/or positional encodings. However, Graph …

Plain Transformers Can be Powerful Graph Learners

2025 · arXiv (Cornell University)

Transformers have attained outstanding performance across various modalities, owing to their simple but powerful scaled-dot-product (SDP) attention mechanisms. Researchers have attempted to migrate Transformers to graph learning, but most advanced Graph Transformers (GTs) have strayed …