Embedding Metadata-Enriched Graphs

This paper presents an on-going research where we study the problem of embedding meta-data enriched graphs, with a focus on knowledge graphs in a vector space with transformer based deep neural networks. Experimentally, we compare ceteris paribus the performance of a transformer-based model with other non-transformer approaches. Due to their recent success in natural language processing we hypothesize that the former is superior in performance. We test this hypothesizes by comparing the performance of transformer embeddings with non- transformer embeddings on different downstream tasks. Our research might contribute to a better understanding of how random walks in- fluence the learning of features, which might be useful in the design of deep learning architectures for graphs when the input is generated with random walks.