μ-Bench: Real-world Micro Benchmarking for SPARQL Query Processing over Knowledge Graphs

Real-world SPARQL querying benchmarks, which make use of the real-world RDF datasets and/or SPARQL queries, are the key element in testing the performance of different RDF Knowledge graph management systems in real-world settings. Over the last years, various real-world SPARQL querying benchmarks have been proposed. Although useful for general purpose SPARQL benchmarking, they do not allow generating microbenchmarks, i.e., small customized benchmarks according to the user specified criteria for a specific use case. These microbenchmarks are important to perform component-based testing, hence pinpoint pros and cons of the systems at micro level. We propose μ-Bench, a microbenchmarking framework for SPARQL query processing over RDF knowledge graphs. The framework makes use of the real-world (collected from query logs of public SPARQL endpoints) SPARQL queries to generate customized benchmarks according to the user defined criteria. The framework utilizes various clustering algorithms to select diverse benchmarks from the given input query log. We generated various microbenchmarks and evaluated state-of-the-art knowledge graph engines. The evaluation results show that specialized microbenchmarking is crucial for identifying the limitations of the various SPARQL query processing engines and other corresponding components.