Dinkel: Testing Graph Database Engines via State-Aware Query Generation

Graph database management systems (GDBMSs) store and manipulate graph data and form a core part of many data-driven applications. To ensure their reliability, several approaches have been proposed to test GDBMSs by generating queries in Cypher, the most popular graph query language. However, Cypher allows queries with complicated state changes and data dependencies, which existing approaches do not support and thus fail to generate valid, complex queries, thereby missing many bugs in GDBMSs. In this paper, we propose a novel state-aware testing approach to generate complex Cypher queries for GDBMSs. Our approach models two kinds of graph state, query context and graph schema. Query context describes the available Cypher variables and their corresponding scopes, whereas graph schema summarizes the manipulated graph labels and properties. While generating Cypher queries, we modify the graph states on the fly to ensure each clause within the query can reference the correct state information. In this way, our approach can generate Cypher queries with multiple state changes and complicated data dependencies while retaining high query validity. We implemented this approach as a fully automatic GDBMS testing framework, Dinkel, and evaluated it on three popular open-source GDBMSs, namely Neo4j, RedisGraph, and Apache AGE. In total, Dinkel found 60 bugs, among which 58 were confirmed and 51 fixed. Our evaluation results show that Dinkel can effectively generate complex queries with high validity (93.43%). Compared to existing approaches, Dinkel can cover over 60% more code and find more bugs within the 48-hour testing campaign. We expect Dinkel's powerful test-case generation to benefit GDBMS testing and help strengthen the reliability of GDBMSs.