Deriving Semantics-Aware Fuzzers from Web API Schemas

We present Schemathesis, a tool for finding semantic errors and crashes in OpenAPI or GraphQL web APIs through property-based testing. Our evaluation, thirty independent runs of eight tools against sixteen containerized open-source web services, shows that Schemathesis wildly outperforms all previous tools.It is the only tool to find defects in four targets, finds 1.4× to 4.5× more unique defects than the respectively second-best tool for each remaining target, and is the only tool to handle more than two-thirds of our target services without a fatal internal error.Our full preprint [5] goes into considerably more detail.