Fuzzing -- whether generating or mutating inputs -- has found many bugs and security vulnerabilities in a wide range of domains. Stateful and highly structured web APIs present significant challenges to traditional fuzzing techniques, as execution feedback is usually limited to a response code instead of code coverage and vulnerabilities of interest include silent information-disclosure in addition to explicit errors. Our tool, Schemathesis, derives structure- and semantics-aware fuzzers from web API schemas in the OpenAPI or GraphQL formats, using property-based testing tools. Derived fuzzers can be incorporated into unit-test suites or run directly, with or without end-user customisation of data generation and semantic checks. We construct the most comprehensive evaluation of web API fuzzers to date, running eight fuzzers against sixteen real-world open source web services. OpenAPI schemas found in the wild have a long tail of rare features and complex structures. Of the tools we evaluated, Schemathesis was the only one to handle more than two-thirds of our target services without a fatal internal error. Schemathesis finds 1.4 times to 4.5 times more unique defects than the respectively second-best fuzzer for each target, and is the only fuzzer to find defects in four targets.
翻译:模糊性 -- -- 无论是生成还是突变输入 -- -- 在广泛的领域发现了许多错误和安全弱点。 典型和结构严密的网络自动识别信息对传统的模糊技术提出了重大挑战,因为执行反馈通常限于一个响应代码,而不是代码覆盖范围,而关注的弱点包括除明确错误外的隐蔽信息披露。我们的工具Schemathesis,从野生网络的API Schematas格式或图形QL格式的 OpenAPI 格式中产生结构学和语义识别的模糊器,使用基于属性的测试工具。在我们评估的工具中,只有1个能够处理超过三分之二的目标服务,而没有致命的内部错误。我们构建了迄今为止对网络API fuzzers的最全面的评估,对16个真实开放源的网络服务运行了8个模糊器。在野生的 OpenAPI schemats 和复杂结构中有很长的尾巴,在我们评估的工具中,只有1个工具可以处理我们的目标服务中的三分之二以上,没有致命的内部错误,有或没有最终用户对数据生成和语义检查的用户定制。我们每个目标的缺陷只有1.4至4.5次。