This report summarizes the second International Verification of Neural Networks Competition (VNN-COMP 2021), held as a part of the 4th Workshop on Formal Methods for ML-Enabled Autonomous Systems that was collocated with the 33rd International Conference on Computer-Aided Verification (CAV). Twelve teams participated in this competition. The goal of the competition is to provide an objective comparison of the state-of-the-art methods in neural network verification, in terms of scalability and speed. Along this line, we used standard formats (ONNX for neural networks and VNNLIB for specifications), standard hardware (all tools are run by the organizers on AWS), and tool parameters provided by the tool authors. This report summarizes the rules, benchmarks, participating tools, results, and lessons learned from this competition.
翻译:本报告概述了第二次神经网络竞争国际核查(VNNN-COMP 2021),这次国际核查是作为第4次ML-Enable自治系统正式方法讲习班的一部分举行的,与第33次计算机辅助核查国际会议(CAV)合用同一地点。12个小组参加了这一竞争。竞争的目的是客观地比较神经网络核查的最新方法的可伸缩性和速度。在这方面,我们使用了标准格式(神经网络的ONNX和用于规格的VNNLIB)、标准硬件(所有工具都由AWS的组织者操作)以及工具作者提供的工具参数。本报告概述了规则、基准、参与工具、结果和从这一竞争中吸取的经验教训。