We propose an adversarial, time-varying test-synthesis procedure for safety-critical systems without requiring specific knowledge of the underlying controller steering the system. From a broader test and evaluation context, determination of difficult tests of system behavior is important as these tests would elucidate problematic system phenomena before these mistakes can engender problematic outcomes, e.g. loss of human life in autonomous cars, costly failures for airplane systems, etc. Our approach builds on existing, simulation-based work in the test and evaluation literature by offering a controller-agnostic test-synthesis procedure that provides a series of benchmark tests with which to determine controller reliability. To achieve this, our approach codifies the system objective as a timed reach-avoid specification. Then, by coupling control barrier functions with this class of specifications, we construct an instantaneous difficulty metric whose minimizer corresponds to the most difficult test at that system state. We use this instantaneous difficulty metric in a game-theoretic fashion, to produce an adversarial, time-varying test-synthesis procedure that does not require specific knowledge of the system's controller, but can still provably identify realizable and maximally difficult tests of system behavior. Finally, we develop this test-synthesis procedure for both continuous and discrete-time systems and showcase our test-synthesis procedure on simulated and hardware examples.
翻译:我们建议对安全临界系统采用对抗性的、时间变化式的测试合成程序,而不需要对控制器指导系统的具体知识。从更广泛的测试和评价角度来说,确定系统行为的困难测试很重要,因为这些测试将澄清有问题的系统现象,以免这些错误产生问题的结果,例如,自主汽车中的人命损失、飞机系统费用高昂的故障等。我们的方法以测试和评价文献中现有的模拟性测试合成程序为基础,提供一种控制器-不可知性测试合成程序,提供一系列基准测试,用以确定控制器的可靠性。为了实现这一点,我们的方法将系统目标编码成一个及时达到的规格。然后,通过将控制屏障功能与这种规格相结合,我们建立一个瞬间困难度的测量标准,其最小化与系统状态上最困难的测试相匹配。我们用这种瞬间困难度测量方法来产生一种对抗性、时间变化式测试合成的测试程序,它并不需要系统控制器的具体知识,但是仍然可以准确地确定系统目标化的达标点目标。我们最后和最难的测试程序是持续和最难的测试系统。