Consumer grade cyber-physical systems (CPS) are becoming an integral part of our life, automatizing and simplifying everyday tasks. Indeed, due to complex interactions between hardware, networking and software, developing and testing such systems is known to be a challenging task. Various quality assurance and testing strategies have been proposed. The most common approach for pre-deployment testing is to model the system and run simulations with models or software in the loop. In practice, most often, tests are run for a small number of simulations, which are selected based on the engineers' domain knowledge and experience. In this paper we propose an approach to automatically generate fault-revealing test cases for CPS. We have implemented our approach in Python, using standard frameworks and used it to generate scenarios violating temperature constraints for a smart thermostat implemented as a part of our IoT testbed. Data collected from an application managing a smart building have been used to learn models of the environment under ever changing conditions. The suggested approach allowed us to identify several pit-fails, scenarios (i.e., environment conditions and inputs), where the system behaves not as expected.
翻译:事实上,由于硬件、网络和软件之间的复杂互动,开发和测试这些系统被认为是一项具有挑战性的任务。提出了各种质量保证和测试战略。最常用的部署前测试方法是模拟系统,并使用循环中的模型或软件进行模拟。在实践中,大多数情况下,测试是为少量的模拟进行,这些模拟是根据工程师的域知识和经验选择的。在本文中,我们提议了一种办法,为计算机辅助系统自动生成错误读取测试案例。我们在Python采用了我们的方法,使用标准框架,并利用它来生成违反温度约束的假想,以建立智能自动自动调温器,作为我们IoT测试台的一部分。从一个管理智能建筑的应用中收集的数据被用于在不断变化的条件下学习环境模型。建议的方法使我们能够在系统没有达到预期的状态时发现一些坑洞、情景(即环境条件和投入)。