Artificial Intelligence (AI) systems planned for deployment in real-world applications frequently are researched and developed in closed simulation environments where all variables are controlled and known to the simulator or labeled benchmark datasets are used. Transition from these simulators, testbeds, and benchmark datasets to more open-world domains poses significant challenges to AI systems, including significant increases in the complexity of the domain and the inclusion of real-world novelties; the open-world environment contains numerous out-of-distribution elements that are not part in the AI systems' training set. Here, we propose a path to a general, domain-independent measure of domain complexity level. We distinguish two aspects of domain complexity: intrinsic and extrinsic. The intrinsic domain complexity is the complexity that exists by itself without any action or interaction from an AI agent performing a task on that domain. This is an agent-independent aspect of the domain complexity. The extrinsic domain complexity is agent- and task-dependent. Intrinsic and extrinsic elements combined capture the overall complexity of the domain. We frame the components that define and impact domain complexity levels in a domain-independent light. Domain-independent measures of complexity could enable quantitative predictions of the difficulty posed to AI systems when transitioning from one testbed or environment to another, when facing out-of-distribution data in open-world tasks, and when navigating the rapidly expanding solution and search spaces encountered in open-world domains.
翻译:计划在现实世界应用中部署的人工智能(AI)系统经常在封闭的模拟环境中进行研究和开发,在封闭的模拟环境中,所有变量都得到控制,并被模拟器或标签基准数据集所了解。从这些模拟器、测试床和基准数据集向更开放世界域的过渡,给AI系统带来重大挑战,包括领域复杂性的大幅增加和纳入现实世界的新颖之处;开放世界环境包含许多非AI系统培训内容的公开分配要素。在这里,我们建议了一条通向一个通用的、视域复杂程度为独立的计量的路径。我们区分了领域复杂程度的两个方面:内在的和外部的。内在领域的复杂性是本身存在的复杂程度,而没有执行该领域任务的AI代理机构的任何行动或互动。这是域复杂性的一个依赖代理方的方面。外部域复杂性是代理和任务,不属于AI系统的培训范围。当一个域域域域的复杂程度从一个域域域的直流中定义和影响数据流流变的复杂程度,当一个域域域域内预测的复杂程度从一个域际系统到另一个光向另一个域内测试环境时,我们设置了该域内独立的复杂程度的构成部分。</s>