Artificial intelligence (AI) has the potential to greatly improve society, but as with any powerful technology, it comes with heightened risks and responsibilities. Current AI research lacks a systematic discussion of how to manage long-tail risks from AI systems, including speculative long-term risks. Keeping in mind the potential benefits of AI, there is some concern that building ever more intelligent and powerful AI systems could eventually result in systems that are more powerful than us; some say this is like playing with fire and speculate that this could create existential risks (x-risks). To add precision and ground these discussions, we provide a guide for how to analyze AI x-risk, which consists of three parts: First, we review how systems can be made safer today, drawing on time-tested concepts from hazard analysis and systems safety that have been designed to steer large processes in safer directions. Next, we discuss strategies for having long-term impacts on the safety of future systems. Finally, we discuss a crucial concept in making AI systems safer by improving the balance between safety and general capabilities. We hope this document and the presented concepts and tools serve as a useful guide for understanding how to analyze AI x-risk.
翻译:人工智能(AI)有可能大大改善社会,但正如任何强大的技术一样,它带来更大的风险和责任。目前人工智能研究缺乏关于如何管理人工智能系统的长期风险(包括投机性的长期风险)的系统讨论。考虑到人工智能的潜在好处,有人担心,建立比我们更聪明和强大的人工智能系统最终会导致比我们更强大的系统;有人说,这就像玩火和猜测这可能造成生存风险(x-风险)一样。为了增加精确度和基础性的讨论,我们为如何分析人工智能x风险提供了指南,其中包括三个部分:第一,我们审查如何使系统今天变得更加安全,借鉴从危险分析和系统安全角度设计来引导大过程的经过时间考验的概念。接下来,我们讨论对未来系统安全产生长期影响的战略。最后,我们讨论通过改善安全和一般能力之间的平衡使人工系统更安全的关键概念。我们希望这份文件和所提出的概念和工具成为了解如何分析人工智能x-风险的有用指南。