Today, many systems use artificial intelligence (AI) to solve complex problems. While this often increases system effectiveness, developing a production-ready AI-based system is a difficult task. Thus, solid AI engineering practices are required to ensure the quality of the resulting system and to improve the development process. While several practices have already been proposed for the development of AI-based systems, detailed practical experiences of applying these practices are rare. In this paper, we aim to address this gap by collecting such experiences during a case study, namely the development of an autonomous stock trading system that uses machine learning functionality to invest in stocks. We selected 10 AI engineering practices from the literature and systematically applied them during development, with the goal to collect evidence about their applicability and effectiveness. Using structured field notes, we documented our experiences. Furthermore, we also used field notes to document challenges that occurred during the development, and the solutions we applied to overcome them. Afterwards, we analyzed the collected field notes, and evaluated how each practice improved the development. Lastly, we compared our evidence with existing literature. Most applied practices improved our system, albeit to varying extent, and we were able to overcome all major challenges. The qualitative results provide detailed accounts about 10 AI engineering practices, as well as challenges and solutions associated with such a project. Our experiences therefore enrich the emerging body of evidence in this field, which may be especially helpful for practitioner teams new to AI engineering.
翻译:今天,許多系統使用人工智慧 (AI) 解決複雜問題。雖然這通常可以提高系統效益,但開發一個投入生產的基於人工智慧的系統是很困難的任務。因此,需要穩固的AI工程實踐來確保所得系統的質量,並改善開發流程。雖然已經有多種開發基於AI的系統所提出的實踐,但詳細實際應用這些實踐的經驗是罕見的。在本文中,我們旨在通過一個案例研究來收集這些經驗,即開發一個使用機器學習功能在股票市場上進行投資的自主股票交易系統。我們從文獻中選擇了10種AI工程實踐,並在開發過程中系統應用它們,目的是收集有關它們的適用性和有效性的證據。我們使用結構化的現場筆記記錄了我們的經驗。此外,我們還使用現場筆記紀錄了在開發過程中出現的挑戰以及我們用來克服它們的解決方案。然後,我們分析了收集到的現場筆記,並評估了每種實踐對於開發的改善程度。最後,我們將證據與現有文獻進行了比較。大多數應用的實踐都對我們的系統進行了改進,儘管程度各不相同,但我們能夠克服所有重大挑戰。定性結果提供了有關10種AI工程實踐以及此類項目的挑戰和解決方案的詳細說明。因此,我們的經驗豐富了這一領域不斷增長的證據庫,這可能對初次接觸AI工程的實踐團隊特別有幫助。