Foundation models can be disruptive for future AI development by scaling up deep learning in terms of model size and training data's breadth and size. These models achieve state-of-the-art performance (often through further adaptation) on a variety of tasks in domains such as natural language processing and computer vision. Foundational models exhibit a novel {emergent behavior}: {In-context learning} enables users to provide a query and a few examples from which a model derives an answer without being trained on such queries. Additionally, {homogenization} of models might replace a myriad of task-specific models with fewer very large models controlled by few corporations leading to a shift in power and control over AI. This paper provides a short introduction to foundation models. It contributes by crafting a crisp distinction between foundation models and prior deep learning models, providing a history of machine learning leading to foundation models, elaborating more on socio-technical aspects, i.e., organizational issues and end-user interaction, and a discussion of future research.
翻译:基础模型通过在模型规模和培训数据的广度和广度和广度方面扩大深层次的学习,可能干扰未来AI的开发。这些模型在自然语言处理和计算机愿景等领域实现各种任务的最新业绩(通常是通过进一步调整)。基础模型展示了一种新颖的{行为}:{文体学习}使用户能够提供一个查询和几个例子,从中得出一个模型的答案,而不必接受关于这类询问的培训。此外,模型的[男女生化}可能取代许多任务特定模型,由少数公司控制的非常小的模型,导致对AI的权力和控制的转变。本文为基础模型提供了简短的介绍。它有助于在基础模型和先前的深层学习模型之间作出明确的区分,提供了机器学习的历史,从而形成基础模型,就社会技术方面,即组织问题和最终用户互动问题和对未来研究的讨论作了更多的阐述。