This article explores the natural language generation capabilities of large language models with application to the production of two types of learning resources common in programming courses. Using OpenAI Codex as the large language model, we create programming exercises (including sample solutions and test cases) and code explanations, assessing these qualitatively and quantitatively. Our results suggest that the majority of the automatically generated content is both novel and sensible, and in some cases ready to use as is. When creating exercises we find that it is remarkably easy to influence both the programming concepts and the contextual themes they contain, simply by supplying keywords as input to the model. Our analysis suggests that there is significant value in massive generative machine learning models as a tool for instructors, although there remains a need for some oversight to ensure the quality of the generated content before it is delivered to students. We further discuss the implications of OpenAI Codex and similar tools for introductory programming education and highlight future research streams that have the potential to improve the quality of the educational experience for both teachers and students alike.
翻译:文章探索了大型语言模型的自然语言生成能力,并应用于制作在编程课程中常见的两种学习资源。用OpenAI Codex作为大型语言模型,我们创建了编程练习(包括样本解决方案和测试案例)和代码解释,从质量和数量上对这些内容进行评估。我们的结果表明,大多数自动生成的内容都是新颖的和明智的,在某些情况下可以像现在这样使用。当制作练习时,我们发现,仅仅通过提供关键词来影响编程概念和它们所包含的相关主题,就非常容易影响它们所包含的相关主题。我们的分析表明,大规模基因化机器学习模型作为教员的工具很有价值,尽管在向学生提供编程之前,还需要进行某种监督,以确保生成的内容的质量。我们进一步讨论了OpenAI Codex和类似工具对介绍性教学教育的影响,并突出有可能提高师生教育经验质量的未来研究流。