The widespread usage of computer-based assessments and individualized learning platforms has resulted in an increased demand for the rapid production of high-quality items. Automated item generation (AIG), the process of using item models to generate new items with the help of computer technology, was proposed to reduce reliance on human subject experts at each step of the process. AIG has been used in test development for some time. Still, the use of machine learning algorithms has introduced the potential to improve the efficiency and effectiveness of the process greatly. The approach presented in this paper utilizes OpenAI's latest transformer-based language model, GPT-3, to generate reading passages. Existing reading passages were used in carefully engineered prompts to ensure the AI-generated text has similar content and structure to a fourth-grade reading passage. For each prompt, we generated multiple passages, the final passage was selected according to the Lexile score agreement with the original passage. In the final round, the selected passage went through a simple revision by a human editor to ensure the text was free of any grammatical and factual errors. All AI-generated passages, along with original passages were evaluated by human judges according to their coherence, appropriateness to fourth graders, and readability.
翻译:随着计算机化评估和个性化学习平台的普及,对快速生产高质量测试的需求越来越大。自动化题目生成(AIG)是使用题目模型和计算机技术来生成新题目的过程,旨在减少在每个步骤上都依赖人类科目专家的情况。 AIG已经在测试开发中使用了一段时间,但是机器学习算法的使用为该过程的效率和效果带来了巨大的潜力。本文提出的方法利用OpenAI最新的基于转换器的语言模型GPT-3来生成阅读材料。通过精心设计的提示,使用现有的阅读材料确保了AI生成的文本有类似于四年级阅读材料的内容和结构。对于每个提示,我们生成多个阅读材料,最终材料是根据Lexile分数与原始材料的一致性而选定的。在最后一轮中,所选材料经过人类编辑的简单修订,以确保文本没有任何语法和事实错误。人类评审员根据材料的连贯性,适宜四年级学生的度和易读性来评估所有AI生成的材料以及原始材料。