Knowledge tracing refers to the problem of estimating each student's knowledge component/skill mastery level from their past responses to questions in educational applications. One direct benefit knowledge tracing methods provide is the ability to predict each student's performance on the future questions. However, one key limitation of most existing knowledge tracing methods is that they treat student responses to questions as binary-valued, i.e., whether the responses are correct or incorrect. Response correctness analysis/prediction is easy to navigate but loses important information, especially for open-ended questions: the exact student responses can potentially provide much more information about their knowledge states than only response correctness. In this paper, we present our first exploration into open-ended knowledge tracing, i.e., the analysis and prediction of students' open-ended responses to questions in the knowledge tracing setup. We first lay out a generic framework for open-ended knowledge tracing before detailing its application to the domain of computer science education with programming questions. We define a series of evaluation metrics in this domain and conduct a series of quantitative and qualitative experiments to test the boundaries of open-ended knowledge tracing methods on a real-world student code dataset.
翻译:知识追踪是指从学生过去对教育应用问题的答复中估计每个学生的知识组成部分/技能掌握水平的问题。知识追踪方法提供的一个直接的好处是能够预测每个学生未来问题的成绩。然而,大多数现有知识追踪方法的一个关键限制是,他们把学生对问题的答复视为二进制评价,即答复是否正确或不正确。反应正确性分析/预测容易操作,但失去了重要信息,特别是对于开放式问题:确切的学生答复可能提供更多关于他们知识状况的信息,而不仅仅是答复正确性。我们在本文件中首次探讨开放性知识追踪,即分析和预测学生对知识追踪设置中的问题的开放式答复。我们首先为开放式知识追踪提供了一个通用框架,然后详细说明其对计算机科学教育领域的应用和编程问题。我们界定了该领域的一系列评价指标,并进行一系列定量和定性实验,以测试现实世界学生代码数据集中开放式知识追踪方法的界限。