Block-based visual programming environments are increasingly used to introduce computing concepts to beginners. Given that programming tasks are open-ended and conceptual, novice students often struggle when learning in these environments. AI-driven programming tutors hold great promise in automatically assisting struggling students, and need several components to realize this potential. We investigate the crucial component of student modeling, in particular, the ability to automatically infer students' misconceptions for predicting (synthesizing) their behavior. We introduce a novel benchmark, StudentSyn, centered around the following challenge: For a given student, synthesize the student's attempt on a new target task after observing the student's attempt on a fixed reference task. This challenge is akin to that of program synthesis; however, instead of synthesizing a {solution} (i.e., program an expert would write), the goal here is to synthesize a {student attempt} (i.e., program that a given student would write). We first show that human experts (TutorSS) can achieve high performance on the benchmark, whereas simple baselines perform poorly. Then, we develop two neuro/symbolic techniques (NeurSS and SymSS) in a quest to close this gap with TutorSS. We will publicly release the benchmark to facilitate future research in this area.
翻译:以街区为基础的视觉编程环境正越来越多地被用来向初学者引入计算机概念。 鉴于编程任务是开放式的,概念性,新学生在这些环境中学习时往往会挣扎。 AI驱动的编程教师在自动协助挣扎的学生方面有很大的希望, 并且需要几个组成部分来实现这一潜力。 我们调查学生建模的关键组成部分, 特别是自动推断学生对预测( 综合) 其行为的错误认识的能力。 我们引入了一个新的基准, 学生合成, 围绕以下挑战: 对于一个特定的学生, 在观察学生尝试固定的参考任务后, 综合学生对新目标任务的尝试。 这个挑战与程序合成相似; 然而, 而不是合成一个{ 溶液} (例如, 一位专家将写成的方案), 这里的目标是综合学生预测( 综合) 行为( 综合) 。 我们首先显示, 人类专家( TuorSS) 可以在基准上取得高的成绩, 而在观察学生尝试固定的参考任务后, 完成简单的基线。 然后, 我们开发两个神经/ SS 的搜索领域, 我们将接近Symal 。