Block-based visual programming environments are increasingly used to introduce computing concepts to beginners. Given that programming tasks are open-ended and conceptual, novice students often struggle when learning in these environments. AI-driven programming tutors hold great promise in automatically assisting struggling students, and need several components to realize this potential. We investigate the crucial component of student modeling, in particular, the ability to automatically infer students' misconceptions for predicting (synthesizing) their behavior. We introduce a novel benchmark, StudentSyn, centered around the following challenge: For a given student, synthesize the student's attempt on a new target task after observing the student's attempt on a fixed reference task. This challenge is akin to that of program synthesis; however, instead of synthesizing a {solution} (i.e., program an expert would write), the goal here is to synthesize a {student attempt} (i.e., program that a given student would write). We first show that human experts (TutorSS) can achieve high performance on the benchmark, whereas simple baselines perform poorly. Then, we develop two neuro/symbolic techniques (NeurSS and SymSS) in a quest to close this gap with TutorSS.
翻译:以街区为基础的视觉编程环境正越来越多地被用来向初学者引入计算机概念。 鉴于编程任务是开放式和概念性的, 新学生常常在这些环境中学习时挣扎。 AI驱动的编程导师在自动帮助挣扎的学生方面有很大的希望, 并且需要几个组成部分来实现这一潜力。 我们调查学生建模的关键组成部分, 特别是自动推断学生对预测( 综合) 其行为的错误认识的能力。 我们引入了一个新的基准, 学生合成, 围绕以下挑战: 对于一个特定的学生, 在观察学生尝试固定的参考任务后, 综合学生尝试新的目标任务。 这个挑战与程序合成相似; 然而, 而不是将{ 溶液} (即, 一位专家将撰写的程序) 合成, 这里的目标是将学生的行为预测( 综合) ( 某个学生将撰写的程序 ) 。 我们首先显示, 人类专家( TuorSS) 可以在基准上取得高的成绩, 而在观察学生尝试固定的参考任务之后, 简单的基准运行不力。 然后, 我们开发两个神经/ SS 接近 Symal 技术( SS) 。