Socratic questioning is an educational method that allows students to discover answers to complex problems by asking them a series of thoughtful questions. Generation of didactically sound questions is challenging, requiring understanding of the reasoning process involved in the problem. We hypothesize that such questioning strategy can not only enhance the human performance, but also assist the math word problem (MWP) solvers. In this work, we explore the ability of large language models (LMs) in generating sequential questions for guiding math word problem-solving. We propose various guided question generation schemes based on input conditioning and reinforcement learning. On both automatic and human quality evaluations, we find that LMs constrained with desirable question properties generate superior questions and improve the overall performance of a math word problem solver. We conduct a preliminary user study to examine the potential value of such question generation models in the education domain. Results suggest that the difficulty level of problems plays an important role in determining whether questioning improves or hinders human performance. We discuss the future of using such questioning strategies in education.
翻译:专家质询是一种教育方法,使学生能够通过问他们一系列深思熟虑的问题来找到复杂问题的答案。 产生具有实际意义的问题具有挑战性,需要理解问题所涉及的推理过程。 我们假设这样的质询策略不仅能够提高人类的性能,而且能够帮助数学词词问题解决者。 在这项工作中,我们探索了大型语言模型(LMS)在提出指导数学词问题解决的顺序问题方面的能力。我们根据投入调节和强化学习提出了各种有指导的问题生成计划。在自动和人的质量评估中,我们发现LMS受问题属性的限制,产生了优异的问题,提高了数学词问题解答者的总体性能。我们进行了初步用户研究,以研究这类问题生成模式在教育领域的潜在价值。结果表明,问题的困难程度在确定质询是否改进或阻碍人类的性能方面起着重要作用。我们讨论了在教育中使用这类质询策略的未来。