We asked ChatGPT to participate in an undergraduate computer science exam on ''Algorithms and Data Structures''. We evaluated the program on the entire exam as posed to the students. We hand-copied its answers onto an exam sheet, which was subsequently graded in a blind setup alongside those of 200 participating students. We find that ChatGPT narrowly passed the exam, obtaining 20.5 out of 40 points. This impressive performance indicates that ChatGPT can indeed succeed in challenging tasks like university exams. At the same time, the tasks in our exam are structurally similar to those on other exams, solved homework problems, and teaching materials that can be found online. Therefore, it would be premature to conclude from this experiment that ChatGPT has any understanding of computer science. The transcript of our conversation with ChatGPT is available at \url{https://github.com/tml-tuebingen/chatgpt-algorithm-exam}, and the entire graded exam is in the appendix of this paper.
翻译：我们请ChatGPT参加了一场针对"算法和数据结构"的本科生计算机科学考试。我们对整场考试进行了评估，将ChatGPT的答案手工抄写到答题卡上，并与参加考试的200名学生的答案一同进行了盲审。我们发现ChatGPT勉强通过了考试，获得了40分中的20.5分。这一优异表现表明，ChatGPT确实可以在像大学考试这样的挑战性任务中取得成功。与此同时，我们考试中的任务在结构上与其他考试、课程作业问题以及网上可以找到的教材非常相似。因此，从这个实验中得出ChatGPT具有计算机科学理解能力的结论是过早的。我们的ChatGPT对话记录可以在\url{https://github.com/tml-tuebingen/chatgpt-algorithm-exam}中找到，整个评分过的考试可以在本文附录中找到。
相关内容
微信扫码咨询专知VIP会员