Providing timely and meaningful feedback remains a persistent challenge in higher education, especially in large courses where teachers must balance formative depth with scalability. Recent advances in Generative Artificial Intelligence (GenAI) offer new opportunities to support feedback processes while maintaining human oversight. This paper presents an study conducted within the AICoFe (AI-based Collaborative Feedback) system, which integrates teacher, peer, and self-assessments of engineering students' oral presentations. Using a validated rubric, 46 evaluation sets were analyzed to examine agreement, correlation, and bias across evaluators. The analyses revealed consistent overall alignment among sources but also systematic variations in scoring behavior, reflecting distinct evaluative perspectives. These findings informed the proposal of an enhanced GenAI model within AICoFe system, designed to integrate human assessments through weighted input aggregation, bias detection, and context-aware feedback generation. The study contributes empirical evidence and design principles for developing GenAI-based feedback systems that combine data-based efficiency with pedagogical validity and transparency.
翻译:在高等教育中,提供及时且有意义的反馈始终是一项持续的挑战,尤其是在大型课程中,教师必须在形成性评价的深度与可扩展性之间取得平衡。生成式人工智能(GenAI)的最新进展为支持反馈过程同时保持人工监督提供了新的机遇。本文介绍了在AICoFe(基于人工智能的协作反馈)系统中进行的一项研究,该系统整合了教师、同伴及工程专业学生对口头演示的自我评估。通过使用经过验证的评分标准,分析了46组评估数据,以检验不同评估者之间的一致性、相关性及偏差。分析结果显示,各评估来源在整体上具有一致的对齐性,但也存在评分行为的系统性差异,反映了不同的评价视角。这些发现为在AICoFe系统中提出一种增强型GenAI模型提供了依据,该模型旨在通过加权输入聚合、偏差检测和情境感知反馈生成来整合人工评估。本研究为开发基于GenAI的反馈系统提供了实证证据和设计原则,这些系统将基于数据的效率与教学有效性及透明度相结合。