本研究选取edX上的两门MOOCs课程,课程名称分别是“Data,Analytics,and Learning”和“Introduction to Engineering and Engineering Mathematics”。其中,第一门课程作为探索性分析,选取参与各项学习活动且坚持学完所有课程内容的学生作为研究对象,即在后面设计的各分析指标中均有计算数值,其目的是剔除多数无效和不完整的数据样本,最终确定311个数据分析条目;第二门课程作为验证性分析,由于是对所有学习行为分析指标进行分析,涉及的数据涵盖课程中的视频学习、互动讨论、学习评价和文本学习等不同学习模块,而学习者在不同学习模块中的活跃程度存在差异,各指标的样本数量有所区别。为了选取有意义数据,使各指标在同一样本数量上进行分析,在计算完成各分析指标数据后,通过id选择所有指标的共同样本,并剔除在所有指标上的缺失值,对数据进行二次处理和指标计算,最后得到分析样本数2383个。在数据使用授权和范围上,已得到课程负责人和所在学校数据审核委员会的批准,准许使用剔除学生个人信息的数据。
学习结果是衡量学习者学习成效的主要方式,而MOOCs环境下学习结果的测量则需要形成性评价的支持。已有的基于网络的评价系统侧重某一方面的内容评价,例如:ASSISTMENT系统支持教师基于网络对学习者数学测试进行过程评价[6];ACED(Adaptive Content with Evidence-based Diagnosis)系统通过创建自适应诊断系统评价学习者的知识和技能[7]。然而,MOOCs环境下的学习活动涉及内容学习、互动交流和练习评测,某一方面的学习行为表现并不能准确反映其真实的学习结果,应当综合其学习行为过程进行评定。MOOCs环境下的形成性评价在评价角色、评价频率、评价内容和反馈等方面相对于传统的形成性评价都有所不同[8]。在评价角色上,应围绕学习者的学习特征展开,侧重分析学习者的个性特征和学习成长[9];在评价频率上应当是对学习活动过程行为的连续性记录,而非间接性评价[10];在评价内容上,要对学习者的各方面内容进行结构性分析和立体化评价,以真实了解其学习状况[11];在学习反馈上,依据学习诊断结果为学习者提供改善学习成效的学习反馈,以促进学习者进一步发展[12]。学习结果的测量应当依据上述评价方面进行分析,并以结果判断和建议反馈方式作为结果输出。
[1] SIEMENS G. Learning analytics:the emergence of a discipline [J].American behavioral scientist,2013,57(10):1380-1400.
[2] FREITAS S I,MORGAN J,GIBSON D.Will MOOCs transform learning and teaching in higher education Engagement and course retention in online learning provision[J]. British journal of educational technology,2015,46(3):455-471.
[3] BRANSFORD J D,BROWN A L,COCKING R R.How people learn:brain,mind,experience,and school[M].Washington,D.C.:National Academy Press,1999.
[4] 基思·索耶主编.剑桥学习科学手册[M]. 徐晓东,译.北京:教育科学出版社,2012.
[5] RANI M,NAYAK R,VYAS O P.An ontology-based adaptive personalized e-learning system,assisted by software agents on cloud storage[J]. Knowledge-based systems,2015,90(12):33-48.
[6] KOEDINGER K R,MCLAUGHLIN E A,HEFFERNAN N T.A quasi-experimental evaluation of an on-line formative assessment and tutoring system[J]. Journal of educational computing research,2010,43(4):489-510.
[7] SHUTE V J,GRAF E A,HANSEN E G.Designing adaptive,diagnostic math assessments for individuals with and without visual disabilities[J]. ETS research report series,2006,5(1):25-37.
[8] SHUTE V J.Tensions,trends,tools,and technologies:time for an educational sea change[J].ETS research report series,2006(1):32-49.
[9] BELCADHI L C.Personalized feedback for self-assessment in lifelong learning environments based on semantic web[J]. Computers in human behavior,2016,55(2):562-570.
[10] DUNLOSKY J,RAESON K A.Do students use testing and feedback while learning?A focus on key concept definitions and learning to criterion[J]. Learning and instruction,2015,39(10):32-44.
[11] RITZHAUPT A D,KEALY W A.On the utility of pictorial feedback in computer-based learning environments[J]. Computers in human behavior,2015,48(7):525-534.
[12] VALDEZ A.Computer-based feedback and goal intervention:learning effects[J]. Educational technology research and development, 2012,60(5):769-784.
[13] CHAI K,POTDAR V,CHANG E. User contribution measurement model for web-based discussion forums[C]//The 3rd IEEE International Conference on Digital Ecosystems and Technologies.New York:Institute of Electrical and Electronics Engineers,2009:347-352.
[14] LOWES S,LIN P,KINGHORN B.Exploring the link between online behaviours and course performance in asynchronous online high school courses[J]. Journal of learning analytics,2015,2(2):169-194.
[15] YOU J W.Identifying significant indicators using LMS data to predict course achievement in online learning[J]. The internet and higher education,2016,29(4):23-30.
Design and Empirical Research of Learning Outcome Prediction Based on CIEO Analysis from the Perspective of Learning Computing
MOU Zhijia
(School of Humanities, Jiangnan University, Wuxi Jiangsu 214122)
[Abstract] Designing systematic learning outcome prediction theory, which can be guided, understood and operated, is a prescription to improve learning effectiveness. From the perspective of learning activity theory, this paper expounds the learning process and learning outcomes in MOOCs environment, and proposes the analysis of CIEO learning outcome prediction and the working model of learning outcome prediction. The model includes theoretical level, participation level and behavioral level. Among them, the theoretical level covers personalized learning theory, project response theory and social cognitive theory, which guide the learning content, learning interaction and learning evaluation in the participation level respectively. The participation level is designed from the main manifestation of learning activities. The behavioral layer, following the "goal-process-result", is designed based on the specific performance of learning behaviors. Then, the indicators of learning behavior analysis are designed, and six kinds of indicators oriented to learning results are formed. That is, completion and mastery degree based on learning content, participation and contribution degree based on learning interaction, completion degree and pass rate based on learning evaluation behavior. Finally, multiple regression analysis is used to explore the correlation between learning behavior indicators and learning results. Attribute selection method, prediction classification and text analysis are used to verify the importance and accuracy of learning behavior analysis indicators, and the calculation equation of learning result prediction is obtained.