Algorithm-Generated Versus Human-Generated Academic Plans: Determining Optimality from Community College Articulation Agreements

Our study examined one pain point users may have with community college articulation agreements: cross-referencing multiple articulation agreement reports to manually develop an optimal academic plan. Optimal is defined as the minimal set of community college courses that satisfy the transfer requirements for multiple universities a student is preparing to apply to. We recruited 24 California community college transfer students to participate in a research session that consisted of an experiment, survey, and interview. We developed a low-fidelity prototype of a report that contains an algorithmically-generated optimal academic plan. We experimentally compared the prototype to ASSIST, California's official statewide database of articulation agreement reports. Compared to students who used the prototype, students assigned to use ASSIST reports to manually create an optimal academic plan underperformed in optimality mistakes, time required, and usability scores. Moving to our non-experimental results, a sizable minority of students had a negative assessment of counselors' ability and willingness to manually create optimal academic plans using ASSIST. Our last results revolved around students' recommendations for supplemental software features to improve the optimization prototype.