Assessing prompting frameworks for enhancing literature reviews among university students using ChatGPT

Writing literature reviews is a common component of university curricula, yet it often poses challenges for students. Since generative artificial intelligence (GenAI) tools have been made publicly accessible, students have been employing them for their academic writing tasks. However, there is limited evidence of structured training on how to effectively use these GenAI tools to support students in writing literature reviews. In this study, we explore how university students use one of the most popular GenAI tools, ChatGPT, to write literature reviews and how prompting frameworks can enhance their output. To this aim, prompts and literature reviews written by a group of university students were collected before and after they had been introduced to three prompting frameworks, namely CO-STAR, POSE, and Sandwich. The results indicate that after being exposed to these prompting frameworks, the students demonstrated improved prompting behaviour, resulting in more effective prompts and higher quality literature reviews. However, it was also found that the students did not fully utilise all the elements in the prompting frameworks, and aspects such as originality, critical analysis, and depth in their reviews remain areas for improvement. The study, therefore, raises important questions about the significance of utilising prompting frameworks in their entirety to maximise the quality of outcomes, as well as the extent of prior writing experience students should have before leveraging GenAI in the process of writing literature reviews. These findings are of interest for educators considering the integration of GenAI into academic writing tasks such as literature reviews or evaluating whether to permit students to use these tools.