A Conversation with ChatGPT about the Impact of the COVID-19 Pandemic on Education: Comparative Review Based on Human–AI Collaboration

Abstract

Background/purpose – ChatGPT, an AI-powered chatbot designed with generative pre-trained transformer architecture, have intrigued millions of people from diverse backgrounds since its first release, and generated excitement with its groundbreaking performance in numerous use cases they have been tested. Its ability to generate coherent and contextually relevant responses across a wide range of topics have also attracted the attention of researchers, and several scholars have investigated the potential uses of this AI-based tool to promote scientific work. The current study was designed with a similar purpose, and aimed to conduct scientific research on the effects of COVID-19 pandemic on education with a collaborative analysis of human-artificial intelligence. Materials/methods – The data were generated from simultaneous interviews with ChatGPT-3.5 and 4, and their responses were evaluated comparatively in terms of the accuracy, clarity, conciseness, and breadth of information provided for (1) the definition of the COVID-19 pandemic, (2) the negative impact of the COVID-19 pandemic on education, (3) the positive impact of the COVID-19 pandemic on education, (4) recommendations to improve the effectiveness of teaching and learning during the COVID-19 pandemic. We evaluated the responses using a trichotomous rating system, and also calculated Cohen’s kappa values to assess inter-rater agreements for each category of evaluation. Results – The results indicated that both versions demonstrated a strong potential to generate accurate, clear and concise information with a satisfactory breadth. As can be expected, ChatGPT-4 provided more categorized and synthesized information, and was better at demonstrating critical reasoning. Conclusion – These results indicate that ChatGPT is a promising tool that can support scientific research process in collaboration with human intelligence, and its newer versions could be developed in a way that these chatbots could be ethically and safely integrated into different stages of scientific work. Copyright © 2023 by the author(s). This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (CC-BY-4.0), where it is permissible to download and share the work provided it is properly cited.