LEVERAGING CHATGPT FOR EDUCATIONAL TEXT ANALYSIS AIMING ASSESSMENT GENERATION

. With the rise of new technologies, especially tools using Large Language Models (LLM), many areas of our lives are seeing exciting changes. In education, LLM can dramatically help to improve both learning and testing processes, benefiting everyone involved — from those creating learning materials to the student's educational performance. This article explores how LLM tools, particularly ChatGPT by OpenAI, can help develop better tests and quizzes by analyzing existing educational materials and texts. The goal here is to show how tools that create new t ext, like ChatGPT, can improve the quality of tests by using already available educational materials. In this article, we propose a high-level architectur e of assessment generation based on the ChatGPT.


Introduction.
Education constantly seeks innovative methods to enhance learning and assessment processes.The traditional methods of educational assessment, primarily the generation of tests and quizzes, often entail a substantial investment of time and effort from educators.Moreover, these conventional approaches may need more dynamic adaptability to cater to students' diverse learning curves and educational pathways.With its prowess in processing and analyzing data, LLM seems promising in automating and enriching the assessment generation process.Among the list of LLM tools, OpenAI products, like ChatGPT, showcase the capability to analyze and generate text based on given contexts [6,7,8].
This article aims to introduce a high-level solution for integrating ChatGPT for better assessment generation and outline the main direction the article intends to take.
Related work.Various researchers have delved into leveraging LLM for text analysis in context of education.Among the approaches explored, some researchers have utilized concept maps to develop assessment systems [1,2,3,5,10].Others describe how to decompose texts into logical parts for further analysis [1,2,3,5].
A concept map as a tool to assess the understanding of a certain topic was described in various papers.For instance, in the article [1], the authors introduced the Artificial Intelligence-based Student Learning Evaluation tool (AISLE), which is based on concept maps to evaluate the understanding of different topics.The main idea of the proposed concept is to allow students to describe the educational content through a concept map.Each topic starts from some primary node and then expands to multiple levels.Roughly said, the level of hierarchy and number of concepts students identified explains how deep the understanding of a particular topic is.
There are other researches where authors decompose text in a more granular way, which is an essential concept to understand in the scope of our research.The article [2] proposed to break down educational content into smaller, manageable units termed Concepts, Theses, and Hierarchy of Educational Content [2].This structured approach facilitates a more organized and coherent assessment generation process.
Based on different schemas for assessment generation, it can be created around concepts or theses.These schemas are the backbone for generating alternative answers for the tests, sourced from correct and incorrect answers.This not only aids in crafting dynamic and maintainable tests but also provides a system to inform users about their knowledge gaps in specific areas or content portions.The outlined hierarchical relation of Educational Content, Theses, and Concepts simplifies the maintenance of tests and ensures their alignment with the educational content, thereby fostering a more effective and meaningful assessment process.This approach to defining different building blocks of any text is essential in the process of using them by LLM in a formalized way.
The Based on the available data, the interest in ChatGPT, particularly in education, has risen significantly.Following the recent statistics, the estimated number of visits to ChatGPT is reaching 2 billion points, and an estimated 80% of major Fortune 500 companies are expected to utilize ChatGPT in their business operations permanently [7].This says that AI technologies, particularly LLM, are becoming inevitable in all areas of our lives, including education.
Methodology.As mentioned above, it is crucial to leverage LLM to formalize the process of analyzing information.The methodology for leveraging LLM in dissecting educational text for assessment generation hinges on adjusting sophisticated AI techniques such as Natural Language Processing (NPL) and Machine Learning algorithms.In the current article, we aim to describe a high-level architecture of the possible solution based on using OpenAI's ChatGPT tool to decompose content into fundamental building blocks, like Concepts and Theses, subsequently crafting a concept map and utilizing it in the assessment process.
The whole process consists of four stages (Figure 1).Feedback from expert to system to improve automation process article [6] analyzed the integration of AI tools in education.The article stated that integrating OpenAI's GPT-3 model into educational products led to a 25% improvement in student engagement and learning outcomes.GPT-3 facilitates personalized, efficient, and effective learning experiences for students.The United States, India, and China are the leading adopters of GPT-3 in education, accounting for 60% of the global adoption [6].The article [6] described five key areas of GPT model usage in the learning process: a) customized content generation, b) conversational learning, c) intelligent tutoring, d) streamlined operations, and e) automated assessments [6].