Development of an intellectualized program system based on rag-architecture
Main Article Content
Abstract
The article presents an approach in the creation of an intellectualized dialogue system for automation of communication with university applicants during admission campaigns. Retrieval-Augmented Generation (RAG) architecture, which combines search of relevant information with generation of a response using a Large Language Model, is the basis of the development. Fine-tuning of the GPT-2 model with regulatory documentation of a higher educational facility was used to provide support for Ukrainian language. A hybrid mechanism of response generation, which combines fragments extracted from a knowledge base with segments generated by the language model, was proposed. An architecture, which ensures the relevance, accuracy and completeness of data, as well as lowering the workload on university employees, was realized. The quality of the system was rated using both objective (response speed, BERTScore, relevance) and subjective metrics (completeness, convenience, search flexibility), which allowed to record the efficiency of the approach. Presented instrument demonstrates the perspective of the usage of RAG approaches in applied tasks of the educational sphere, especially in support systems for admission committees.
Article Details
Issue
Section
This work is licensed under a Creative Commons Attribution 4.0 International License.
References
[1] Three out of four people use AI at work. Microsoft. (2024). https://news.microsoft.com/annual-wti-2024/
[2] J. Swacha, M. Gracel, Retrieval-augmented generation (RAG) chatbots for education: A survey of applications, Applied Sciences. 15 (2025) 4234. https://doi.org/10.3390/app15084234.
[3] T.T. Nguyen, et al., NEU-chatbot: Chatbot for admission of National Economics University, Computers and Education: Artificial Intelligence. 2 (2021) 100036. https://doi.org/10.1016/j.caeai.2021.100036.
[4] N. Chidipothu, et al., Improving large language model (LLM) performance with retrieval-augmented generation (RAG): Development of a transparent generative AI university support system for educational purposes, Journal of Big Data and Artificial Intelligence. 3 (2025) 1. https://doi.org/10.54116/jbdai.v3i1.50.
[5] M.-T. Nguyen, et al., Building a chatbot for supporting the admission of universities, in: 2021 13th International Conference on Knowledge and Systems Engineering (KSE), Bangkok, Thailand, 2021. https://doi.org/10.1109/KSE53942.2021.9648677.
[6] L.S.T. Nguyen, T.T. Quan, URAG: Implementing a unified hybrid RAG for precise answers in university admission chatbots – A case study at HCMUT, Communications in Computer and Information Science. (2025) 82–93. https://doi.org/10.1007/978-981-96-4285-4_7.
[7] malteos, gpt2-uk, Hugging Face. https://huggingface.co/malteos/gpt2-uk.
[8] E. Latif, X. Zhai, Fine-tuning ChatGPT for automatic scoring, Computers and Education: Artificial Intelligence. 6 (2024) 100210. https://doi.org/10.1016/j.caeai.2024.100210.
[9] N. Ding, et al., Enhancing chat language models by scaling high-quality instructional conversations, in: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Singapore, 2023. https://doi.org/10.18653/v1/2023.emnlp-main.183.
[10] C.W. Schmidt, et al., Tokenization is more than compression, in: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Miami, FL, USA, 2024, pp. 678–702. https://doi.org/10.18653/v1/2024.emnlp-main.40.
[11] D. Gyawali, Comparative analysis of CPU and GPU profiling for deep learning models, arXiv. (2023). https://arxiv.org/abs/2309.02521.
[12] Z. Ul Abideen, Autoregressive models for natural language processing, Medium. https://medium.com/@zaiinn440/autoregressive-models-for-natural-language-processing-b95e5f933e1f.
[13] M. Prytula, O. Sinkevych, I. Olenych, Comparison of zero-shot approach and retrieval-augmented generation for analyzing the tone of comments in the Ukrainian language, Electronics and Information Technologies. 28 (2024). https://doi.org/10.30970/eli.28.1.
[14] T. Zhang, V. Kishore, F. Wu, K.Q. Weinberger, Y. Artzi, BERTScore: Evaluating text generation with BERT, arXiv. (2019). https://arxiv.org/abs/1904.09675.
[15] V. Yatsyshyn, et al., Technology of relational database management systems performance evaluation during computer systems design, Scientific Journal of the Ternopil National Technical University. 109 (2023) 59–??. https://doi.org/10.33108/visnyk_tntu2023.01.054.