Page 41 - 2024

Ph.D.
(Computer Applications)
FRAMEWORK TO EVALUATE AND IMPROVE EDUCATIONAL
PROCESSES IN INDIAN SCHOOLS AND UNIVERSITIES

Ph.D. Scholar : Raval Helly Yogeshkumar
Research Supervisor : Dr. Satyen M. Parikh

Regi. No.: 18276211004
Abstract :
The evolution of question answering (QA) systems has dramatically reshaped human-
computer interaction, with AI-driven conversational agents becoming essential across
various domains. From Joseph Weizenbaum's ELIZA in the 1960s, which used early
natural language processing (NLP) techniques to simulate basic conversation, to the
emergence of advanced models like GPT-2, Mistral and LLaMA, the capabilities of these
systems have evolved significantly. Today, conversational agents are not only capable of
understanding and responding to complex queries but also exhibit human-like
conversational depth.

This research explores the journey of QA systems, from rule-based methodologies to
state-of-the-art deep learning techniques. Despite this progress, existing systems still
face critical challenges, particularly in managing ambiguous questions, retaining
conversational context, and commonsense reasoning tasks. To tackle these issues, this
research introduces LLMHarmonized, an innovative self-learning agent that integrates the
capabilities of two leading models Mistral and LLAMA2 within a hybrid architecture called
the Self-Learning Intelligist Agent. This agent continually improves its performance
through interaction and feedback, enhancing its accuracy, contextual understanding, and
ability to resolve ambiguity.

The framework's effectiveness is demonstrated through extensive experimentation on
benchmark datasets. On the CoQA dataset, LLMHarmonized achieves an outstanding F1
score of 86.7, illustrating its ability to maintain context and deliver precise answers in
conversational settings. On the NQ-OPEN dataset, which tests ambiguity resolution in
open-domain questions, the system attains an F1 score of 34.9 and Exact Match 45.8,
showcasing its ability to handle challenging queries. Moreover, the agent demonstrates
strong commonsense reasoning, achieving an accuracy of 61.7% on the
CommonsenseQA dataset, highlighting its capacity for understanding and applying
everyday logic.

By leveraging a self-learning approach and integrating advanced language models,
LLMHarmonized significantly advances the capabilities of modern QA systems. This
research not only addresses persistent challenges in context comprehension and

01