DEVELOPMENT OF A DEEP LEARNING BASED HUMAN COMPUTER INTERACTIVE SYSTEM

Subject Area: Computer Science

Quality education is a fundamental right, but individuals with impairments, such as vision loss, often face significant challenges in accessing equitable learning environments. This study focuses on designing an intuitive system that leverages deep learning technologies to enhance human-computer interaction for visually impaired users, with a primary focus on improving accessibility in educational settings. By utilizing You Only Look Once-Version 5 (YOLOv5), a robust object detection algorithm that can accurately identify individuals in real-time, facilitating fast login processes based on facial biometrics was presented. In addition to the authentication system, an improved human-computer interaction platform that adapts to the specific needs of visually impaired users was modelled. This system integrates voice commands and speech-to-text capabilities, empowering student to navigate and interact independently with computer interface while sitting for examination. To further enhance this system, an adaptive environmental noise cancellation algorithm was proposed, utilizing the Least Mean Squares (LMS) filter to reduce background noise. The system's performance was evaluated through real-world tests scenario, validating its effectiveness in providing a user-friendly, accessible solution for visually impaired individuals in educational settings. The results of the trained YOLOV-5 model reported precision score of 0.91, recall of 0.90 and F1-score of 0.98. Comparative analysis with other state of the art algorithms reported our model as the most reliable due to the integration of the adaptive filter.