Mental Health Detection System Using Multimodal AI Techniques

Abstract

Mental health problems such as depression, anxiety, and stress are increasing worldwide and often remain unnoticed because people hesitate to seek help, fear judgment, or lack proper support. Many individuals and healthcare providers want accurate assessments, but they often struggle to get real-time, objective information. To solve this, we have proposed a new system using Multimodal AI technology. This system uses text, audio, and video inputs to detect emotional states in real time. Text input is analyzed using NLP and sentiment analysis, audio input is processed through speech-to-text and tone-based feature extraction using Librosa, and video input is analysed using DeepFace and OpenCV for facial emotion recognition. The system combines these results to classify the user’s mental state as mild, neutral, or severe, and provides instant personalized suggestions through a secure web interface. AI is a powerful tool for mental health because it supports early detection, real-time monitoring, and personalized feedback, helping users manage their well-being more effectively.