Multimodal AI Dataset for Emotion and Heart Rate Recognition in Real-World Conditions

Background:

Accurately estimating human physiological signals such as heart rate from video remains challenging in real world environments. Existing datasets and algorithms often rely on posed expressions or controlled lighting, which fail to represent authentic, spontaneous human behavior. Methods using fixed facial regions are highly sensitive to motion and expression changes, resulting in noisy and unreliable heart rate predictions. Moreover, the lack of comprehensive, multimodal datasets capturing spontaneous expressions limits the development and benchmarking of robust models for affective computing and remote physiological monitoring.

Technology Overview:

The MMSE HR dataset comprises 102 synchronized video and physiological recordings from 40 subjects, collected using standard video sensors and a Biopac Mp150 system. It captures spontaneous emotional expressions with ground truth heart rate and blood pressure signals. The invention also introduces a Self Adaptive Matrix Completion (SAMC) algorithm that dynamically identifies reliable facial sub regions and denoises chrominance signals for heart rate estimation. This joint optimization approach mitigates noise from movement and illumination changes, enabling robust, real time, non contact physiological monitoring under naturalistic conditions.

Advantages:

• Authentic multimodal dataset captures spontaneous emotional expressions and physiological signals
• Self Adaptive Matrix Completion algorithm enhances robustness to motion, illumination, and expression variability
• Non contact heart rate estimation enables fully remote physiological monitoring
• High quality synchronized recordings support development of advanced AI models
• Broad demographic diversity improves algorithm generalization and cross population accuracy
• Facilitates benchmarking and validation of emotion recognition and health monitoring systems

Applications:

• Telehealth and remote patient monitoring
• Affective computing and emotion aware AI systems
• Driver and passenger monitoring for automotive safety
• Human computer interaction and adaptive learning environments
• Sports and wellness performance tracking
• Media testing and consumer engagement analytics

Intellectual Property Summary:

• Z. Zhang et al. Multimodal Spontaneous Emotion Corpus for Human Behavior Analysis 2016 IEEE Conference on Computer Vision and Pattern Recognition CVPR Las Vegas NV USA 2016 pp. 3438 to 3446 doi 10.1109/CVPR.2016.374

Stage of Development:

Algorithm and Dataset

Licensing Status:

This technology is available for licensing.

Licensing Potential:

Relevant for telehealth companies, AI developers, automotive safety system providers, and emotion aware computing platforms seeking validated multimodal datasets and robust remote physiological monitoring algorithms.

Additional Information:

Information available upon request.