Deep learning-based emotion recognition using unimodal facial expressions or physiological signals: A review

Emotion recognition has become a key component of intelligent systems, enabling improved human–computer interaction across domains such as healthcare, education, and robotics. Progress has been achieved using facial expressions and physiological signals, particularly Electroencephalography (EEG) and Electrocardiogram (ECG), supported by advances in deep learning.
This paper presents a comprehensive review of unimodal emotion recognition based on facial expressions or physiological signals, with each modality analysed independently to understand signal-specific characteristics and modelling strategies. In addition to commonly studied modalities, this review covers physiological signals including Galvanic Skin Response (GSR), Photoplethysmography (PPG), Electrooculography (EOG), Electromyography (EMG), Respiration Rate (RR), Skin Temperature (SKT), and functional near-infrared spectroscopy (fNIRS).
The review emphasises the design and evaluation of deep learning architectures, including Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Long Short-Term Memory (LSTM), Gated Recurrent Units (GRU), and emerging approaches such as transformer-based models, Vision Transformers (ViTs), and transfer learning techniques. Studies are analysed using a structured evaluation framework considering model design, preprocessing strategies, datasets, evaluation protocols, and performance. Unlike existing surveys, this work provides a critical analysis of prior studies, highlighting strengths, limitations, and trade-offs, with emphasis on generalisation capability and evaluation strategies. The review identifies risks associated with improper data partitioning, where data leakage can lead to overestimated performance. Key challenges include limited dataset sizes, lack of standardised evaluation protocols, and inconsistencies in performance reporting.
This study provides a structured understanding of current research trends and outlines future directions for developing more robust, reliable, and generalisable emotion recognition systems.