Deep Learning-Based Stress Recognition Using Multimodal Physiological Signals
Keywords:
stress recognition; multimodal physiological signals; deep learning; wearable sensing; affective computing; system robustness; data governance; fairnessAbstract
Stress recognition using multimodal physiological signals has developed rapidly as wearable sensing, data infrastructure, and deep learning methods have matured. Unlike self-report instruments that rely on intermittent subjective assessment, physiological sensing can capture continuous autonomic and somatic changes associated with stress. However, translating laboratory demonstrations into reliable real-world systems requires careful examination of signal acquisition, preprocessing, model architecture, fusion strategy, robustness, fairness, and governance. This paper presents a system-level review of deep learning-based stress recognition from physiological modalities including electrocardiography, electrodermal activity, electromyography, respiration, and photoplethysmography. It considers the structural trade-offs among convolutional recurrent, attention-based, and hybrid architectures for modeling nonstationary physiological time series. The discussion emphasizes multimodal fusion, cross-subject generalization, contextual awareness, and edge deployment rather than isolated accuracy benchmarks. The paper further addresses data governance, algorithmic fairness, regulatory implications, and sustainability concerns that emerge when stress recognition systems are deployed across clinical, occupational, and consumer settings. By integrating technical and socio-technical perspectives, the paper argues that robust stress recognition depends not only on deep model expressiveness but also on data quality, individual variability, infrastructure design, and responsible deployment. The analysis provides a forward-looking research agenda for building dependable multimodal affective systems.
References
1. Picard, R. W., Vyzas, E., & Healey, J. (2001). Toward machine emotional intelligence: Analysis of affective physiological state. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(10), 1175-1191.
2. Healey, J. A., & Picard, R. W. (2005). Detecting stress during real-world driving tasks using physiological sensors. IEEE Transactions on Intelligent Transportation Systems, 6(2), 156-166.
3. Selye, H. (1950). Stress and the general adaptation syndrome. British Medical Journal, 1(4667), 1383-1392.
4. Kreibig, S. D. (2010). Autonomic nervous system activity in emotion: A review. Biological Psychology, 84(3), 394-421.
5. Schmidt, P., Reiss, A., Duerichen, R., Marberger, C., & Van Laerhoven, K. (2018). Introducing WESAD, a multimodal dataset for wearable stress and affect detection. Proceedings of the 2018 International Conference on Multimodal Interaction, 400-408.
6. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 25, 1097-1105.
7. Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735-1780.
8. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30, 5998-6008.
9. Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 1798-1828.
10. Sarker, I. H. (2021). Deep learning: A comprehensive overview on techniques, taxonomy, applications and research directions. SN Computer Science, 2(6), 420.
11. Giannakakis, G., Grigoriadis, D., Giannakaki, K., Simantiraki, O., Roniotis, A., & Tsiknakis, M. (2019). Review on psychological stress detection using biosignals. IEEE Transactions on Affective Computing, 10(2), 202-220.
12. Gjoreski, M., Luštrek, M., Gams, M., & Gjoreski, H. (2017). Monitoring stress with a wrist device using context. Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 159-168.
13. Chen, T., & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 785-794.
14. Zheng, X., Dwyer, V. M., Barrett, L. A., Derakhshani, M., & Hu, S. (2023). Rapid vital sign extraction for real-time opto-physiological monitoring at varying physical activity intensity levels. IEEE Journal of Biomedical and Health Informatics, 27(7), 3107-3118.
15. Tao, D., Tan, H., Wang, H., & Zhou, X. (2019). A survey of multimodal machine learning. IEEE Transactions on Neural Networks and Learning Systems, 30(8), 2455-2477.
16. Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, 1724-1734.
17. Pan, S. J., & Yang, Q. (2010). A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22(10), 1345-1359.
18. Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press.
19. Mehrabian, A. (1996). Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in temperament. Current Psychology, 14(4), 261-292.
20. Solovey, E. T., Zec, M., Garcia Perez, F., Reimer, B., & Mehler, B. (2014). Classifying driver workload using physiological and driving performance data: Two field studies. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 4057-4066.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Journal of Data Intelligence and AI Systems
This work is licensed under a Creative Commons Attribution 4.0 International License.
This article is published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
