ECG is an attractive option to assess stress in serious Virtual Reality (VR) applications due to its non-invasive nature. However, the existing Machine Learning (ML) models perform poorly. Moreover, existing studies only perform a binary stress assessment, while to develop a more engaging biofeedback-based application, multi-level assessment is necessary. Existing studies annotate and classify a single experience (e.g. watching a VR video) to a single stress level, which again prevents design of dynamic experiences where real-time in-game stress assessment can be utilized. In this paper, we report our findings on a new study on VR stress assessment, where three stress levels are assessed. ECG data was collected from 9 users experiencing a VR roller coaster. The VR experience was then manually labeled in 10-seconds segments to three stress levels by three raters. We then propose a novel multimodal deep fusion model utilizing spectrogram and 1D ECG that can provide a stress prediction from just a 1-second window. Experimental results demonstrate that the proposed model outperforms the classical HRV-based ML models (9% increase in accuracy) and baseline deep learning models (2.5% increase in accuracy). We also report results on the benchmark WESAD dataset to show the supremacy of the model.
翻译:ECG是评估严重虚拟现实(VR)应用中压力的一种有吸引力的选择,因为它具有非侵入性质。然而,现有的机器学习(ML)模型运行不良。此外,现有的研究只进行二进制压力评估,而开发一种更具参与性的生物反馈应用,需要多层次评估。现有的研究将单一经验(例如观看VR视频)评分和分类为单一压力水平,这再次妨碍设计动态经验,以便利用实时的游戏压力评估。在本文中,我们报告了关于VR压力评估的新研究的结果,其中评估了三种压力水平。ECG数据是从9个拥有VR滚动海岸的用户那里收集的。然后,VR经验被3个分机算手工标记为10秒到3个压力水平。然后,我们提出利用光谱和1D ECG的新型多式联运深度聚变模型,能够仅仅从一秒窗口提供压力预测。实验结果显示,拟议的模型比基于经典HRV的ML模型的模型(即评估了三个压力水平)。ECG的数据数据模型(我们也将精确度提升到WES基准模型的精确度)。