Deep Transfer Learning based COVID-19 Detection in Cough, Breath and Speech using Bottleneck Features

We present an experimental investigation into the automatic detection of COVID-19 from coughs, breaths and speech as this type of screening is non-contact, does not require specialist medical expertise or laboratory facilities and can easily be deployed on inexpensive consumer hardware. Smartphone recordings of cough, breath and speech from subjects around the globe are used for classification by seven standard machine learning classifiers using leave-$p$-out cross-validation to provide a promising baseline performance. Then, a diverse dataset of 10.29 hours of cough, sneeze, speech and noise audio recordings are used to pre-train a CNN, LSTM and Resnet50 classifier and fine tuned the model to enhance the performance even further. We have also extracted the bottleneck features from these pre-trained models by removing the final-two layers and used them as an input to the LR, SVM, MLP and KNN classifiers to detect COVID-19 signature. The highest AUC of 0.98 was achieved using a transfer learning based Resnet50 architecture on coughs from Coswara dataset. The highest AUC of 0.94 and 0.92 was achieved from an SVM run on the bottleneck features extracted from the breaths from Coswara dataset and speech recordings from ComParE dataset. We conclude that among all vocal audio, coughs carry the strongest COVID-19 signature followed by breath and speech and using transfer learning improves the classifier performance with higher AUC and lower variance across the cross-validation folds. Although these signatures are not perceivable by human ear, machine learning based COVID-19 detection is possible from vocal audio recorded via smartphone.