In this paper we presented mmPose-NLP, a novel Natural Language Processing (NLP) inspired Sequence-to-Sequence (Seq2Seq) skeletal key-point estimator using millimeter-wave (mmWave) radar data. To the best of the author's knowledge, this is the first method to precisely estimate upto 25 skeletal key-points using mmWave radar data alone. Skeletal pose estimation is critical in several applications ranging from autonomous vehicles, traffic monitoring, patient monitoring, gait analysis, to defense security forensics, and aid both preventative and actionable decision making. The use of mmWave radars for this task, over traditionally employed optical sensors, provide several advantages, primarily its operational robustness to scene lighting and adverse weather conditions, where optical sensor performance degrade significantly. The mmWave radar point-cloud (PCL) data is first voxelized (analogous to tokenization in NLP) and $N$ frames of the voxelized radar data (analogous to a text paragraph in NLP) is subjected to the proposed mmPose-NLP architecture, where the voxel indices of the 25 skeletal key-points (analogous to keyword extraction in NLP) are predicted. The voxel indices are converted back to real world 3-D coordinates using the voxel dictionary used during the tokenization process. Mean Absolute Error (MAE) metrics were used to measure the accuracy of the proposed system against the ground truth, with the proposed mmPose-NLP offering <3 cm localization errors in the depth, horizontal and vertical axes. The effect of the number of input frames vs performance/accuracy was also studied for N = {1,2,..,10}. A comprehensive methodology, results, discussions and limitations are presented in this paper. All the source codes and results are made available on GitHub for furthering research and development in this critical yet emerging domain of skeletal key-point estimation using mmWave radars.
翻译:在本文中,我们仅提供了 mmPose- NLP, 一个新的自然语言处理( NLP) 激励了 Nseq2Seq2Seq) 骨骼键点测量器, 使用毫米波( mmWave) 雷达数据。 作者最了解的是, 这是第一个精确估计25个骨骼键点的方法, 仅使用 mmWave 雷达数据即可。 骨骼构成估计对于从自主车辆、 交通监测、 病人监测、 深度分析, 到防御安全法鉴定, 以及帮助进行预防和可操作的决策。 使用 mmWave雷达的骨骼点测量器, 使用 mmWave 雷达点- cloud( PCl) 数据首先解析( 在 NLP 显示的直径直径分析器中), 使用 NLP 直径解的直径解( 在使用 25- IML 的直径解算法中, 用于 25- IML 的直径计算结果。