VIP内容
主题： A Survey of Methods for Model Compression in NLP
摘要： 近年来，基于Transformer的语言模型在神经机器翻译，自然语言推理和许多其他自然语言理解任务方面取得了实质性进展。 通过语言建模损失的变体进行自我监督的预训练意味着，在广泛的语料库上训练的模型可以提高在一系列任务上的下游性能。 但是，高参数数量和大计算量意味着BERT和友人的生产部署仍然很困难。 值得庆幸的是，在过去的两年中，已经开发出了多种技术来缓解疼痛并缩短预测时间。 特别是，本文重点介绍在基础模型预训练后应用的以下方法，以减少预测的计算成本：
- 数值精度降低
- 操作融合
- 修剪
- 知识蒸馏
- 模块更换
成为VIP会员查看完整内容
热门内容
最新内容
最新论文
In this paper we presented mmPose-NLP, a novel Natural Language Processing (NLP) inspired Sequence-to-Sequence (Seq2Seq) skeletal key-point estimator using millimeter-wave (mmWave) radar data. To the best of the author's knowledge, this is the first method to precisely estimate upto 25 skeletal key-points using mmWave radar data alone. Skeletal pose estimation is critical in several applications ranging from autonomous vehicles, traffic monitoring, patient monitoring, gait analysis, to defense security forensics, and aid both preventative and actionable decision making. The use of mmWave radars for this task, over traditionally employed optical sensors, provide several advantages, primarily its operational robustness to scene lighting and adverse weather conditions, where optical sensor performance degrade significantly. The mmWave radar point-cloud (PCL) data is first voxelized (analogous to tokenization in NLP) and $N$ frames of the voxelized radar data (analogous to a text paragraph in NLP) is subjected to the proposed mmPose-NLP architecture, where the voxel indices of the 25 skeletal key-points (analogous to keyword extraction in NLP) are predicted. The voxel indices are converted back to real world 3-D coordinates using the voxel dictionary used during the tokenization process. Mean Absolute Error (MAE) metrics were used to measure the accuracy of the proposed system against the ground truth, with the proposed mmPose-NLP offering <3 cm localization errors in the depth, horizontal and vertical axes. The effect of the number of input frames vs performance/accuracy was also studied for N = {1,2,..,10}. A comprehensive methodology, results, discussions and limitations are presented in this paper. All the source codes and results are made available on GitHub for furthering research and development in this critical yet emerging domain of skeletal key-point estimation using mmWave radars.
