We receive several essential updates on our smartphones in the form of SMS, documents, voice messages, etc. that get buried beneath the clutter of content. We often do not realize the key information without going through the full content. SMS notifications sometimes help by giving an idea of what the message is about, however, they merely offer a preview of the beginning content. One way to solve this is to have a single efficient model that can adapt and summarize data from varied sources. In this paper, we tackle this issue and for the first time, propose a novel Adaptive Beam Search to improve the quality of on-device abstractive summarization that can be applied to SMS, voice messages and can be extended to documents. To the best of our knowledge, this is the first on-device abstractive summarization pipeline to be proposed that can adapt to multiple data sources addressing privacy concerns of users as compared to the majority of existing summarization systems that send data to a server. We reduce the model size by 30.9% using knowledge distillation and show that this model with a 97.6% lesser memory footprint extracts the same or more key information as compared to BERT.
翻译:我们收到的智能手机的一些关键更新信息以短信、文件、语音信息等的形式,被隐藏在内容的杂乱之中。我们常常不理解关键信息,而不通过完整内容。短信通知有时会帮助我们,通过提供信息内容的理念,但是,它们只是提供了初始内容的预览。解决这个问题的一个办法是有一个单一有效的模型,能够对来自不同来源的数据进行修改和总结。在本文件中,我们首次处理这一问题,提出一个新的适应性Beam搜索,以提高可应用于SMS、语音信息并可以扩展至文件的可视抽象汇总信息的质量。据我们所知,这是第一个可调整到多个数据源的可满足用户隐私关切的可与大多数向服务器发送数据的现有合成系统相比的可视抽象汇总管道。我们利用知识蒸馏,将模型的大小减少30.9%,并显示这个模型的记忆足迹比BERP少97.6%,可以提取同样或更多关键的信息。