DISC: A Plug-and-Play Compression Method for CNN Partition-Offloading

Partitioning CNN model computation between edge devices and servers has been proposed to alleviate edge devices' computing capability and network transmission limitations. However, due to the large data size of the intermediate output in CNN models, the transmission latency is still the bottleneck for such partition-offloading. Though compression methods on images like JPEG-based compression can be applied to the intermediate output data in CNN models, their compression rates are limited, and the compression leads to high accuracy loss. Other compression methods for partition-offloading adopt deep learning technology and require hours of additional training. In this paper, we propose a novel compression method DISC for intermediate output data in CNN models. DISC can be applied to partition-offloading systems in a plug-and-play way without any additional training. It shows higher performance on intermediate output data compression than the other compression methods designed for image compression. Further, AGLOP is developed to optimize the partition-offloading system by adjusting the partition point and the hyper-parameters of DISC. Based on our evaluation, DISC can achieve over 98% data size reduction with less than $1\%$ accuracy loss, and AGLOP can achieve over 91.2% end-to-end execution latency reduction compared with the original partition-offloading.