Mixed-precision quantization (MPQ) suffers from time-consuming policy search process (i.e., the bit-width assignment for each layer) on large-scale datasets (e.g., ISLVRC-2012), which heavily limits its practicability in real-world deployment scenarios. In this paper, we propose to search the effective MPQ policy by using a small proxy dataset for the model trained on a large-scale one. It breaks the routine that requires a consistent dataset at model training and MPQ policy search time, which can improve the MPQ searching efficiency significantly. However, the discrepant data distributions bring difficulties in searching for such a transferable MPQ policy. Motivated by the observation that quantization narrows the class margin and blurs the decision boundary, we search the policy that guarantees a general and dataset-independent property: discriminability of feature representations. Namely, we seek the policy that can robustly keep the intra-class compactness and inter-class separation. Our method offers several advantages, i.e., high proxy data utilization, no extra hyper-parameter tuning for approximating the relationship between full-precision and quantized model and high searching efficiency. We search high-quality MPQ policies with the proxy dataset that has only 4% of the data scale compared to the large-scale target dataset, achieving the same accuracy as searching directly on the latter, and improving the MPQ searching efficiency by up to 300 times.
翻译:混合精密量度(MPQ)在大型数据集(如ISLVRC-2012)上,大量限制其在真实世界部署情景中的实用性,因此,大规模数据集(如ISLVRC-2012)的政策搜索过程耗时(即每层的位宽任务),严重限制了其在真实世界部署情景中的实用性。在本文件中,我们提议为大型模型培训的模型使用一个小型代理数据集,以搜索有效的 MPQ 政策。这打破了在模型培训和MPQ政策搜索时需要一致数据集的常规,这可以大大提高MPQ的搜索效率。然而,分散的数据分布在寻找这种可转移的MPQ政策时带来了困难。受这种观察的驱使,量化缩小了等级差幅和模糊了决定界限。我们搜索了一种保证一个通用和数据集独立的属性:特征显示的不稳定性。也就是说,我们寻求一种政策能够稳健地保持类内缩缩缩和级间分离。我们的方法提供了一些优势,例如,高代理数据数据的利用,没有超超超级的精确度检索时间,我们只能通过全面搜索和高等级数据比值数据比值比例数据,从而实现数据比重数据比重数据比重数据比重数据比重数据比重,从而全面检索数据比重数据比重。