The lottery ticket hypothesis (LTH) has shown that dense models contain highly sparse subnetworks (i.e., winning tickets) that can be trained in isolation to match full accuracy. Despite many exciting efforts being made, there is one "commonsense" rarely challenged: a winning ticket is found by iterative magnitude pruning (IMP) and hence the resultant pruned subnetworks have only unstructured sparsity. That gap limits the appeal of winning tickets in practice, since the highly irregular sparse patterns are challenging to accelerate on hardware. Meanwhile, directly substituting structured pruning for unstructured pruning in IMP damages performance more severely and is usually unable to locate winning tickets. In this paper, we demonstrate the first positive result that a structurally sparse winning ticket can be effectively found in general. The core idea is to append "post-processing techniques" after each round of (unstructured) IMP, to enforce the formation of structural sparsity. Specifically, we first "re-fill" pruned elements back in some channels deemed to be important, and then "re-group" non-zero elements to create flexible group-wise structural patterns. Both our identified channel- and group-wise structural subnetworks win the lottery, with substantial inference speedups readily supported by existing hardware. Extensive experiments, conducted on diverse datasets across multiple network backbones, consistently validate our proposal, showing that the hardware acceleration roadblock of LTH is now removed. Specifically, the structural winning tickets obtain up to {64.93%, 64.84%, 60.23%} running time savings at {36%~80%, 74%, 58%} sparsity on {CIFAR, Tiny-ImageNet, ImageNet}, while maintaining comparable accuracy. Code is at https://github.com/VITA-Group/Structure-LTH.
翻译:彩票假设( LTH) 显示, 密机包含高度稀疏的亚网络( 即 { 36, 赢票 ) 。 尽管做出了许多令人兴奋的努力, 但有一个“ 共通” 很少受到挑战: 胜票是通过迭接规模的裁剪( IMP) 找到的, 由此产生的经剪裁子网络只有无结构的松散。 这一差距限制了在实际中赢得票的吸引力, 因为高度不正常的稀释模式对硬件的加速具有挑战性。 与此同时, 直接取代了IMP非结构化的剪接机, 损坏性更严重, 通常无法找到赢票。 在本文中, 我们展示了第一个积极的结果, 结构性稀释的胜票票可以被有效地找到。 核心理念是, 在每轮( 未结构化) IMP 之后加上“ 后处理技术 ”, 以结构松懈。 具体地, 我们首先“ 补” 正在一些被认为很重要的渠道中, 重新获得元素, 然后是“ 重新组合 ” 非零星, 在结构结构模型中, 中, 快速地展示着 组织 。