Hierarchical text classification consists in classifying text documents into a hierarchy of classes and sub-classes. Although artificial neural networks have proved useful to perform this task, unfortunately they can leak training data information to adversaries due to training data memorization. Using differential privacy during model training can mitigate leakage attacks against trained models, enabling the models to be shared safely at the cost of reduced model accuracy. This work investigates the privacy-utility trade-off in hierarchical text classification with differential privacy guarantees, and identifies neural network architectures that offer superior trade-offs. To this end, we use a white-box membership inference attack to empirically assess the information leakage of three widely used neural network architectures. We show that large differential privacy parameters already suffice to completely mitigate membership inference attacks, thus resulting only in a moderate decrease in model utility. More specifically, for large datasets with long texts we observed Transformer-based models to achieve an overall favorable privacy-utility trade-off, while for smaller datasets with shorter texts convolutional neural networks are preferable.
翻译:等级化文本分类包括将文本文件分为等级和亚类。虽然人工神经网络已证明对完成这项任务有用,但不幸的是,由于培训数据记忆化,它们可以向对手泄露培训数据信息。在模型培训期间使用不同的隐私可以减轻对受过培训的模型的泄漏攻击,使模型能够安全共享,而降低模型准确性。这项工作调查了在有不同隐私保障的等级化文本分类中隐私-效用交易,并确定了提供优厚权衡的神经网络结构。为此,我们使用白箱成员攻击来对三大广泛使用的神经网络结构的信息泄漏进行经验性评估。我们表明,巨大的差异隐私参数已经足以完全减轻成员对神经网络结构的泄漏,从而只能导致模型效用的适度下降。更具体地说,对于我们所观测的基于变换模型的长文本的大型数据集,以便实现总体有利的隐私-效用交易,而对于使用较短的短文本神经网络的较小数据集则更为可取。