In Federated learning (FL), multiple clients collaborate to learn a model through a central server but keep the data decentralized. Personalized federated learning (PFL) further extends FL to handle data heterogeneity between clients by learning personalized models. In both FL and PFL, all clients participate in the training process and their labeled data is used for training. However, in reality, novel clients may wish to join a prediction service after it has been deployed, obtaining predictions for their own unlabeled data. Here, we defined a new learning setup, Inference-Time PFL (IT-PFL), where a model trained on a set of clients, needs to be later evaluated on novel unlabeled clients at inference time. We propose a novel approach to this problem IT-PFL-HN, based on a hypernetwork module and an encoder module. Specifically, we train an encoder network that learns a representation for a client given its unlabeled data. That client representation is fed to a hypernetwork that generates a personalized model for that client. Evaluated on four benchmark datasets, we find that IT-PFL-HN generalizes better than current FL and PFL methods, especially when the novel client has a large domain shift. We also analyzed the generalization error for the novel client, showing how it can be bounded using results from multi-task learning and domain adaptation. Finally, since novel clients do not contribute their data to training, they can potentially have better control over their data privacy; indeed, we showed analytically and experimentally how novel clients can apply differential privacy to their data.
翻译:在联邦学习(FL)中,多个客户通过中央服务器合作学习模型,但数据分散。个性化联邦学习(PFL)进一步扩展FL,通过学习个性化模型处理客户之间的数据差异。在FL和PFL中,所有客户都参与培训过程并使用其标签数据进行培训。然而,在现实中,新客户可能希望加入一个预测服务,通过中央服务器学习一个模型,但保留数据。在这里,我们定义了一个新的学习设置,即“推断-时间PFL(IT-PFL)”,该模式是一组客户培训的模型,在一组客户之间通过学习个人化模型,需要稍后对新的没有标签的客户进行数据差异评估。我们提出了一个新颖的方法,在超网络模块和一个编码模块的基础上解决了这个问题。具体地说,新客户在使用未贴标签的数据后,我们培训一个可以学习客户代表的编码。这个客户代表可以输入一个真正的超级网络,为该客户创建个人化模型。在四个基准数据集上评估了新的没有标签的客户,我们发现他们的内部数据变换的客户,我们发现他们的内部数据方法,最终显示他们的I-LLFFFL的客户如何超越了他们的大数据。我们如何显示他们的通用数据。