We introduce meta-learning algorithms that perform zero-shot weight-space adaptation of neural network models to unseen tasks. Our methods repurpose the popular generative image synthesis techniques of natural language guidance and diffusion models to generate neural network weights adapted for tasks. We first train an unconditional generative hypernetwork model to produce neural network weights; then we train a second "guidance" model that, given a natural language task description, traverses the hypernetwork latent space to find high-performance task-adapted weights in a zero-shot manner. We explore two alternative approaches for latent space guidance: "HyperCLIP"-based classifier guidance and a conditional Hypernetwork Latent Diffusion Model ("HyperLDM"), which we show to benefit from the classifier-free guidance technique common in image generation. Finally, we demonstrate that our approaches outperform existing multi-task and meta-learning methods in a series of zero-shot learning experiments on our Meta-VQA dataset.
翻译:我们引入了对神经网络模型进行零光重量-空间调整的元学习算法,对神经网络模型进行隐形任务。我们的方法是重新利用自然语言指导和传播模型的流行基因图像合成技术,以产生适应任务的神经网络加权数。我们首先培训一个无条件的超网络基因模型,以产生神经网络加权数;然后我们培训第二个“指导”模型,根据自然语言任务说明,通过超网络潜伏空间,以零光速方式找到高性能任务适应加权数。我们探索了两种潜在的空间指导替代方法:“HyperCLIP”基于分类法的指导和一个有条件的超网络液态扩散模型(“HyperLDDMD”),我们从图像生成中常见的无分类指导技术中受益。最后,我们展示了我们的方法在Meta-VQA数据集的一系列零点学习实验中超越了现有的多任务和元学习方法。