Natural Language Processing algorithms have made incredible progress, but they still struggle when applied to out-of-distribution examples. We address a challenging and underexplored version of this domain adaptation problem, where an algorithm is trained on several source domains, and then applied to examples from unseen domains that are unknown at training time. Particularly, no examples, labeled or unlabeled, or any other knowledge about the target domain are available to the algorithm at training time. We present PADA: An example-based autoregressive Prompt learning algorithm for on-the-fly Any-Domain Adaptation, based on the T5 language model. Given a test example, PADA first generates a unique prompt for it and then, conditioned on this prompt, labels the example with respect to the NLP prediction task. PADA is trained to generate a prompt which is a token sequence of unrestricted length, consisting of Domain Related Features (DRFs) that characterize each of the source domains. Intuitively, the generated prompt is a unique signature that maps the test example to a semantic space spanned by the source domains. In experiments with 3 tasks (text classification and sequence tagging), for a total of 14 multi-source adaptation scenarios, PADA substantially outperforms strong baselines.
翻译:自然语言处理算法取得了令人难以置信的进展, 但是当应用到分布范围外的例子时它们仍然在挣扎着。 我们处理一个具有挑战性和探索不足的域适应问题版本, 其中算法在几个源域上接受培训, 然后应用到培训时未知的隐蔽域的例子。 特别是, 算法在培训时没有实例、 标签或未贴标签, 或其他关于目标域的知识 。 我们展示 PADAD: 以 T5 语言模型为基础, 用于在飞行中 Any- Domain 适应的基于示例的自动递反快速学习算法 。 根据一个测试示例, PADADA首先为这个域生成一个独特的提示, 然后, 以这个提示为条件, 标出 NLP 预测任务的例子。 PADADA 训练它生成一个不限制长度的象征性序列, 包括每个源域特性的 DOmain 相关特性 。 直观地, 生成了一个独特的信号, 将测试示例绘制成一个由源域所覆盖的语系空间的测试示例。 在实验中, 3项任务( 高级 ADADADADA 级和多级排序中, 多级 的模型) 的模型中, 。