The continuous development of Question Answering (QA) datasets has drawn the research community's attention toward multi-domain models. A popular approach is to use multi-dataset models, which are models trained on multiple datasets to learn their regularities and prevent overfitting to a single dataset. However, with the proliferation of QA models in online repositories such as GitHub or Hugging Face, an alternative is becoming viable. Recent works have demonstrated that combining expert agents can yield large performance gains over multi-dataset models. To ease research in multi-agent models, we extend UKP-SQuARE, an online platform for QA research, to support three families of multi-agent systems: i) agent selection, ii) early-fusion of agents, and iii) late-fusion of agents. We conduct experiments to evaluate their inference speed and discuss the performance vs. speed trade-off compared to multi-dataset models. UKP-SQuARE is open-source and publicly available at http://square.ukp-lab.de.
翻译：----
随着问题回答 (QA) 数据集的不断发展，研究社区开始关注多领域模型的发展。一种流行的方法是使用多数据集模型，即在多个数据集上训练模型，以学习它们的规律并防止过度拟合单个数据集。然而，随着 QA 模型在 GitHub 或 Hugging Face 等在线存储库中越来越多，另一种选择正在变得可行。最近的研究表明，合并专家代理可以比多数据集模型获得更大的性能提升。为了方便进行多代理模型研究，我们扩展了 UKP-SQuARE，一个用于 QA 研究的在线平台，以支持三个代理系统家族：i) 代理选择，ii) 代理的早期融合，以及 iii) 代理的晚期融合。我们进行实验来评估它们的推理速度，并讨论与多数据集模型相比的性能与速度之间的权衡。UKP-SQuARE 是一个开源的、公开的在线平台，网址为 http://square.ukp-lab.de。
相关内容
微信扫码咨询专知VIP会员