Asking clarification questions is an active area of research; however, resources for training and evaluating search clarification methods are not sufficient. To address this issue, we describe MIMICS-Duo, a new freely available dataset of 306 search queries with multiple clarifications (a total of 1,034 query-clarification pairs). MIMICS-Duo contains fine-grained annotations on clarification questions and their candidate answers and enhances the existing MIMICS datasets by enabling multi-dimensional evaluation of search clarification methods, including online and offline evaluation. We conduct extensive analysis to demonstrate the relationship between offline and online search clarification datasets and outline several research directions enabled by MIMICS-Duo. We believe that this resource will help researchers better understand clarification in search.
翻译：询问澄清问题是一个积极的研究领域;然而,培训和评估搜索澄清方法的资源是不够的;为解决这一问题,我们描述了由306个查询查询(共1 034对查询-澄清对一对)组成的新的免费数据集MIMIMS-Duo,这是一个有多个澄清(共1 034对查询-澄清对一对查询-澄清)的新的可自由获取的306个查询查询数据集;MIMIMS-Du,载有关于澄清问题及其候选答复的细微说明,并通过对搜索澄清方法,包括在线和离线评估进行多维评价,加强现有的MIMISS数据集;我们进行了广泛的分析,以显示离线搜索澄清数据集和在线搜索澄清数据集之间的关系,并概述了MIMISS-Du促成的若干研究方向;我们认为,这一资源将有助于研究人员更好地了解搜索中的澄清。
相关内容
- Today (iOS and OS X): widgets for the Today view of Notification Center
- Share (iOS and OS X): post content to web services or share content with others
- Actions (iOS and OS X): app extensions to view or manipulate inside another app
- Photo Editing (iOS): edit a photo or video in Apple's Photos app with extensions from a third-party apps
- Finder Sync (OS X): remote file storage in the Finder with support for Finder content annotation
- Storage Provider (iOS): an interface between files inside an app and other apps on a user's device
- Custom Keyboard (iOS): system-wide alternative keyboards
Source: iOS 8 Extensions: Apple’s Plan for a Powerful App Ecosystem
微信扫码咨询专知VIP会员