This paper presents a new test collection for Legal IR, FALQU: Finding Answers to Legal Questions, where questions and answers were obtained from Law Stack Exchange (LawSE), a Q&A website for legal professionals, and others with experience in law. Much in line with Stack overflow, Law Stack Exchange has a variety of questions on different topics such as copyright, intellectual property, and criminal laws, making it an interesting source for dataset construction. Questions are also not limited to one country. Often, users of different nationalities may ask questions about laws in different countries and expertise. Therefore, questions in FALQU represent real-world users' information needs thus helping to avoid lab-generated questions. Answers on the other side are given by experts in the field. FALQU is the first test collection, to the best of our knowledge, to use LawSE, considering more diverse questions than the questions from the standard legal bar and judicial exams. It contains 9880 questions and 34,145 answers to legal questions. Alongside our new test collection, we provide different baseline systems that include traditional information retrieval models such as TF-IDF and BM25, and deep neural network search models. The results obtained from the BM25 model achieved the highest effectiveness.
翻译：本文提出了一个新的法律信息检索测试集：FALQU。其问题和答案来自 Law Stack Exchange（LawSE），这是一个面向法律专业人士以及其他法律经验者的问答网站，与 Stack Overflow 类似。它涵盖了知识产权、版权、刑法等多个主题，而且问题不限于一个国家，用户可以就不同国家的法律提出问题。因此，FALQU 中的问题能够代表真实的用户信息需求，避免了使用实验生成问题。同时，答案由该领域的专家提供。FALQU 是我们所知道的第一个使用 LawSE 构建测试集的测试集，它包含了 9880 个法律问题和 34145 个答案。除了我们的新测试集外，我们还提供了不同的基准系统，其中包括传统信息检索模型，如 TF-IDF 和 BM25，以及深度神经网络文件检索模型。结果表明，BM25 模型的效果最佳。
相关内容
微信扫码咨询专知VIP会员