By virtue of being prevalently written in natural language (NL), requirements are prone to various defects, e.g., inconsistency and incompleteness. As such, requirements are frequently subject to quality assurance processes. These processes, when carried out entirely manually, are tedious and may further overlook important quality issues due to time and budget pressures. In this paper, we propose QAssist -- a question-answering (QA) approach that provides automated assistance to stakeholders, including requirements engineers, during the analysis of NL requirements. Posing a question and getting an instant answer is beneficial in various quality-assurance scenarios, e.g., incompleteness detection. Answering requirements-related questions automatically is challenging since the scope of the search for answers can go beyond the given requirements specification. To that end, QAssist provides support for mining external domain-knowledge resources. Our work is one of the first initiatives to bring together QA and external domain knowledge for addressing requirements engineering challenges. We evaluate QAssist on a dataset covering three application domains and containing a total of 387 question-answer pairs. We experiment with state-of-the-art QA methods, based primarily on recent large-scale language models. In our empirical study, QAssist localizes the answer to a question to three passages within the requirements specification and within the external domain-knowledge resource with an average recall of 90.1% and 96.5%, respectively. QAssist extracts the actual answer to the posed question with an average accuracy of 84.2%. Keywords: Natural-language Requirements, Question Answering (QA), Language Models, Natural Language Processing (NLP), Natural Language Generation (NLG), BERT, T5.
翻译:由于通常以自然语言(NL)写成,要求容易出现各种缺陷,例如不一致性和不完全性。因此,要求往往受制于质量保证过程。这些过程如果完全手工进行,是乏味的,可能由于时间和预算压力而进一步忽视重要的质量问题。在本文件中,我们提议采用Qassist -- -- 问答(QA)方法,在分析NL要求时向利益攸关方提供自动化援助,包括需要工程师。提出一个问题并获得即时答案,对各种质量保证假设(例如不完全性检测)是有用的。因此,对要求的回答自动具有挑战性,因为寻找答案的范围可能超出给定的要求规格。为此,Qassist为开采外部域知识资源提供支持。我们的工作是首先将QA和外部域知识汇集起来,以应对需要的工程挑战。我们评价了一个涵盖三个应用领域的数据集的QA,对387个问题的答复,例如不完全的检测。回答与要求有关的要求自动具有挑战性,因为对要求的模型范围可以超越给给给给给定的答案的范围。 Qal-alia-alalalalalalalalalalalalal 答案,对三个域的答案。我们用了最近的里程的里数,对90 A 。在内部的里程的里程的里程的里程的里程的答案,对里程的答案,对里程的答案,对里程的答案,对里程的答案,对一个大里程的答案。在三个里程的里程的答案,我们用的实验学问题进行了实验性研究。