Experimentation with software prototypes plays a fundamental role in software engineering research. In contrast to many other scientific disciplines, however, explicit support for this key activity in software engineering is relatively small. While some approaches to improve this situation have been proposed by the software engineering community, experiments are still very difficult and sometimes impossible to replicate. In this paper, we propose the concept of an experimentation workbench as a means of explicit support for experimentation in software engineering research. In particular, we discuss core requirements that an experimentation workbench should satisfy in order to qualify as such and to offer a real benefit for researchers. Beyond their core benefits for experimentation, we stipulate that experimentation workbenches will also have benefits in regard to reproducibility and repeatability of software engineering research. Further, we illustrate this concept with a scenario and a case study, and describe relevant challenges as well as our experience with experimentation workbenches.
翻译:软件原型实验在软件工程研究中起着根本作用。然而,与许多其他科学学科不同,软件工程中的这一关键活动得到的明确支持相对较小。虽然软件工程界提出了改进这种情况的一些办法,但实验仍然非常困难,有时无法复制。在本文件中,我们提出了实验工作台的概念,作为明确支持软件工程研究实验的一种手段。我们特别讨论了实验工作台应满足的核心要求,以便符合这一条件,为研究人员提供真正的利益。我们规定,除了实验的核心好处外,实验工作台也将有利于软件工程研究的可复制性和可重复性。此外,我们用假设和案例研究来说明这一概念,并描述相关的挑战以及我们在实验工作台方面的经验。