Employing Continuous Integration inspired workflows for benchmarking of scientific software -- a use case on numerical cut cell quadrature

Scientific software often offers numerous (open or closed-source) alternatives for a given problem. A user needs to make an informed choice by selecting the best option based on specific metrics. However, setting up benchmarks ad-hoc can become overwhelming as the parameter space expands rapidly. Very often, the design of the benchmark is also not fully set at the start of some project. For instance, adding new libraries, adapting metrics, or introducing new benchmark cases during the project can significantly increase complexity and necessitate laborious re-evaluation of previous results. This paper presents a proven approach that utilizes established Continuous Integration tools and practices to achieve high automation of benchmark execution and reporting. Our use case is the numerical integration (quadrature) on arbitrary domains, which are bounded by implicitly or parametrically defined curves or surfaces in 2D or 3D.