Background: Static Application Security Testing (SAST) tools purport to assist developers in detecting security issues in source code. These tools typically use rule-based approaches to scan source code for security vulnerabilities. However, due to the significant shortcomings of these tools (i.e., high false positive rates), learning-based approaches for Software Vulnerability Prediction (SVP) are becoming a popular approach. Aims: Despite the similar objectives of these two approaches, their comparative value is unexplored. We provide an empirical analysis of SAST tools and SVP models, to identify their relative capabilities for source code security analysis. Method: We evaluate the detection and assessment performance of several common SAST tools and SVP models on a variety of vulnerability datasets. We further assess the viability and potential benefits of combining the two approaches. Results: SAST tools and SVP models provide similar detection capabilities, but SVP models exhibit better overall performance for both detection and assessment. Unification of the two approaches is difficult due to lacking synergies. Conclusions: Our study generates 12 main findings which provide insights into the capabilities and synergy of these two approaches. Through these observations we provide recommendations for use and improvement.
翻译:背景:静态应用安全测试工具旨在协助开发者发现源代码中的安全问题。这些工具通常使用基于规则的方法扫描源代码的安全脆弱性,但是,由于这些工具的重大缺陷(即高假正率),软件脆弱性预测的学习方法正在成为一种受欢迎的方法。目标:尽管这两种方法的目标相似,但其相对价值尚未探讨。我们提供了对源代码安全分析工具和SVP模型的经验性分析,以确定其相对能力。方法:我们评估了几个共同的SAST工具和关于各种脆弱性数据集的SVP模型的探测和评估业绩。我们进一步评估了将这两种方法结合起来的可行性和潜在好处。结果:SAST工具和SVP模型提供了类似的检测能力,但SVP模型在检测和评估方面表现出更好的总体业绩。由于缺乏协同作用,很难统一这两种方法。结论:我们的研究得出了12项主要结论,对这两种方法的能力和协同作用提供了深入了解。我们通过这些观察提出了使用和改进的建议。