Large Language Model-Powered Smart Contract Vulnerability Detection: New Perspectives

This paper provides a systematic analysis of the opportunities, challenges, and potential solutions of harnessing LLMs to dig out vulnerabilities within smart contracts based on our ongoing research. For the smart contract vulnerability detection task, the key to achieving practical usability lies in detecting as many true vulnerabilities as possible while minimizing the number of false positives. However, our empirical study using LLM as a detection tool reveals interesting yet contradictory findings: generating more answers with higher randomness largely increases the likelihood of a correct answer being generated while inevitably leading to a higher number of false positives, resulting in exhaustive manual verification efforts. To mitigate this tension, we propose an adversarial framework dubbed GPTLens that breaks the traditional one-stage detection into two synergistic stages $-$ generation and discrimination, for progressive detection and fine-tuning, wherein the LLM plays dual roles, i.e., auditor and critic, respectively. The goal of auditor is to identify multiple diverse vulnerabilities with intermediate reasoning, while the goal of critic is to evaluate the accuracy of identified vulnerabilities and to examine the integrity of the detection reasoning. Experimental results and illustrative examples demonstrate that auditor and critic work together harmoniously to yield significant improvements over the traditional one-stage detection. GPTLens is intuitive, strategic, and entirely LLM-driven without relying on specialist expertise in smart contracts, showcasing its methodical generality and potential to detect a broad spectrum of vulnerabilities. Our code is available at: https://github.com/git-disl/GPTLens.