Automated Test-Case Generation for Solidity Smart Contracts: the AGSolT Approach and its Evaluation

Blockchain and smart contract technology represent novel approaches to trusted services computing, opening the way to services designed specifically for trusted computing. Nevertheless, testing smart contracts is still in its infancy, with plenty of challenges not yet fully explored. We argue that existing tools are primarily for vulnerabilities detection and do not produce test suites suited for human oracles. In this paper, we present AGSOLT, a tool for Automated Generation of Solidity Test Suites. We evaluate the tool's efficiency by implementing two search algorithms to automatically generate test suites for stand-alone Solidity smart contracts, considering some of the blockchain-specific challenges. Subsequently, to test AGSOLT in a realistic service operations scenario, we compared a random search algorithm and a genetic algorithm on a set of 36 real-world service applications featuring smart contracts. We found that AGSOLT is capable of achieving high branch coverage with both approaches and even discovered that some of the most popular Solidity smart contracts on GitHub have design flaws that might, for example, make code functions easily un-executable as a result of requiring too much gas. We conclude that AGSOLT provides a very valuable addition to service operations' pipelines supporting trusted computing applications based on smart contracts.