Open bibliographic data and the Italian National Scientific Qualification: measuring coverage of academic fields

The importance of open bibliographic repositories is widely accepted by the scientific community. For evaluation processes, however, there is still some skepticism: even if large repositories of open access articles and free publication indexes exist and are continuously growing, assessment procedures still rely on proprietary databases in many countries. This is mainly due to the richness of the data available in these proprietary databases and the services provided by the companies they are offered by. This paper investigates the current status of open bibliographic data, in particular of three of the most used open resources, namely Microsoft Academic Graph, Crossref and OpenAIRE, evaluating their potentialities as substitutes of proprietary databases for academic evaluation processes. In particular, we focused on the Italian National Scientific Qualification (NSQ), the National process for University Professor qualification in Italy, which uses data from commercial indexes, and investigated similarities and differences between research areas, disciplines and application roles. The main conclusion is that open datasets are ready to be used for some disciplines, even if there is still room for improvement, but there is still a large gap to fill in others and a stronger effort is required from researchers and institutions.