Metrics and Mechanisms: Measuring the Unmeasurable in the Science of Science

What is scientific knowledge, and how is it created, accumulated, transformed, and used? If we want to know the answers to these questions, we need to be able to uncover the structures and mechanisms of science, in addition to the metrics that are easily collectable and quantifiable. In this review article, we link metrics to mechanisms, by demonstrating how emerging metrics not only offer complementaries to the existing metrics, but also shed light on the underlying mechanisms related to ten key quantities of interest in the Science of Science, including discovery significance, finding replicability, knowledge cumulativeness, and beyond. We classify existing theories and findings into three fundamental properties of science: hot and cold science, soft and hard science, fast and slow science. We suggest that curiosity about structure and mechanisms of science since Derek J. de Solla Price, Eugene Garfield, Robert K. Merton, and many others complement the zeitgeist in pursuing new, complex metrics without understanding the underlying processes.