The research process includes many decisions, e.g., how to entitle and where to publish the paper. In this paper, we introduce a general framework for investigating the effects of such decisions. The main difficulty in investigating the effects is that we need to know counterfactual results, which are not available in reality. The key insight of our framework is inspired by the existing counterfactual analysis using twins, where the researchers regard twins as counterfactual units. The proposed framework regards a pair of papers that cite each other as twins. Such papers tend to be parallel works, on similar topics, and in similar communities. We investigate twin papers that adopted different decisions, observe the progress of the research impact brought by these studies, and estimate the effect of decisions by the difference in the impacts of these studies. We release our code and data, which we believe are highly beneficial owing to the scarcity of the dataset on counterfactual studies.
翻译:研究过程包括许多决定,例如,如何获得权利和在何处发表论文。在本文中,我们提出一个调查这些决定的影响的一般框架。调查这些影响的主要困难在于我们需要了解反事实结果,而实际上并不存在这些结果。我们框架的关键见解来自现有的对双胞胎的反事实分析,研究人员将双胞胎视为反事实单位。拟议框架将一对相互引证为双胞胎的文件视为一对文件。这类文件往往是平行的、关于类似专题的、在类似社区的工作。我们调查通过不同决定的双胞胎文件,观察这些研究所产生的研究影响的进展,并估计这些研究影响的不同决定的影响。我们发布我们的代码和数据,我们认为,由于反事实研究的数据集稀缺,我们认为这些代码和数据非常有益。