Numerous national research assessment policies set the goal of promoting "excellence" and incentivise scholars to publish their research in the most prestigious journals or with the most prestigious book publishers. We investigate the practicalities of the assessment of book outputs based on the prestige of book publishers (Denmark, Finland, Flanders, Lithuania, Norway). Additionally, we test whether such judgments are transparent and yield consistent results. We show inconsistencies in the levelling of publishers, such as the same publisher being ranked as prestigious and not-so-prestigious in different states or in consequent years within the same country. Likewise, we find that verification of compliance with the mandatory prerequisites is not always possible because of the lack of transparency. Our findings support doubts about whether the assessment of books based on a judgement about their publisher yields acceptable outcomes. Currently used rankings of publishers focus on evaluating the gatekeeping role of publishers but do not assess other essential stages in scholarly book publishing. Our suggestion for future research is to develop approaches to evaluate books by accounting for the value added to every book at every publishing stage, which is vital for the quality of book outputs from research assessment and scholarly communication perspectives.
翻译:许多国家的研究评估政策都设定了促进“优秀”和激励学者在最有声望的杂志上或与最有声望的图书出版商发表其研究成果的目标。我们根据图书出版商的声望(丹麦、芬兰、佛兰德斯、立陶宛、挪威)调查图书产出评估的实用性。此外,我们测试这类判断是否透明并产生一致的结果。我们发现出版商的分级存在不一致之处,例如同一出版商在不同国家或在同一国家随后几年中被评为有声望和不乐观的出版商。同样,我们发现,由于缺乏透明度,并不总是能够核实对强制性先决条件的遵守情况。我们的调查结果证明,根据对其出版商的判断对图书进行的评估是否产生可接受的结果。目前,出版商的排名侧重于评价出版商的门面作用,但并不评估学术书籍出版的其他必要阶段。我们关于未来研究的建议是制定方法,通过计算每一出版阶段所增加的价值来评价书籍,这对于从研究评估和学术交流角度分析的图书产出的质量至关重要。