学术论文是发布自己或团队最新研究进展正式且最快捷的途径,也是和同行交流想法最方便、高效的方式。当同行评议(Peer review)作为学术成果正式发布的必经之路已运行200余年[1]时,用正确的姿势进行review rebuttal便成为提高论文录用机率甚至扭转乾坤的最后一搏。本文从“What is peer review?”、“How to rebuttal?”及“Does rebuttal matter?”三部分来聊聊学术论文(主要针对人工智能领域会议和期刊)rebuttal的那些事。
与会议论文相似,期刊论文也是由其他研究者进行同行评议之后决定是否予以录用。不过期刊审稿由于没有deadline(期刊的special issue除外),通常不会在较短时间集中接收到的非常多稿件。一般而言,接收到的稿件会由期刊主编(Editor-in-Chief)分配给副主编(Associate Editor),之后副主编会聘请一些相关的研究者作为审稿人进行评审,有时副主编本身也会负责论文的评审意见和推荐等工作。而最终决定论文是否录用的仍然是期刊主编。 与会议论文不同的是,正是由于没有明确的deadline,期刊审稿意见的“出炉”时间往往极不确定且普遍周期较长(3-8个月不等),第一轮评审之后审稿人一般会给出如下四种推荐意见之一:1)接收(Accept);2)接收但需要微幅修改(Accept with a minor revision);3)接收但需要大幅修改(Accept with a major revision,有些期刊如TNNLS则称作Reject & Resubmit);4)拒稿(Reject)。之后,副主编会根据若干审稿人的结果综合评定给出meta review,若meta review意见为2)和3)中其一,作者便需要准备一个详尽的revision list来对每位审稿人的意见逐条进行response和rebuttal,同时还需要提供修改过的文章供审稿人进行下一轮评审。同样,和会议论文不同的是,会议论文这一的rebuttal过程最多只有一次。而期刊论文如此过程可以反复多次,更像一场你一言我一语的“交谈”,最终直到全部审稿人均认为达到“Accept”的程度。但需注意,有的期刊major revision只有一轮,若经过一次major revision后的文章仍未达到录用或minor revision标准,有时便会做拒稿处理。可见,有效的rebuttal/response对于期刊论文的录用仍很关键。
Rebuttal时需要揣摩审稿人倾向,“一切可以团结的力量都要团结,不中立的可以争取为中立,反动的也可以分化和利用”。有的审稿人会在意见中明确表示,“如果解决了xxx,我就会提升评分”,对此一定要充分争取;对于某些审稿人提出的不足(如novelty),可能刚巧是另一位审稿人提出的优点(“This paper is interesting and novel”),一定要为我所用,让两位审稿人在paper discussion中“短兵相接”;对于borderline的审稿人,一定要充分“拉拢腐蚀”;对于初审给了positive分数的审稿人,一定要巩固基础;对于初审给了negative分数的审稿人,一定要放绝大多数的精力和rebuttal篇幅来解释澄清,争取“冰释前嫌”;
Please note that Assigned Reviewer #id has made some statements that are either against the common-sense in our field or self-contradictory (ironically his/her own confidence rating is "very confident"). blabla
We want to bring to your attention the very flawed review #id. This reviewer is self-contradictory, cf. Comment #id1, Comment #id2, and Response #id. blabla
We would like to raise attention to AC that unfortunately Reviewer #id holds a very biased view towards the contributions of our paper. blabla
常用句式
以下列举一些rebuttal中的常用句式,供大家选择使用:
开头
Thank you for your suggestion.
Thank you for the positive/detailed/constructive comments.
We sincerely thank all reviewers and ACs for their time and efforts. Below please find the responses to some specific comments.
We thank the reviewers for their useful comments. The common questions are first answered, then we clarify questions from every individual review.
We thank the useful suggestions from the reviewers. Some important or common questions are first addressed, followed by answers to individual reviews.
表达同意
We thank the reviewer for pointing out this issue.
We agree with you and have incorporated this suggestion throughout our paper.
We have reflected this comment by …
We can/will add/compare/revise/correct ... in our revised manuscript/our final version.
Due to the rebuttal policy, “authors should not include new experimental results in the rebuttal”, additional results may not be included. However, we will add these mentioned experiments and discussions in our final version. Thank you for the constructive comment.(对于CVPR/ICCV/ECCV rebuttal不能提供新结果的政策)
表达不同意
We respectfully disagree with Reviewer #id that ...
The reviewer might have overlooked Table #id ...
We can compare ... but it is not quite related to our work ...
We have to emphasize that ...
The reviewer raises an interesting concern. However, our work ...
Thank you for the comment, but we cannot fully agree with the comment. As stated/emphasized ...
You have raised an important point; however, we believe that ... would be outside the scope of our paper because …
This is a valid assessment of …; however, we believe that ... would be more appropriate because ...
解释澄清
We have indeed stated/included/discussed/compared/reported/clarified/elaborated ... in our original paper ... (cf. Line #id).
As we stated in Line #id, ...
We have rewritten ... to be more in line with your comments. We hope that the edited section clarifies …
额外信息与解释
We have included a new figure/table (cf. Figure/Table #id) to further illustrate…
We have supplemented the xxx section with explanations of ...
Thank you for the comment. We will explore this in future work.
[1] Aliaksandr Birukou, Joseph Wakeling, Claudio Bartolini, Fabio Casati, Maurizio Marchese, Katsiaryna Mirylenka, Nardine Osman, Azzurra Ragone, Carles Sierra, and Aalam Wassef. Alternatives to peer review: Novel approaches for research evaluation. Frontiers in Computational Neuroscience, 2011, 5:56.[2] CVPR/ICCV/ECCV等会议在使用Toronto Paper Matching System作为准则之一进行论文分配: http://www.cs.toronto.edu/~zemel/documents/tpms.pdf[3] 指审稿人或AC在投稿池子中申请审稿,按程度不同可大致分为:eager、willing、in a pinch、not willing等。AAAI/IJCAI等会议在沿用此方式。[4] http://cvpr2019.thecvf.com/files/CVPR%202019%20-%20Welcome%20Slides%20Final.pdf[5] Yang Gao, Steffen Eger, Ilia Kuznetsov, Iryna Gurevych, Yusuke Miyao. Does My Rebuttal Matter? Insights from a Major NLP Conference. NAACL-HLT 2019 -End-