Writing Effective Revision Response Letters

• Read original article here

When you submit a paper to a journal for the first time, the most likely outcome (besides rejection) is not usually acceptance — for myself, I think I have only had a single paper accepted to a journal in the first round during my entire career. Rather, the most common outcome is a request for a major or a minor revision, the implication being that the journal editor and reviewers think your work is promising, but that it has certain imperfections that need to be addressed prior to publication. Also implicit in revision requests is the very real possibility that a paper will ultimately be rejected if the authors fail to adequately address reviewer feedback in new revisions.

In my opinion, this mechanism of a multi-round review process, where the authors and reviewers can engage in a conversation about a piece of research, is liberating in comparison to fire-and-forget one-shot conference rejections, and ultimately leads to better science. But that’s the topic of another blog post and not the point I am trying to make today.

When submitting a new version of a paper in response to a revision request, it is often required that the authors also submit a letter that discusses reviewer feedback from the previous round and outlines the changes made to the paper in response. Every time we get to the point when my students need to write such a response letter, I find myself reiterating the same basic guidelines. To save myself some time and effort, I decided to write a small guide on how to do this instead. Then I realized that there are probably others in the community who would benefit from such a guide, so I decided to post it online (as I write this, we are getting very close to the second round for the IEEE VIS 2017 conferences, so this is foremost in my mind).

Let’s start with the most basic question: how do revision letters even look? Well, the word “letter” offers a clue. While there are many ways to structure your letter, I suggest using the simplest possible format: a simple document containing the following parts:

First of all, this is a letter, so don’t forget a brief introduction providing a little bit of context (part 1). It is customary and respectful to thank your reviewers here. Then follows the executive summary of changes (part 2). For a busy reviewer, this is the important bit: how the authors have modified their work in response to the reviews they received. That busy reviewer will likely look for their own name (i.e., reviewer label such as R2 or R3) to ensure that the authors paid attention to them. And finally, you will need to close the letter (part 3) with some basic pleasantries (again, it’s a letter, and the reviewers spent a lot of their time giving you all this invaluable feedback, so this should not just be empty words).

The first three parts of the letter should ideally fit on a single screen (or page). What follows after these three, the line-by-line responses (part 4), is often (much) lengthier, and may in some cases be omitted if there are space limitations to your revision letter.

If there are no such restrictions, I find a line-by-line response to be the most transparent and most courteous way to address reviewer feedback. It essentially shows the reviewers that the authors have nothing to hide and are earnestly trying to discuss each of their comments. In my experience, the best way to do this is to respond to reviews like how you respond to emails inline: by quoting the review (using either > like in emails, or some other typographic convention to differentiate the review from your response) and then giving your response underneath. For unimportant parts (review form boilerplate, whitespace, or comments about the work that warrant no response), I usually abridge the reviews and write “[snip]” to show that I’ve elided certain parts. It’s all a part of being as open and transparent as possible in order to cultivate trust between the authors and the reviewers.

Finally, sometimes I ask my students to actually highlight the revisions they made in the text of the revised paper itself, for example by coloring all the new and modified text so that it stands out from the rest. This is an example of respecting the reviewers’ time by eliminating the need for them to have to compare (or even create a visual diff) the new revision with the previous one. It is often more work to do this, but if you are making major revisions already, it is a great way to showcase how committed you are to transparency and making your work better. (Obviously this won’t work for situations where the new revision you submit is the camera-ready for publication, but that doesn’t happen often in my experience.)

Of course, there are many other ways to structure your feedback. An alternative approach, which I’ve seen some people use, is to include a table that summarizes the main comments from the reviewers and then responds to them. Essentially, it’s a specialized form of the list of changes (part 2 in my list above). I don’t like it myself, because tables are not always space-efficient for long prose and may not be supported in a text-only format, but that’s a matter of taste. I continue to feel that including a line-by-line response (part 4) is the most courteous and transparent way to respond to reviews.

Writing revision response letters is not unlike writing rebuttals, and I actually wrote a guide on this last year. The main difference is that rebuttals mainly deal with defending the paper, clarifying certain points, and rebutting reviewer feedback, with just a sprinkling of suggested changes to be made. In other words, rebuttals deal with things that willbe done if the paper is accepted.

In contrast, a revision response letter is written fait accompli, i.e. it talks about changes that have already been made. This eases the burden of having to motivate how a proposed change is doable within the allotted time, because you’ve already made the change! In other words, if in a rebuttal you would have a hard time getting away with promising a new user study, in a revision letter you can proudly and factually just state that you conducted the new study and would the reviewers please take a look? This is your chance to blow their socks off.

As I stated in my rebuttal guide, an important aspect of communicating with reviewers is to ensure that you respect the contributions of all of the reviewers, even the ones who were either positive (little to comment on), wrote a short review (again, little to comment on), or raised arguments that were not unique (here you may feel that there is no need to reiterate already discussed points). If you don’t do this, or if you don’t do it well, you run the risk of alienating one or more of your reviewers.

There are basically three strategies that an author can use to handle a specific critique in a review (where a single review often contains many different critiques):

If we put ourselves in the shoes of a reviewer for a moment, you will see that the first strategy is very frustrating. The reviewer read the paper and found a flaw that should be addressed, but the author didn’t even deign to address it in their response. This is clearly not a desirable situation, and, as an author, you should try to avoid it. This is why a line-by-line response is useful because you are basically forcing yourself to respond to everything the reviewer says in some way.

Of the other two strategies to handle critiques, reviewers naturally feel the best about S3, because the authors are taking their critique so seriously that they changed the paper in response. It is the ultimate validation, because as an author, you are acknowledging that the reviewer brought up such an important point that there was a need to change the paper itself so that future readers would also get the benefit of this insight. Sometimes a combination of S2 and S3 work best. If as an author you’re finding yourself using strategy S2 all the time, you are basically writing a rebuttal; this is fine, but you should know that there is a risk that the reviewers feel that you are arrogant and that their feedback is being ignored. In my opinion, actually making tangible revisions to the paper (strategy S3) that will benefit future readers, and not just the reviewers, should be the recommended approach whenever possible.

Here’s a small point, but an important one: when you actually made changes to the paper in response to reviewer feedback, be sure to make this explicit in your response letter. Too many times have I seen my students bury the fact that they ran a new study or built a new example deep in a paragraph, to the point that reviewers are likely to miss this. If you made a change to the paper, particularly if it is a significant change, be sure to use clear wording to state that this is new work you did in response to the reviews. Otherwise, if you are unclear, it may sound like you are trying the second strategy, i.e. just discussing their feedback with them and pointing to existing things that were already there in the paper. Remember that reviewers have limited time to hunt around for differences between one version and the next. This is also why a list of changes that summarizes significant revisions is useful to include at the beginning of the letter.

It is important to realize that peer review is not an egalitarian process. The editors and reviewers have virtually all of the power (that power is distributed across several people, which hopefully will minimize the risk for unilateral unfair decisions, but it is still true). Rebuttals and revision response letters give the authors a mechanism for talking back to the reviewers to defend their work, but in the end, the final decision always rests with the reviewers. In other words, it doesn’t really matter if you as an author “win” the argument; if you manage to alienate the reviewers in the process, they may still decide to reject your paper.

This doesn’t mean that authors should be weak and give in to every whim and wish of the reviewers. Reviewers will generally listen to reason if you give it to them in a respectful yet assertive manner. For example, if one of the reviewers is asking for an entirely new user study, but you think it is a waste of time (presumably not because you can’t be bothered, but because it is not scientifically justified), clearly and calmly arguing for your viewpoint may still sway the reviewers; perhaps not the particular reviewer who brought up the point, but the editor and other reviewers may be enough to still carry your paper to acceptance.

However, it does mean that you should check your ego at the door. Sometimes when reviewers misunderstand your work or use a provocative turn of phrase, it can be tempting to respond in kind with a snarky or even an outright hostile response. Try to avoid the temptation; given the unequal power dynamic of the situation, your snarkiness is likely only going to result in a rejection, and your audience (your co-authors and the small set of reviewers) is not big enough to warrant burning your bridges. If “meek” is defined as “humbly patient or docile, as under provocation from others” (Dictionary.com) or “enduring injury with patience and without resentment” (Merriam-Webster), then you should be meek (not weak).

You should likely pick your battles to maximize your chance of ultimate acceptance. Don’t go in with a “rebuttal mindset”, where your goal is to do as little work as possible (i.e. strategy S2). It’s called a “revision” for a reason. If the reviewers ask for a better introduction, rewrite it. If they list missing references, cite them all. If they need a clarification, make it. Save your powder for the lines of critique you feel strongly about and which may not be scientifically justifiable.

Finally, you should recognize that sometimes there are required changes that you really need to make for a paper to be accepted. These are not open for debate, and trying strategy S2 (or, worse, S1) on these will result in your paper being summarily rejected. There are usually clues about this in the reviews themselves. For example, pay close attention to when the primary reviewer or the associate editor lists “required changes”. Ignore these at your peril. Really, as I said earlier, actually making the changes should be your main strategy.

Writing revision response letters is almost an art in itself, and can sometimes be nearly as time-consuming as actually making the changes themselves. In fact, starting to write the response letter often tells you exactly what you need to do to change your paper so that you can defend yourself effectively. The response letter also becomes a changelog for your work. For this reason, revising the paper and writing the letter often go hand in hand.