Tuesday Science just doesn't have the same ring to it, does it? I'll have to keep myself honest about this - although if that video repost doesn't buy me an extra few days of leniency, I don't know what does. How often do we run into something that scientifically domain-specific and funny?
For reasons that will hopefully soon cease to be relevant, we've been dealing this week with responding to, rather than writing, reviews. Despite the fact that I really do view reviews as fluid conversations between authors, referees, and editors, I find that the process of responding to reviews involves a substantially different mindset than writing one in the first place. The reasons are probably both obvious and irrelevant. When writing a review, you're squeezing out a bit of time to deal with a situation that's been unwantedly thrust upon you. You're an outside observer, and unless you've already seen the work presented (which, admittedly, is the most commonly targeted review situation), you're likely to start out pretty dubious. You're like a child who wanders into the middle of a movie.
Conversely, when you're responding to a review, somebody's just stabbed your baby; vengeance must be had. The first rule of responding to reviewers is never send the first thing you write. Write it - let it all out! - but don't send it. This is probably familiar advice for any upsetting email, any it's equally applicable to review responses. I actually find that writing a near stream-of-consciousness initial point-by-point response does help frame the eventual "real" response, and it can organize your thoughts and vent a lot of bile in the process. Sort of the scientific version of primal scream therapy.
Once that's out of the way, there are three main considerations in a response:
- First, referees almost never re-read the paper in any depth (except for figures). Consider the response to be a self-contained document, and include quotes from each review point to which you respond, quotes from the original paper as needed, quotes from every modification/addition to the manuscript, and any new relevant data (again, figures can be excluded). And for the love of anything, track changes in your resubmitted manuscript, or at least go through and highlight everything that changed by hand.
- Second, obey the principle of least effort whenever possible. That is, do the simplest possible additional experiments that will address reviewer questions, make the smallest possible changes to the manuscript in response (don't be afraid to add to the supplement, though), and avoid doing either whenever possible (I'd actually suggest ignoring minor quibbles if you think you can get away with it). This is not out of laziness or a desire to be contentious, but out of consideration for the fact that, presumably, the manuscript as originally submitted was (at least in your opinion) a finished product. The goal of a review is not to create a new product, it's to polish and clean something that's already complete. Don't fix what's not broken, and don't let the referees scare you into it (although placate them when necessary).
- Finally, take neither yourself nor the reviewers too seriously. It's easy to be a jerk as a reviewer, and it's easy to be a jerk responding to one - don't fall into that trap. A modicum of formality is necessary, and it's standard practice to thank the reviewers profusely even if they're goofs, but I smile at the occasional joke cracked in a review or response.
There's perhaps not a lot to say in addition without launching straight into an example - review responses, and even individual referee comments, are best handled on a case-by-case basis. A single reviewer can, obviously, make suggestions that range from the ridiculous to the sublime. Although I'm tempted to reuse the mock Sleipnir example I cooked up earlier, I'm going to go out on a limb here and discuss a portion of the actual response I gave to reviews of a recent publication. If by some astronomical chance one of the referees ever reads this, it may be more of a look inside my thought process than I'd care to share...
We'd like to thank the editor and both referees for your helpful comments on this manuscript. We greatly appreciate the input, and we've enclosed a revised version that will hopefully address the concerns in the reviews, to which we detail our responses below. ==== First, many thanks to Referee #1 for your detailed review of the tutorial's example. We've addressed these issues and your broader structural comments as follows: ---- * "I would prefer if the authors instead spent a few more words on explaining what exactly is done in each step, and perhaps moved the command lines and scripts to supplement." This is an excellent suggestion, and we've made the appropriate modifications - the step-by-step instructions, scripts, etc. are now provided in a supplement, and the box contains a more intuitive description of each processing stage necessary to hypothesize a collection of cell cycle-linked kinase target interactions. ---- 1. "The authors aim to find targets of cell-cycle kinases. To this end, they start by making a set of 223 kinases that are annotated with the GO terms 'cell cycle' and 'kinase activity'. However, since the authors are looking for protein targets of kinases, they should have limited the query to 'protein kinase activity' since only 135 of the 223 kinases they retrieve are protein kinases." Also a good point, and we've made the appropriate change (fortunately, while only a bit more than half of the kinases are protein kinases, 51 of the 55 kinases in the cell cycle intersection are protein kinases, so few downstream results change). ---- 2. "The authors then retrieve protein-protein interactions from several databases to find potential targets of the kinases. However, interactions between kinases and their substrates are very transient for which reason they are generally not found by the types of assays that were used to produce the vast majority of the data in these databases." This is a valid biological point, although it fortunately has a minimal impact on the example of computational data manipulation provided here. For clarity, we've included the following text in the revised, more intuitive version of the Box 1 instructions: >>> Note that this will provide a conservative underestimate, since many transient kinase-target interactions are difficult to detect based on high-throughput data. ==== In response to Referee #2, we're glad you appreciated the manuscript, and many thanks for your time and input. In response to your specific comments: ---- * "Figure 1: The significance of the first 2 network diagrams at the top (for the 'possible inputs' part of the figure) is not so clear. Sequence and microarray data can be used to produce networks, but they generally aren't described in these terms. Perhaps all of the 'input' network diagrams could be removed, as I'm not sure they add that much (unless a description is added to the legend)." This is an interesting point, and one which it's difficult to delve into in much detail in this short tutorial. One of the reasons biological networks are such a popular model is that they remain uniform in structure - directed, weighted graphs - while providing sufficient generality to describe almost every biological system or measurement. While this is most often leveraged in a predictive context (i.e. producing a biological network as an output model), it's also often an important aspect of data integration: almost every experimental result can be viewed as a network, providing a uniform framework for input (as well as output). Hence illustrating each data type as a network in Figure 1. While this somewhat philosophical argument would be out of place for a tutorial and isn't presented in detail in the manuscript, it's implied in several of the analysis steps, as well as in Figure 1. Although there's no room to do it justice in the figure itself, we've added the following text to the figure legend, which is perhaps the most appropriate place to convey it as a mindset for integrative analyses: >>> Note that both the "output" pathway model and the "input" experimental data are represented as networks: directed regulatory binding site targets, undirected weighted coexpression, and undirected interactions, respectively. As demonstrated by the sample analysis in Box 1, biological networks provide a uniform framework within which both experimental data and predicted models can be represented, facilitating integrative analyses. ==== Finally, the two editorial points - the organizational structure of "Methods and Pitfalls" and the content of the example Box - are included in these revisions as well (points made by Referee #2 and #1, respectively). We have tracked changes in the resubmitted manuscript, such that all affected text is highlighted (as well as being listed here). Thank you again for your time and effort, and for helping to improve the manuscript. We hope that these changes have made it more appropriate for publication, and we look forward to your response.
First, the boring part: notation. Since review responses are sometimes (although decreasingly often) submittable only as plain text, I try to use as consistent of punctuation as possible to delineate portions of my response. Here, referees are separated by lines of equals (====), referee comments are separated by lines of dashes (----), individual comments are bulleted (*) or numbered (1. etc.) to parallel their bulleting/numbering in the reviews themselves, and additions to the text are quoted with greater-thans (>>>). When I make deletions from the manuscript (none were called for here), I quote the deleted text with less-thans (<<<).
The first and last paragraphs are almost completely boilerplate. I always try to start by thanking the editor and reviewers for their time. I do appreciate review feedback, because I don't generally have enough self-confidence to really believe that I'm not making horrible mistakes until several additional pairs of eyes have taken a look. Referee input does almost always improve the manuscript ("almost" being an operative word), and in the closing paragraph, we do look forward to the subsequent editorial response. Nothing in these two paragraphs should surprise anyone.
In responding to individual referees (and, in this case, a couple of specific editorial comments; yes, the editor can throw in a review, too), I start each section with an overview section hitting the same major points that they did. In hindsight, I was actually a bit too effusive in this response, and I would have preferred to just state the one or two major items discussed in either case (an extended tutorial-style example and an overall positive, minimal review).
In the first response point ("I would prefer if the authors instead..."), I've moved a detailed section from the manuscript into the supplement and replaced it with a simpler, intuitive overview. I liked the change, and in general adding to the supplement at the expense of the main paper is usually good. Papers are too long; make them shorter!
The next few responses are fairly straightforward, and the next interesting example is in response to the second referee ("Figure 1: The significance...") This is a long, three-part response in which I aimed to do three things: dodge the issue, explain why in the response, and make a minimal change to the manuscript in the interest of satisfying the request. Don't be afraid to explain why you did something in the response: why you wrote or did something in the original paper, why you did or didn't make a requested change, or why you agree or disagree with a comment. Explanations are crucial, and it's much easier to be accepting of something with which you disagree if you can at least understand the reasoning behind it. Responses are even more personal than reviews, and you can be speculative, philosophical, or chatty in a response in a way that you never could in a publication.
The final remark (with respect to the editorial comments) highlights one last important principle of review responses: the more you can interrelate the responses to different referees, and the more you can kill multiple birds with single stones, the better. Don't be shy about calling out positive comments by one referee in your response to a negative comment by another! The editor is ultimately making a decision about the acceptability of your paper, and the hope is that it'll be at least a marginally democratic process; if the community as a whole would like your manuscript in that journal, that should be reflected by the reviewers. In responding to them, you can very much say things like, "I don't think I want to change X for #3 because #1 and #2 both told me that X is great. Likewise, I've already fixed Y while responding to #1 by doing such-and-such."
An omission in this example response, due to the fact that it's a snippet of an already short set of comments on a high-level review article, is the importance of specific data. Inasmuch as I encourage changing the manuscript itself as little as is practical, it is critical to provide evidence, data, and references to the referees whenever possible. Adding citations to a manuscript is easy; adding them to the supplement is easy; adding them to the response is easy. Provide numbers and mini-experiments in the response; add figures to the supplement. In God we trust and all.
The goal of a review response is obviously to improve your product, though, and regardless of all else, take that point seriously! It's easy even at a macroscopic level to get locked into a few trains of thought or to inadvertently get caught wearing intellectual blinders. At the microscopic level of a research project on which you've been laboring for months or years, it's almost inevitable. A response to reviewers is a chance to let someone else do your work for you - when a change isn't hard to make and will make your research better, do it. And be grateful! Remember that even if authors who attempt to identify referees are almost always wrong, they know who you are, and the editor knows who everyone is, so misbehavior can make your future with that group of individuals very awkward.
Now if only all of this was as easy as I'm trying to make it sound... I'll have to keep my own advice in mind the next time a referee calls for sixteen new experiments!