The 4 Jobs of a Referee in Peer Review
Ethan Siegal,
Jul 15, 2013
I reviewed another paper today, one of the better papers I've seen recently, and so naturally gave it the 'publish without changes' seal of approval. In my comments I was critical of the assumptions, thrust of the argument, and even the references used to support the work. Reading academic literature is like watching history being rewritten before my eyes. Anyhow, this paper usefully reconciles how I can be so frustrated with a paper and yet give it the thumbs up. The referee, argues the author, has four jobs:
- To verify that the introductory section(s) adequately sets up, explains, and places their work into its appropriate historical and scientific context
- To examine the methods used to obtain and analyze the data presented in this paper? Are they sound? Are there questions about their validity?
- To determine whether results are clear, complete, and actually reflect what was in the data (if any)
- To determine whether conclusions reached be justified based on both the combined, pre-existing breadth of knowledge of the field and the work done in this paper?
In other words, a reviewer doesn't get to 'go meta' on the paper. Sure, I think the cited literature is mostly second-rate derivative stuff, but like it or not, it is state of the art in the context of the journal, and probably the best we could hope for. Similarly, I may doubt the foundations of the empirical work or analysis, but the discipline as a whole does not share my scepticism. So it's in.
Today: 5 Total: 98 [Direct link] [Share]
View full size
