The way peer review is currently run, it’s a cursory check that whatever is in the manuscript isn’t completely crazy. So many papers that have no business getting published make it through the cracks. And many good papers have issues that were missed in peer review. Of course, we’re never going to get things 100\% right. But the current system privileges publication efficiency (i.e. getting more publications) over publication quality. I don’t think the changes we make would even need to be that drastic.
If we expect more from reviewers, the first thing we have to do is recognize the contribution they make to improving the quality of a manuscript. I’ve mentioned previously that I think that a good peer reviewer contributes more to a manuscript that many middle authors:
Think about two people:
— Jeremy Labrecque (@jalabrecque) December 6, 2019
a) a middle author whose only contribution was two minor comments on a manuscript
b) a decent reviewer
Think about which does more to improve a paper. And now think about how much credit each gets. This system sucks. #epitwitter
I’m not talking acknowledging them in end of year lists published by a journal thanking the reviewers from the past year or even through Publons which makes certain aspects of reviewing public. Their name needs to be associated with the paper. Some journals like the BMJ already do this with open peer review. Anyone can go in and see who reviewed a paper and what their comments were. All journals should do this. (And I’d even advocate for some kind of system where you could do the first five peer reviews of your career anonymously to get your reviewing feet under you.)
But I mean something a little more. Each reviewer should have the opportunity to write their own discussion section that will accompany the article. After all, why should the authors be the only ones that get to publish a discussion of their own work? Seems kind of biased, no? Reviewers have already taken the time (hopefully) to carefully read the paper. Why not let them publish how they would have written the discussion? It could even be a way to make visible points of disagreement between the authors and the reviewer. The authors get to keep their original point in their article and the reviewer gets to point out why they disagree. Imagine having multiple viewpoints on an article all published together.
You might say, how are we going to have time for this? This type of review process will take even more time than peer review already takes. Yes, it will take more time. But we get the benefit of multiple perspectives of the same paper and the reviewers get a publication out of it (we can talk about follies of bean-counting publications separately but at least this gives more credit to the reviewer for the work they’ve done). And I hope it would encourage a shift in how we publish as well. Fewer papers but more in each paper. Either papers would explore more than one aspect of broader research question or conduct multiple analyses on one question and triangulate across studies.
