Elizabeth Moylan completed her PhD at the University of Oxford and worked as a post doc before moving into publishing. She is currently Biology Editor at BioMed Central, an open access publisher. In her role as Biology Editor, Elizabeth has editorial responsibility for the biology journals, oversees editorial polices and manages the peer review process.
Speaking at the recent Peer Review: The Nuts and Bolts workshop hosted by Sense about Science on April 25th, Elizabeth was asked to expand on the issue of ‘professional’ peer review, an interesting and novel topic for many in the audience. I asked Elizabeth to share her thoughts on this issue and others below:
What is ‘professional’ peer review?
I’d never really thought about this before until it was asked at the workshop. Do ‘professional’ peer reviewers exist? I thought it was mooted as a suggestion, but could we have professional reviewers who just do peer review?
Personally, I don’t think professional reviewers could be considered peers of the people they review. By definition they are not (practicing) academics, not submitting grants and not part of the peer community.
I am aware of statisticians who are involved in peer review in a professional capacity. Presumably their expertise is more to do with particular skills than a need to keep up in a specific scientific or medical discipline, though they too need to keep abreast of developments.
Do you think professional peer reviews are a good idea?
I think professional peer review wouldn’t work within most discipline because reviewers have to keep up with the ‘state-of-the-art’ in their given field, they need to know where is a particular field is at, and what the standards and norms are. The only way they can do this is by doing research themselves, attending conferences, submitting grants etc., and of course publishing their own papers.
While I can see a role for professional statisticians, I can’t see how professional reviewers will keep in touch with science, so I don’t think they’d be a good option.
Also, editors assess whether a reviewer is suitable or ‘on topic’ to review a given manuscript by looking at their publication record. How would ‘professional’ peer reviewers, who I presume would not be practicing academics, maintain an up to date publication record?
I think having a handful of professional reviewers controlling the field seems risky, and a move away from review by a community of peers.
Should journals pay for peer reviews?
I’m aware that Rubriq pay their reviewers, and I believe some medical journals do too. Most peer reviewers take part in the unpaid activity of peer review because they too are ‘authors’ and they depend on the good will of others get their manuscripts peer reviewed. This reciprocity makes the system work. I think most academics accept this. We do not pay peer reviewers at BioMed Central.
One wonders if people were paid to do peer review would they feel a pressure to be ‘nice’ and feel they can’t ruthlessly reject a flawed manuscript? Some journals pay statisticians as they can say that statistics reviewers are purely performing a service rather than taking part in peer review in its strictest sense.
Other systems pay their reviewers in kind, for example at Peerage of Science completing a review keeps you in credit, but submitting a manuscript deducts credit.
Rather than pay peer reviewers, can’t we give credit in other ways? For example, with open peer review, people can see who reviews articles. Also, we could assign DOIs to open peer review reports and make them citable in their own right. F1000Research are currently leading the way in this. One could then imagine a situation where another academic cited an important point made in a peer reviewer’s report, without necessarily citing the original paper.
Should peer-reviewers be trained?
I think they should receive real life on the job training, for example getting the opportunity to review a paper that their supervisor is reviewing and then submitting a joint review to learn about the aspects they should comment on. There have been some interesting discussions on this topic:
- Are we training pit bulls to review our manuscripts? http://jbiol.com/content/8/3/24
- Domesticating the beast http://www.biomedcentral.com/1741-7007/11/35
Often the reviewer form produced by the journal is a good aid to help structure reviews, and guides the peer reviewer. You can check out our ‘for reviewers’ advice that we give on BioMed Central journals. All journals have a ‘for reviewers’ tab. The majority of reputable journals will also have such guidance.
At BioMed Central we’d encourage our Editors to allow an early career researcher to write a review as long as it was looked over by their mentor too. Peerage of Science have found that it’s the junior post docs who actually write the most thorough reviews, see here: http://www.peerageofscience.org/profs-vs-postdocs-in-peer-reviewing/
You mentioned that some journals have a sharing consortium for reviews, why is this?
A sharing consortium is a group of journals or publishers who agree that if a manuscript is not accepted for publication at the author’s first choice journal, they will share the report with other journals in the consortium. So if a manuscript is sound, but not interesting enough for journal x, it could be considered by journal y.
To avoid the inefficiency of journal y inviting its own peer reviewers to review the paper again, why not use the reviews of journal x (if reviewers will reveal their names to the Editor on journal y): this would be more efficient and can make for a much faster decision.
Most publishing houses operate these ‘cascades’ in-house. At BioMed Central for example if a manuscript isn’t appropriate for Genome Biology, it could be considered by BMC Biology or BMC Genomics and published there if the author agree. It can work in the other direction too.
We have a number of such links between different journals in different subject areas. However, portability of peer review between publishers is the next step. For more reading see:
You can read Elizabeth’s blog following the Peer Review: Nuts and Bolts workshop here: http://blogs.biomedcentral.com/bmcblog/2014/04/28/are-there-alternatives-to-peer-review/