This blog article was written by Eva Amsen for F1000 who we interviewed earlier about the young journal, open peer review and publishing negative results. The paper currently in open peer review at F1000 research reports the failure to reproduce the STAP findings earlier this year – and may set an example for the importance for publishing ‘negative results’.
Interview with Anne Færch Nielsen
Anne Færch Nielsen is a molecular biologist from Denmark and she is an editor at the EMBO Journal. She received her PhD from Aarhus University and the Ribonucleic acid took her from there to Vienna and finally to Heidelberg. As an editor she is responsible for her chemicals and things that go on inside the cell. She followed the debate on the ‘STAP’ (Stimulus-triggered acquisition of pluripotency) cells that rocked the stem cell field since the beginning of the year. We asked her about her thougths on the developments and the connection with post-publication peer review. All opinions are her own, not EMBO’s.
End of January two papers in Nature by Obokata et al. claimed a new method to create pluripotent stem cells in a surprisingly simple way. How was the first reaction in your professional environment to the big news? (more…)
Peer Review: Nuts & Bolts
Professor John Gilbert, Editor-in-Chief of the International Journal of Science Education and Alice Ellingham, Director of the Editorial Office Ltd. discuss the nuts and bolts of peer review at the Sense about Science Workshop on April 25, 2014.
This video was filmed by Lindsay McKenzie and edited by Misha Gajewski.
The Peer Review Watch Gallery: Tumblr blog
A collection of my favourite images, videos and articles relating to peer review.
Some humorous, some critical, all informative.
Enjoy!
Opinion Poll: Should Peer Reviewers be paid?
An interesting question arose at the Peer Review: Nuts and Bolts workshop on the 25th of April – should peer-reviewers be paid?
Rubriq are one publisher who pay their reviewers, saying:
“First, philosophically we feel that reviewers should be compensated for the valuable service they provide for the scientific community. Second, providing payment makes the process more formal, and can lead to more standards, training, and recognition. Finally, in order to be able to deliver high-quality, consistent reviews in a two-week timeline, it is important to provide compensation for that commitment. And by offering reviewers the option of an honorarium in lieu of payment, some of those earnings can even go directly back into research organizations.”
But what do you think?
Have your say in this poll:
You can also listen to the views of a selection of young academics here:
Top 10 chemistry journals and their impact factors for 2012
Every year Thomson Reuters publish a list of impact factors for journals to act as a rating system of how important a journal is; the higher the impact factor the more important the journal.
Many scientists and publishers dislike the impact factor but for the time being there seems little other option for rating journals.
One website, however, that uses a different method to rate journals is SCImago Journal and Country Rank. Here is a comparison between their choice of the top 10 chemistry journals and Thomson Reuters impact factors.
Graph produced using data wrapper.de (click on it to enlarge)
As you can see there is some discrepancy between the each organisations ratings, but Thomson Reuters does say that you should not rely alone on impact factor to determine how useful a journal is.
Interview with Elizabeth Moylan, Biology Editor at BioMed Central
Elizabeth Moylan
Elizabeth Moylan completed her PhD at the University of Oxford and worked as a post doc before moving into publishing. She is currently Biology Editor at BioMed Central, an open access publisher. In her role as Biology Editor, Elizabeth has editorial responsibility for the biology journals, oversees editorial polices and manages the peer review process.
Speaking at the recent Peer Review: The Nuts and Bolts workshop hosted by Sense about Science on April 25th, Elizabeth was asked to expand on the issue of ‘professional’ peer review, an interesting and novel topic for many in the audience. I asked Elizabeth to share her thoughts on this issue and others below:
What is ‘professional’ peer review?
(more…)
Public discussion of problematic studies increases the number of corrective actions. True?
Several online forums developed over the last years to foster open discussion of peer-reviewed scientific publications, PeerPub and PeerJ are two of them. The integrity of data is central to the discussion – assuming that discussed openly problems with data will helo to correct the scientific record.
But is this assumption justified?
Paul S. Brookes, a researcher at the University of Rochester in the United States, wanted to find out. He created a blog as a platform for people to submit questionable data along with the respective publications to him to be published and discussed in an open forum. (more…)
it just happened: post-publication peer review.
End of January, two Nature papers hit the media: Acid bath offers easy path to stem cells, explained Nature. Major Discovery, titled the BBC. A Breakthrough for Science, wrote the WSJ. A young japanese scientist, Haruko Obokata, from the Riken Centre for Developmental Biology had published results showing she could derive induced embryonic stem cells in a surprinsingly easy way.
The same day, Paul Knoefler, professor in the Department of Cell Biology at the UC Davis and one of the 50 most influential people in the stem cell field, wrote a critical post on his blog explaining the findings of Obokata. At the same time he raised six key questions, that according to him need to be answered before the importance of the study could be judged. A few days later the two publications started to face many questions, we summarized the developments during this phase in an earlier post.
Since then Knoefler run an informal poll on his blog where he invites the opinion of fellow stem cell experts to vote on the question “Do you believe in STAP stem cells?“. Participation the poll increased steadily from 400 to over 1000 in the latest edition as the topic grew more controversial over the past months. We accumulated the results from his polls in the graphic below.
He emphasizes that this polling is obviously not scientific but may reflect dynamic changes in the judgement of people as the discussion around the paper evolves. Without a formal post-publication peer review system in place, a sort of open peer review just happened as various research labs all around the world tried to reproduce the findings and share their results with the community openly. The papers might be retracted in the end but the stem cell field has certainly benefited from this community driven review.
Knoepfler, in one of his latest post, wrote what can be learned from this case, not only for the stem cell field but for biomedial science in general. Cellular autofluorescence and contamination might be issues restricted to certain fields but a few points, we think, can be important lessons for peer-reviewed research in general. Below is a trimmed list of these points made by Knoepfler:
- To be a good reviewer, data should always trump big names in importance. One of the problems exemplified by the STAP papers is that big name authors can sometimes sway reviewers inappropriately to be lenient on papers. In the end, as a good reviewer, you have to keep focused on the data, not the reputation of the authors.
- To editors, be extra-cautious about those “sexy” papers. A paper like either of the STAP ones is certainly exciting on first read and could have big impact. […] As with the reviewer caution above, editors should not be swayed by big name authors if the story seems too good to be true and if anything, the more excited an editor is about a paper the more cautious they should be in how they handle it. Paradoxical? Perhaps, but I think it’s true.
- To journals, give all manuscripts a thorough automated checkup. EMBO now reportedly has an automated screening process for manuscripts for image issues and EMBO editors have indicated that the STAP papers would not have passed. […] Clearly this kind of automated manuscript checkup should be standard procedure for all journals.
- Check the hype. There is nothing wrong with being excited about a paper or its potential impact, but be cautious about crossing the line to outright hype. Not everything is a “breakthrough” and that’s OK. Good, strong science doesn’t have to be a stunning breakthrough to have a positive impact. Scientists, journals, and institutions need to walk a fine line between advocating for our work publicly (which is needed) and overstating its importance, especially to the public or reporters. Many media folks are prone to hyping science as well. I believe that STAP was hugely hyped by many of the parties involved.
Interview with Professor John Gilbert, Editor of the International Journal of Science Education
Professor John Gilbert is the Editor of the International Journal of Science Education. I asked him for some insight into the role of an editor, what makes a good paper and what percentage of papers get rejected.
Interview with Alice Ellingham, Director of Editorial Office Ltd.
Alice Ellingham is founder and director of Editorial Office Ltd, a company which manages peer review for journals such as PLOS ONE.
I asked her to tell me more about the Editorial Office, her personal thoughts on paying reviewers, and the best way to conduct peer review.
Peer Review Debate VIDEO
On the 2nd April we hosted a debate called Peer Review is Broken, How Can We Fix It? After a brief talk from each of the panelists, the debate was opened up so that the audience could ask questions. This video is the panelists’ talks section of the #prwdebate.
Filmed and edited by Shivali Best and Abby Beall.
Peer Review Debate – Question Time
On the 2nd April we hosted a debate called Peer Review is Broken, How Can We Fix It? After a brief talk from each of the panelists, the debate was opened up so that the audience could ask questions. This audio is the question and answer section of the #prwdebate.
Recorded and edited by Shivali Best and Abby Beall.
Postgraduate perceptions of peer-review
City University has over 150 taught postgraduate courses. As many of the students on these courses will both undergo the peer-review process themselves, and probably peer-review someone else’s work, I thought it would be interesting to survey them on their thoughts of the process.
Here are some of my results:
1) Do you think there should be more open access journals?
Comments:
From a reader point of view, I want more open access journals, but from a writer´s view – no, because I will not publish if I have to pay! I have peer reviewed once and this was a very bad experience. It seemed to me more the cosmetics of the editor having fulfilled the peer review process than real care about my thoughts.
PEER REVIEW WATCH 