Anne Færch Nielsen is a molecular biologist from Denmark and she is an editor at the EMBO Journal. She received her PhD from Aarhus University and the Ribonucleic acid took her from there to Vienna and finally to Heidelberg. As an editor she is responsible for her chemicals and things that go on inside the cell. She followed the debate on the ‘STAP’ (Stimulus-triggered acquisition of pluripotency) cells that rocked the stem cell field since the beginning of the year. We asked her about her thougths on the developments and the connection with post-publication peer review. All opinions are her own, not EMBO’s.
End of January two papers in Nature by Obokata et al. claimed a new method to create pluripotent stem cells in a surprisingly simple way. How was the first reaction in your professional environment to the big news? (more…)
End of January, two Nature papers hit the media: Acid bath offers easy path to stem cells, explained Nature. Major Discovery, titled the BBC. A Breakthrough for Science, wrote the WSJ. A young japanese scientist, Haruko Obokata, from the Riken Centre for Developmental Biology had published results showing she could derive induced embryonic stem cells in a surprinsingly easy way.
The same day, Paul Knoefler, professor in the Department of Cell Biology at the UC Davis and one of the 50 most influential people in the stem cell field, wrote a critical post on his blog explaining the findings of Obokata. At the same time he raised six key questions, that according to him need to be answered before the importance of the study could be judged. A few days later the two publications started to face many questions, we summarized the developments during this phase in an earlier post.
Since then Knoefler run an informal poll on his blog where he invites the opinion of fellow stem cell experts to vote on the question “Do you believe in STAP stem cells?“. Participation the poll increased steadily from 400 to over 1000 in the latest edition as the topic grew more controversial over the past months. We accumulated the results from his polls in the graphic below.
He emphasizes that this polling is obviously not scientific but may reflect dynamic changes in the judgement of people as the discussion around the paper evolves. Without a formal post-publication peer review system in place, a sort of open peer review just happened as various research labs all around the world tried to reproduce the findings and share their results with the community openly. The papers might be retracted in the end but the stem cell field has certainly benefited from this community driven review.
Knoepfler, in one of his latest post, wrote what can be learned from this case, not only for the stem cell field but for biomedial science in general. Cellular autofluorescence and contamination might be issues restricted to certain fields but a few points, we think, can be important lessons for peer-reviewed research in general. Below is a trimmed list of these points made by Knoepfler:
- To be a good reviewer, data should always trump big names in importance. One of the problems exemplified by the STAP papers is that big name authors can sometimes sway reviewers inappropriately to be lenient on papers. In the end, as a good reviewer, you have to keep focused on the data, not the reputation of the authors.
- To editors, be extra-cautious about those “sexy” papers. A paper like either of the STAP ones is certainly exciting on first read and could have big impact. […] As with the reviewer caution above, editors should not be swayed by big name authors if the story seems too good to be true and if anything, the more excited an editor is about a paper the more cautious they should be in how they handle it. Paradoxical? Perhaps, but I think it’s true.
- To journals, give all manuscripts a thorough automated checkup. EMBO now reportedly has an automated screening process for manuscripts for image issues and EMBO editors have indicated that the STAP papers would not have passed. […] Clearly this kind of automated manuscript checkup should be standard procedure for all journals.
- Check the hype. There is nothing wrong with being excited about a paper or its potential impact, but be cautious about crossing the line to outright hype. Not everything is a “breakthrough” and that’s OK. Good, strong science doesn’t have to be a stunning breakthrough to have a positive impact. Scientists, journals, and institutions need to walk a fine line between advocating for our work publicly (which is needed) and overstating its importance, especially to the public or reporters. Many media folks are prone to hyping science as well. I believe that STAP was hugely hyped by many of the parties involved.
In October 2013, a fictional scientist, Ocorrafoo Cobange, who worked at a fake institution, the Wassee Institute of Medicine in Asmara, had a paper accepted for publication by 157 journals, despite the fact the paper had obvious errors that could have been spotted by someone with GCSE level science.
In reality, the author of this experimental paper was John Bohannon, a correspondent for Science who wanted to show the flaws of current peer-review processes.
The debate between peer review and open access reveals a bigger problem: the publishing business.
As Katie Williams points out when discussing academic publishing it is impossible to escape the debate of traditional peer-reviewed versus open access peer-review.
Peer review has always been the gold standard of science. However, ask an academic about peer-review and you’ll likely hear a grumble or two. (more…)
The effectiveness of peer review has been investigated in a new paper entitled Modelling the effects of subjective and objective decision making in scientific peer review that was published on 5th December, in Nature.
Researchers from the School of Economics, Finance and Management at the University of Bristol studied the system and presented a new model that improves on the current peer review system. They used a mathematical model to understand the behaviour of scientists when undertaking a review. (more…)