Yamanaka and the Frailty of Peer Review

More than one back story exists on Shinya Yamanaka and his Nobel Prize, but one that has received little attention this week also raises questions about hoary practice of peer review and publication of research – not to mention the awarding of billions of dollars in taxpayer dollars.

The Yamanaka tale goes back to a 2010 article in the New Scientist magazine by Peter Aldous in which the publication examined more than 200 stem cell papers published from “2006 onwards.” The study showed an apparent favoritism towards U.S. scientists. Also specifically reported were long delays in publication of Yamanaka's papers, including in one case 295 days.

Here is part of what Aldous wrote,

“All's fair in love and war, they say, but science is supposed to obey more noble ideals. New findings are submitted for publication, the studies are farmed out to experts for objective 'peer review' and the best research appears promptly in the most prestigious journals. 

“Some stem cell biologists are crying foul, however. Last year(2009), 14 researchers in this notoriously competitive field wrote to leading journals complaining of "unreasonable or obstructive reviews". The result, they claimed, is that 'publication of truly original findings may be delayed or rejected.' 

“Triggered by this protest, New Scientist scrutinised the dynamics of publication in the most exciting and competitive area of stem cell research, in which cells are 'reprogrammed' to acquire the versatility of those of an early-stage embryo. In this fast-moving field, where a Nobel prize is arguably at stake, biologists are racing feverishly to publish their findings in top journals. 

“Our analysis of more than 200 research papers from 2006 onwards reveals that US-based scientists are enjoying a significant advantage, getting their papers published faster and in more prominent journals (find our data, methods and analyses here). 

“More mysterious, given his standing in the field, is why two of Yamanaka's papers were among the 10 with the longest lags. In the most delayed of all, Yamanaka reported that the tumour-suppressing gene p53 inhibits the formation of iPS cells. The paper took 295 days to be accepted. It was eventually published by Nature in August 2009 alongside four similar studies. 'Yamanaka's paper was submitted months before any of the others,' complains Austin Smith at the University of Cambridge, UK, who coordinated the letter sent to leading journals. 

“Yamanaka suggests that editors may be less excited by papers from non-US scientists, but may change their minds when they receive similar work from leading labs in the US. In this case, Hochedlinger submitted a paper similar to Yamanaka's, but nearly six months after him. Ritu Dhand, Nature's chief biology editor, says that each paper is assessed on its own merits. Hochedlinger says he was unaware of Yamanaka's research on p53 before publication.”

Last week, Paul Knoepfler of UC Davis wrote of other issues dealing with peer review, but coincidentally also dealing with iPS cells. What New Scientist and Knoepfler are discussing is not an isolated situation. It is part of a continuum of complaints, both serious and self-interested but exceedingly pervasive. A Google search today on the term “problems with peer review” turned up 10.1 million references.  Writing on Ars Technica last year, Jonathan Gitlin, science policy analyst at the National Human Genome Research Institute,  summarized many of the issues, citing a “published” (our quotation marks) study that said peer review doesn't work “any better than chance.” Gitlin said,

“A common criticism is that peer review is biased towards well-established research groups and the scientific status quo. Reviewers are unwilling to reject papers from big names in their fields out of fear, and they can be hostile to ideas that challenge their own, even if the supporting data is good. Unscrupulous reviewers can reject papers and then quickly publish similar work themselves.” 

At the $3 billion California stem cell agency, peer review is undergoing some modest, indirect examination nowadays. The agency is moving towards tighter scrutiny of budgets proposed by applicants. And, following a record wave of appeals this summer by disgruntled applicants rejected during peer review, it is also moving to bring the appeal process under more control.

As the agency tries to move faster and more successfully towards development of commercial therapies, it may do well to consider also the frailties of its peer review process and the perils of scientific orthodoxy.