Wednesday, July 25, 2012

The Harsh Message at the California Stem Cell Agency

Grant reviewers have delivered a harsh message in the latest $243 million research round at the California stem cell agency – at least that is one way to look at it.

In effect, they told the governing board of the $3 billion enterprise that the overwhelming majority of applicants in its signature disease team round do not measure up, despite the fact that CIRM had early on partially vetted their efforts. Indeed, the reviewers said that the researchers deserve only $113 million instead of the full $243 million that was budgeted.

Obviously the results of the review can be interpreted in other ways as well. But the review outcome should raise some flags within the stem cell agency and its 29-member board, which meets tomorrow in Burlingame. It may not auger well for future rounds that also involve CIRM's newly energized drive to push research into the clinic.   

One interpretation of the review results could well be that CIRM's goals are unrealistic, that the agency is trying to move too fast for the normally glacial pace of research and development. Another interpretation is that the science is not good enough in California to accomplish what the agency is seeking to do, a view expressed by some in the early days of the nearly 9-year-old program. Another is that the reviewers themselves don't know enough or have failed to do their homework, which some of the rejected applicants have argued in their appeals. Yet another is that the CIRM review process is inadequate to the task of meeting CIRM's goals. And still another interpretation is that the normal peer review process on which CIRM's procedures are based is mightily flawed, a general contention argued by some(See here, here and here.)

Or quite possibly the result of the disease team reviews could reflect a combination of all of the above, to one degree or another.

Little is known about the substance of what goes on during the grant review process, aside from the staff-written review summaries. Even CIRM board members, who see only the summaries, have complained from time to time about not having enough information to make a good judgment on an application. Reviews are conducted behind closed doors. Information about the economic and professional interests of reviewers is withheld from the public by the stem cell agency.  

Here is a look, however, at what we do know. Initially the universe of applicants in this round totalled 36. That was the number that applied for planning grants for this round. Without a planning grant, they could not apply for a full $20 million award, with some exceptions. The exception process was controlled by CIRM President Alan Trounson, not reviewers. CIRM used the planning grants and the exception process not only to assist applicants but to winnow out weak applications.

Nineteen researchers won planning awards. With exceptions included and minus dropouts, 22 applied later for the big money. Out of the 22, only six were recommended for funding by reviewers, who are known more or less formally as the Grants Working Group. (See the four items at the end of this piece for a list of reviewers involved.)

In the past, reviewers have sometimes not approved sufficient applications to consume the entire amount budgeted for a round. But they have never produced a shortfall as great as in this case. It is all the more dramatic since this round carries a lot of weight for CIRM, which is pushing hard to commercialize research and fulfill at least part of the promises that were made to California voters in 2004 to win approval of creation of the stem cell agency.

One reflection of the unusual nature of the round is the record pace of researchers' appeals of negative decisions by reviewers. At least nine of the 15 rejected scientists are willing to say publicly that something is is not quite right in the review process, ranging from missing facts to inconsistencies in CIRM's endorsement of particular paths of research.

It is safe to say that CIRM directors tomorrow will pluck some applications out of the reject bin and increase the total awarded. But they should also examine the process to determine what generated this particular outcome. The Institute of Medicine, which is currently engaged in a $700,000 examination of CIRM, also might scrutinize this round with some care, given its size and importance to the California stem cell research effort.

1 comment:

  1. The following comment concerning the latest disease team grant round was received from a representative of a company that has applied multiple times unsuccessfully for funding from CIRM. It was asked that the comment be posted anonymously.

    “In reading most of the current material it all appears to be the same stuff.  Applicants complain that the reviewers must be reading some other application and at the same time they do not take the progress made by the applicant during the waiting period into consideration.  That was certainly true for my company and is a common complaint by several companies I personally know. ...  CIRM is just as broken as it was 4 or 5 years ago.  I do not see Duane Roth’s name anywhere in this process – so much (for) the advocate for business.”

    (Editor's note: Roth is co vice chair of the agency and has complained for years about the lack of funding for business.)