Monday, November 02, 2009

Text of Csete's Description of CIRM Grant Monitoring

On June 17, 2009, Marie Csete, then chief scientific officer for the $3 billion California stem cell agency, briefed its directors in San Diego on how CIRM monitors the progress of its grantees. Following her presentation, one director asked her how many grants had been terminated because of lack of progress. Three, she replied.

Here is the transcript of her remarks and the discussion by directors. Here is a link to the slides that she used. They begin on p. 22 of the file.

Csete: I guess the last thing I'm supposed to speak about is our mechanism for looking at progress reports. And the first opportunity we had to aggregate data on this was for the SEED grants, which are now more or less in their second year of funding for most of the investigators. And as a reminder, I think all the way back a few years, this was a grant program designed to develop human embryonic stem cell biology in the state. And since it was early, it was more idea based rather than preliminary data-based, and we really hoped to attract nonstem cell biologists to the field as well as cell and developmental biologists who were working perhaps on other stem cells, but had not done human embryonic stem cell work.

As such, the seed grants were acknowledged to be rather high risk, high gain. And I was involved in the SEED grants as a reviewer. So I have insight into the process from the beginning even before I came here to CIRM. I have to say that the overwhelming message I want to leave you with here is that despite a slow start, that was the bump in the road, that the SEEDs are really overwhelmingly successful. And we looked this week to find that there are already 64 papers coming out of the SEED program even though, again, these were new investigators in this field.

So it is important also, thanks to Bettina (Stephen, a CIRM science officer), to remind you that progress reports are not just progress reports -- and this has been an education for both the science office and for our scientists – that they really serve as a focus of a way that the science officers and our grantees can have a point of communication. It also allows us to get a heads up on where the data is, on what papers are being submitted, on potential patents that are coming out. Also when we discuss the progress reports in the science office meeting, it allows us to match scientists from our individual portfolios with other scientists whose progress we're hearing about in the meeting. In general, I have to say that I've gotten a lot of positive feedback from our PI's about the interactions with our science officers and our grantees.

So this is the process that we've sort of come to. And I've condensed a very complex diagram that has arrows going out every which way. On the right(on a slide she presented), you have a lot of gold stars. And the gold stars is what usually happens. On the left I put symbols showing how much communication happens at each one of these steps. So for the vast majority of the progress reports, things look good, and we generate an NGAs(notice of grant approval) with the next year's funding on the SEED grants.

But what I'm showing you in the middle is what happens when we receive a progress report that's not satisfactory. So we did receive some where we saw that the projects were not advancing. They were slow. The vast majority were those gold stars that, you know, went right back for the second year of funding.

In general, I have to say it was slow progress. It wasn't bad progress. And it allowed us to identify some issues that are, I think, endemic with a new agency and new ideas. institutions had trouble getting lines for their investigators. People had trouble hiring post docs who were able to do the work. We had trouble getting some of the NGAs out the door for various reasons. So right away, by having a communication with the PI after the progress report was in, we could help them. We could intervene and make the right calls to try to get and kick start these programs.

When there was insufficient data for us to judge how much work had been done, the science officer would request supplemental data. That often required a couple of phone calls and a couple of exchanges of e-mail because, again, we were interested in hearing what people would normally not send in as a progress report: difficulty getting cell lines grown, difficulty doing certain kinds of experiments so that we could see common features across our SEED grantees and allow them to help each other.

When the supplemental data suggested that there was still insufficient progress, I would look at the report, and we also had discussions with the entire science office. At that point, if we couldn't come to a way to jump start a project, we would have a conference call with the PI, and at this point we'd bring in the institutional official as well, the science officer who has this PI in their portfolio, and I, and we would have another call to try to get this program back on track. And very often what that meant was that we made a plan with the investigator to give them some more time so that they could generate some data and try to pick up where the progress was slow. So the time differed depending on the problems that were there, hiring problems, for example. If after this time period another supplemental progress report comes in and there was inadequate progress, we decided that we would notify the investigator that the project just didn't seem to be going anywhere, that there was no real plan to get it back on track and that there was a potential for termination.

Again, the AOOs were all involved in this as well. and if there was no response within two weeks to that potential termination letter, then the grant was terminated and the second year's funding was not advanced.

So what did we learn from this process? We learned that it's critical for us to be working with the PIs to keep the grants on track and how appreciative the PIs are when we do work with them to keep them on track. By the way, we also found several grants that would not have gone on because the investigator was interested in not pursuing the original goal of the research and was going to drop the work. We felt that these areas were so critical for the ideas that were part of the seed program, that we found other investigators who were co-PIs or related to the grant to take over and worked with these new investigators and found mentorship to keep that work going. So it went in both directions. We saw adequate progress where the grants would not have gone on had we not intervened. And I think that this is a very interim report for you because the final success of the SEEDs will be seen over the next year when the final reports come back. We know that papers are going out. We know that research is proceeding apace now; I think much to the effort of each of the individual science officers who worked very closely with the grantees. But it will be important to evaluate how many new labs were brought into human embryonic stem cell and pluripotent stem cell research. And I should also say that a lot of the investigators left to their own devices would have stopped what they were doing with their SEED grants and simply gone on to derive ips cells, and we would have had no portfolio had we (not) been actively managing the grants. But most importantly, the success of this program will be determined on how these investigators go into others of our programs and other large-scale funded grants with the work that was developed from the SEED. And we already have success in that area. We're seeing that one SEED grantee continued on and got an early translation award from CIRM last month.

So overall we've had enormous success, I think, with the SEED program, and we're still in the middle of it. And we've learned a lot about process that will help us to be managing larger scale projects and to work with our investigators in a really positive way.

CIRM Chairman Robert Klein: So, Dr. Csete, will you remind us the number of SEED grants originally awarded?

Csete: Seventy-four.

Klein: So 74 grants, and we've seen 62 or 63 papers at this point.

Dr. Csete: Sixty-four papers.

Klein: So a very high level of productivity. Thank you very much.
Dr. Bloom?

Director Floyd Bloom: This kind of nurturing interactive relationship with the PIs is absolutely unique in the grant world. And so I think it's a wonderful thing that you've instituted. It's going to be a tremendous amount of additional work on your staff to be able to do that, but it's highly commendable, and it's going to make the difference between success or failure, particularly for these intermediate level of successful early experiments where they have to be encouraged to go on and push. So it's a wonderful thing you've done.

Director Ricardo Azziz: I just want to echo that, for starters. I think it takes a tremendous amount of work to help these investigators forward. Again, presumably, they are also very appreciative of your efforts. of the 74 applications, how many – you spoke about the process that you are going through -- how many have been terminated for nonproductivity?

Csete: Three.

Azziz: Three of the 74. Thank you.

Klein: Okay. any additional board comment? Thank you very much, Dr. Csete.

No comments:

Post a Comment

Search This Blog