Big Science or Little Science?

Study shows highly funded elite researchers may not have greater impact

A new study takes a quantitative look at the efficiency of research funding, in an attempt to answer the question: Is it more effective to fund numerous, moderately sized grants, or to put the money into fewer elite researchers? It depends on the goal, says David Currie of the University of Ottawa, but the idea that more money leads to more and more impact is not at all a given.

A new study takes a quantitative look at the efficiency of research funding, in an attempt to answer the question: Is it more effective to fund numerous, moderately sized grants, or to put the money into fewer elite researchers? It depends on the goal, says David Currie of the University of Ottawa, but the idea that more money leads to more and more impact is not at all a given.

This article also exists in French ("Aux grands maux les grands remèdes ? Quid du financement de la recherche ?"), translated by Timothée Froelich.

Following announcements like President Obama’s decision to support the BRAIN initiative, a $100 million neuroscience project, or the similarly ambitious European Human Brain Project, the question of how to most effectively fund science is sometimes raised. Different models exist, from these “mega” team projects, to putting a large amount of money into a small number of top researchers, to distributing grants more broadly among all researchers.  In Canada, the Natural Sciences and Engineering Research Council (NSERC) provides funding for most of the nation’s university scientists. “They used to fund many modest grants,” says David Currie, a macroecologist at the University of Ottawa. “Now they’re talking about ‘fostering excellence’.” Concentrating research funds in relatively few hands is increasingly becoming the funding style of the National Science Foundation in the US, as well as in France. In light of this trend, Professor Currie wanted to know which is more effective in terms of scientific impact.

In a recent study published in PLOS ONE (available on MyScienceWork), Professor Currie and his team examined Canadian research funds awarded by the NSERC in three disciplines: Integrative Animal Biology, Inorganic & Organic Chemistry and Evolution & Ecology. Starting with grant recipients in 2002, they tracked the funding and impact of these researchers over two four-year periods (2002-2006 and 2006-2010).

Measuring impact is another subject of much debate in the scientific community, but the metrics used to evaluate individual researchers in this study were fairly classic: number of peer-reviewed publications, number of citations for these publications, number of citations of the grantee’s most highly cited paper (“since producing a single prominent study might be more highly valued by NSERC than many small studies”), and the number of very highly cited papers published by the grantee (above a threshold number of citations).

Currie et al. analyzed the relationship between grant money received and scientific outcome, and found that the impact of research does increase with funding. However, “the impact increases more slowly than the funding—that is, doubling the money does not double the impact,” David Currie says. “Two researchers receiving moderate grants have more impact than one who is funded with a grant twice as big.” The connection is highly variable and the numbers showed that the most highly cited article of one “big” researcher should get 14% fewer citations than the best article of any random pair of two a mid-level scientists. Similarly, two small grants were shown to yield 20% more high impact articles than a single grant that is twice as big.

The study also took into account researchers’ other sources of funding and found that, surprisingly, additional grants on top of a NSERC funds, even much bigger ones, did not increase individual productivity at all. Currie acknowledges that this is probably not a fair comparison to make across fields (some research being much more expensive to carry out), but within fields most research will stay close to what are considered “normal costs”.

Smaller projects for the greater good

Although their findings seem to show that there is “no synergistic effect to putting a lot of money in the hands of only one researcher”, David Currie acknowledges that flagship projects, like BRAIN or the Human Genome Project, can also be a valid way of structuring research. “If there’s a specific goal that requires a lot of resources—for example, when JFK vowed to put a man on the moon—it may make sense to put a lot of resources on that problem. But if your goal is to maximize the overall productivity of the community, it’s better to spread the money out.”

Today, the scientific machine is producing more PhDs than university jobs, so those who do enter the system are generally really good, David Currie argues. His team’s results suggest that productivity would be increased by allowing more of these highly qualified scientists to carry out their research. In addition, the authors compare the benefit of funding a diversity of researchers to that of genetic diversity: it “increases the probability that some researcher (like some genetic mutant) will possess characteristics that will flourish in an unpredictable future.”

“Science typically goes to places where we don’t know what the impact will be,” Professor Currie points out. As research leads us into realms where we can’t predict its applications, it will likely be more efficient to control the proliferation of mega-projects and let many smaller ones bloom.