The Evaluation Process - the know and the don’t knows

Starting with the obvious. 

The proposal is assessed individually by experts who are not employees of the Commission but rather third party experts engaged with the commission on a particular contract for the duration of that call topic evaluation. Depending on the call, this could be anything from 3 to upto 5 (possibly more) experts, each of whom will independently read the application, analyse the project and assess the quality of the Open Science methodology. And yes. They will individually produce a report which is then followed by a consensus in Brussels where experts agree or disagree on certain aspects (including the Open Science element) to produce a final score.

The Score Card

A comfort zone to be avoided  

Much like the code of points for a gymnast at the Olympics (perhaps with the exemption of having a rulebook that defines the scoring system), evaluators have to agree on a score between 0-15. Seeking solace in a 10 or 11 simply because it is above the “technical threshold” should be dodged especially in the case of highly competitive calls where the funding threshold hovers around the 13-14 (potentially more) mark. All it takes is a downgrade by half a point on a particular detail, say lack of conviction behind the Open Science methodology, and now you are no longer eligible for funding. As Xavier says “details are extremely important because they are not details in fact they are fundamentals in any proposal”.

Proposal development process

The I see you see of your proposal.

Rule #1 Create a process. 

Consortium partners, when well balanced, include RTOs, academic partners, research centers and industry partners. With this balance comes a divergence in research priorities, allocation of time and views on elements including open science. All it takes is a new person that has never worked on new projects and before you know, there is a misalignment with the enterprise and your project. To avoid being swayed by one large partner at the latter stage of the process, Xavier advises submitters to resolve probable conflict of interests early on and establish a common vision at the very first consortium call. A transparent discussion on the vision of your project, the innovation tracks and most definitely the Open Science methodology (regarding open publications as well as the FAIR data management) should be established early on. Even in the case of mono applicant programs, like the EIC Accelerator, the Data Management Plan has to be a part of the application. While Open Science dissemination, in its many flavors for many stakeholders, is not up for democracy, when it comes to a proposal evaluation, it is best to preestablish the open science protocols during the consortium, notably on who the final decision maker (typically the coordinator in charge of the technical activities) is, should there be a misalignment of interests. 

Key Impact Pathways

The hidden door to success 

Now for some tricky bit of overlooked intel. 

The evaluation criteria is mostly found in sections 1 and 3 and it is easy to assume that the Key Impact Pathways (found in section 2) is not directly related to Open Science. This is however a poor interpretation, says Xavier, that could severely affect proposal scoring. In practice, it is very much related to Open Science since one of the impact pathways is in fact scientific impact and naturally Open Science is a catalyst in that effect. Including Open Science methodology in section 2 as part of the Key Impact Pathways will work in favour of the proposal. 

The Open Science Element 

Doing it right

Cramming 65 pages of text into 45 pages is the new game. Not opting to include evolution criteria in implementation (closely monitored by the consortium and the commission) could question your expertise for Open Science warns Xavier.  The key takeaway here is to capture that in the dedicated pages for Open Science and Data Management Plan in your proposal. Highlight the team willing and able to take the lead on this. Not to be forgotten is listing the Open Access publications and the FAIR data the project will generate as a part of the deliverables in the work program. 

Another noteworthy pointer according to Xavier is more often than not, submitters forget or underestimate the items classified in the new Form A, and end up listing relevant publications (with persistent identifiers) of the scientists involved in the project and do so without paying attention to whether the said publications are open or closed, dampening their credibility in the Open Science element. Track record is key.

Gender Balance

The tie breaker

Yes, excellence and impact remains the main evaluative criterias, BUT where there is a tie in the final score, gender balance in research groups will swing it. Even if it is not Open Science related, this criteria can easily dock off half a point if its three dimensions are not met with. 

1. As of 2022, any RTO or public body, should have a gender equality plan in place. This is essentially a formal document signed by the management and published on their website disclosing their gender equality plan. 

2. Not to be confused with gender balance, but often is, the gender dimension in a proposal is NOT an opportunity to rave about a team that is fully balanced but rather the relevance of integrating sex and gender analysis within the research and an alayze of its impact. “It is more about analyzing the impact of research and the relations towards sex and gender”, advices Xavier.

3. And finally, gender balance, the final of the very final tie breaker. The balance between men and women in the team will be the determining factor and the key pointer according to Xavier here is to not to overlook anyone and everyone, from the research team to the marketing specialist in charge of the simulation expectation of the project should be included in the listing. 

Advice from an expert

400 man hours.

That is what you are looking at going into a proposal submission. First timers without the luxury of seeking assistance from expert consultants best consider joining the consortium with experienced coordinators. Start early on bearing in mind to account for a minimum of six months (three months to build the consortium + three months to read the proposal). Xavier Aubry shares more detailed tips and pointers towards building a successful proposal here with access to slides. 

Mr. Xavier Aubry is the  Founder & Managing Partner at Zaz Ventures. 

Zaz Ventures is one of the most active consulting firms in Europe on the Horizon 2020 program. My. Aubry has been involved in EC funded research and innovation projects for over 15 years. Under his tenure, Zaz Ventures has helped over 200 deep tech innovators to raise over €450M of funding to bring their technologies to market. He has personally been involved in supporting 60 EIC Accelerator interviews since 2018, resulting in 31 companies awarded EIC Accelerator funding.