Anabel Sánchez: "We have published a guide to help research centers understand, articulate and communicate their impact"

CREAF's Impact Officer, Anabel Sánchez, has just published, together with two other international experts, a guide that proposes a change in perspective on how research centers understand, articulate and communicate their impact. The guide, called the Research Institute Impact Narrative Guide, RIING (for its acronym in English) , is published in open access and offers a practical framework to help institutions build solid, evidence-based impact narratives that go beyond scientific results and also incorporate values, culture and the research environment. This approach allows centers to better explain the value they bring to the scientific system and to society, and is particularly useful in contexts such as attracting talent, collaborating with administrations and companies, obtaining competitive funding or institutional evaluations.

Well, when I started with this responsibility within CREAF, I began to see that in our system it was not easy to find someone working on the vision of impact from the institutions with a strategic perspective. There were people in other centers, for example, providing support when preparing project proposals for calls where they ask to justify the impact of the research, but not strategic roles.

Well, one of the first things I did was try to find international networks at a European level that worked on the impact of research. The search for networks, conferences and people gave rise to a network of contacts, which today is very large, and which allows me to feel much more supported in my work. One of the first people I contacted was Dr. Giovanna Lima , who had initially been Impact officer at Trinity College, and then Head of the Impact Program at Erasmus University Rotterdam. Together we did many things, such as training for European research managers on strategic approaches to the impact of research from an institutional point of view.

We saw that we had a shared concern. We noted that many institutions were approaching impact, especially as a result of the demands of European calls, especially Horizon Europe, where impact weighs heavily in the evaluation. But at the same time, this coincided with a deeper change in the way research is evaluated, increasingly oriented towards qualitative narratives, context, values and collective contributions.

This first step opened the door to going beyond compliance with a requirement and thinking about impact as a strategic and institutional learning tool, which helps centers better explain what role they play in the scientific system and in society. It is in this framework that we began to sense the need for a guide like RIING.

In this guide, when we talk about impact from the point of view of a research center, we are referring above all to the relevance and effects that its activities, actions and decisions have on the different environments in which it operates. In other words, impact has to do with what changes, what is strengthened or what is made possible by what an institution does, beyond the fact of having carried out a specific activity. From this perspective, a research center generates impact when it contributes to the generation of knowledge within the academy itself, to the professional development of the people who work there, to contributions to society, within the institution itself, etc.

When we talk about impact, we seek to look back at what we do and ask ourselves “So what?” with each action we take. For example, if tomorrow we want to set up a mentoring program with 20 participants, the question we have to ask ourselves is “Okay, we already have the program, now what?” What should happen after this program? What changes should we observe in the people who have participated? How do we contribute to their careers, to the research fabric, what benefit does the center itself get from it, etc.

Yes, clearly. In this process we also contacted Dr. Fernando Borges , from the University of Coimbra and together we saw that these questions are now also those that are beginning to be asked by the evaluation entities when evaluating centers in the Portuguese research system. We are immersed in a very obvious change of context. The evaluation entities do not leave aside what has been done so far, but they approach it in a different way. A good example is the latest evaluation of the CERCA centers in Catalonia that has already incorporated this change of context and perspective. Instead of focusing mainly on aggregate indicators, it was asked to identify a limited number of key contributions and explain their relevance, why it is relevant, what it has implied and for whom? Not only at a social level, but of all the activity that takes place within a research center. Give me 5 key projects for the entity, yes, but why are these the relevant ones? It is no longer just the number of publications or projects that counts, but why they are relevant.

Yes, at the level of research staff, narrative evaluation models have been worked on for a long time and there are guides or manuals for doing them, such as the Resumes for Researchers (UK), or the Researcher Impact Framework (RIF), which structures or helps to explain in a narrative way the relevance of a researcher's research in different areas of work of a person who does research, not only production, but also the whole diversity of activities that they can do. In this case, it focuses on four main areas: knowledge generation, contributions to society, development of people and collaborations, support for the research community (such as the management of the center, or participation in internal committees).

What we are doing with the RIING is to transfer this same logic to the institutional level. Research centers, like people, are not just their final results: they are environments that create conditions, drive trajectories, sustain communities and make strategic decisions. The guide adapts this narrative approach to help institutions explain, with evidence, why their contributions are relevant, in which areas they are and for whom, also incorporating a fifth key area: institutional sustainability and resilience.

Yes, this change is very clear, although it is not the same for all programs or territories. More and more agencies and calls are incorporating narrative CV formats that ask to explain the relevance of the contributions, the context in which they have been produced and the specific role of the researcher, instead of limiting themselves to listing publications or indicators.

In Spain, for example, ANECA has already introduced the narrative curriculum in accreditation processes. This does not mean that publications or metrics will disappear, but it does call for a responsible use of these indicators, always contextualized and justified, and with a growing emphasis on aspects such as open science, social impact or other relevant contributions.

At European level, both the ERCs and the Marie Skłodowska-Curie Actions already clearly use this narrative approach. In the case of the ERCs, a CV is requested that highlights the most relevant achievements, explaining why they are important, what the leadership of the researcher has been and how these contributions enable them to develop the proposed project. The quantity of results is not valued, but their quality and significance, in line with the principles of DORA, CoARA and the reform of research evaluation.

In Catalonia, programs such as the Beatriu de Pinós grants also explicitly request a narrative CV, focusing on career path, relevance of contributions, professional development, mobility, leadership and good research practices. Again, metrics may appear, but they are not the central focus of the evaluation.

Overall, what we see is a clear trend towards better evaluating the meaning and value of contributions, rather than making comparisons based solely on numbers. And this same change of perspective is what RIING now wants to bring to the scale of research centers.

The guide aims to be a structured framework to help institutions identify, describe and communicate impact understood as relevance in all areas of action. We want to help explain the relevance of what the centre does, thinking about specific actors, such as governments or decision-makers, and to society in general. This guide places great emphasis on the context of your research environment, and on your values. What you want to promote as an institution must be at the basis of how you define your impact and how you organise and structure yourself strategically to make it possible.

What we propose is to work with impact narratives based on evidence, which allow us to explain the relevance of what an institution does in a contextualized way. Quantitative metrics are not eliminated, but they are integrated into the story and used responsibly. For example, if a center organizes a conference, it is not enough to say that it was done: explaining how many organizations participated, where they came from or what came out of it helps to understand why it is relevant. Also that evidence must be taken into account. Evidence is key, it must be sensitive to the context and to whom you are explaining it, for example, if we are responding to a research evaluation, or if we want to explain to the minister the relevance of CREAF, different narratives are used.

The RIING is structured in 5 areas. Four coincide with those used by research staff, but with an important change: it does not look at what people do, but what the institution does to generate impact or relevance, what activities it promotes and what strategies it has. The fifth area is the only one that is clearly institutional: institutional sustainability and resilience.

For each of these areas, the guide proposes a retrospective exercise: thinking about what has been achieved, through which institutional activities it has been made possible and with what evidence its relevance can be demonstrated.

In the case of knowledge generation, the impact can be explained by focusing on the mobility, accessibility and actual use of knowledge. The key question is: what does the institution do to make this knowledge open, usable and reliable?

This is where institutional supports come into play: what tools we make available, what practices we promote, or what policies we have. For example, having an open and interoperable institutional repository makes it easier for knowledge to circulate well beyond the moment of publication.

Evidence allows us to demonstrate whether this really works: documented reuses, downloads, accesses or citations of data. The expected result is that knowledge is more accessible, more reused and more reliable, and the narrative can also show the evolution over time, for example whether these figures have grown.

In the area of people development and collaborations we can explain whether the center participates in the development of diverse careers, or if it promotes a greater breadth of skills. There could be a structured mentoring program at different levels of the research career, also at the level of research management personnel. Evidence could be the internationalization of people, (depending on what each institution wants to achieve), or that the trained people increase their employability in the research system, to give some examples.

A key issue in this area is decision-making informed by our science. For example, CREAF can help third sector environmental entities use robust scientific knowledge in their interactions with public administrations, thus strengthening better-founded decision-making processes.

This use of knowledge can lead to new collaborations, joint projects or a more structured and continuous relationship between social actors, administrations and the research centre. It is important to understand that these are not isolated cases, but pieces of the same puzzle.

When articulated in a coherent narrative, these contributions allow us to clearly show what values guide the institution, what its specific role is in the system, who the beneficiaries are and what real value the center brings to society.

The guide gives specific examples and for each area gives ideas of what could be achieved in that area of impact, associated with each specific area of impact. However, we put a table with activities, evidence and sources where evidence can be found. All for the five areas of impact. In this part we have put a lot of effort to make it understandable, with significant examples and to make it easy. Of course, it must be clear that it is a guide, not a recipe. The institution must reflect on what it wants to promote, and this is a change in the mental framework of the institution.

Well, we would like to have awakened reflection and have achieved two things that are not always done, firstly, thinking about what has not worked and why it has not worked, and secondly, the role of research management in all of this, in the profiles that are not researchers without whom research cannot be done, managers, technicians, support staff, they create the conditions so that research can be done. It would be very interesting if many of the actions to promote the impact of research were done with much greater collaboration, management and research staff, because many of the actions that make institutional impact possible can only really work if they are thought about and implemented jointly.

I have learned a lot, for me it has been a challenge to start thinking at a broad level, at an institutional and strategic level and to land it in this document. It has been very interesting to work with them both and open my eyes to go beyond the impact of research in terms of contributions to society.

Yes, we sent the draft to different people with different profiles at very diverse institutions and a few gave us feedback. We sought contribution from the international impact community and we have acknowledged them in the document. We are all thinking about how to do it, even if we have started writing it ourselves. This reflection is global.

At CREAF we have made a lot of progress, but we need to continue working on it, we still need to develop a more strategic and shared vision. Having this vision would help us to better align everything that is already being done both from the different offices and from the research staff, and to define shared strategic objectives on where we want to go and how we want to impact society or the research system.

Having these shared objectives would allow efforts to not just be the sum of individual initiatives, but for us all to think and act in a coordinated way, directing activities, resources and decisions towards the same vision of institutional impact.