EU's Agreement on Reforming Research Assessment
The Agreement on Reforming Research Assessment was published on the 27th of July, 2022.
Aalto University has signed the agreement and is therefore a member of the Coalition for Advancing Research Assessment, CoARA. Below is a short overview of the document but we recommend reading the whole Agreement. Please read more about Aalto University's research assessments on this page.
In the context of the Agreement, research assessment encompasses:
• The assessment of research performing organisations and research units, by assessment authorities, research funding and performing organisations
• The assessment of research projects by assessment authorities, research funding and performing organisations, and prize awarding organisations.
• The assessment of individual researchers and research teams by research funding and performing organisations, and prize awarding organisations.
The Agreement focuses on the specific challenges involved in reforming the research assessment of researchers, research projects, research units and research organisations.
The signatories of the agreement agree:
I. To base our actions on the following Principles
Principles for overarching conditions
• Comply with ethics and integrity rules and practices, and ensure that ethics and integrity are the highest priority, never compromised by any counter-incentives.
• Safeguard freedom of scientific research.
• Respect the autonomy of research organisations.
• Ensure independence and transparency of the data, infrastructure and criteria necessary for research assessment and for determining research impacts.
Principles for assessment criteria and processes
• Focus research assessment criteria on quality. Openness of research, and results that are verifiable and reproducible where applicable, strongly contribute to quality. Openness corresponds to early knowledge and data sharing, as well as open collaboration including societal engagement where appropriate. Assessment should rely on qualitative judgement for which peer review is central, supported by responsibly used quantitative indicators where appropriate.
• Recognise the contributions that advance knowledge and the (potential) impact of research results.
• Recognise the diversity of research activities and practices, with a diversity of outputs, and reward early sharing and open collaboration.
• Use assessment criteria and processes that respect the variety of scientific disciplines, research types, as well as research career stages, and that acknowledge multi-, inter-, and trans-disciplinary as well as inter-sectoral approaches, when applicable.
• Acknowledge and valorise the diversity in research roles and careers, including roles outside academia.
• Ensure gender equality, equal opportunities and inclusiveness.
II. Implement the following Commitments
The are two main types of commitments; core commitments (1 – 4) and supporting commitments (5 – 10).
The core commitments enable better recognition of the diverse practices and activities that maximise the quality of research, and move away from inappropriate uses of metrics. The supporting commitments enable the move towards new research assessment criteria, tools and processes, and facilitate mutual learning, communicate progress and ensure that new approaches are evidence informed.
Core commitments
1. Recognise the diversity of contributions to, and careers in, research in accordance with the needs and nature of the research
Changes in assessment practices should recognise the broad diversity of: valuable contributions to science and society; practices that contribute to robust, open, transparent and inclusive research; diverse activities (e.g. teaching, leadership, supervision etc.) and diverse roles (e.g. data stewards, software engineers etc.).
2. Base research assessment primarily on qualitative evaluation for which peer review is central, supported by responsible use of quantitative indicators
Research assessment should rely primarily on qualitative assessment for which peer review is central, supported by responsibly used quantitative indicators where appropriate.
3. Abandon inappropriate uses in research assessment of journal and publication based metrics, in particular inappropriate uses of Journal Impact Factor (JIF) and h-index
Inappropriate uses of journal and publication-based metrics in research assessment should be abandoned. This means moving away from using metrics like the Journal Impact Factor (JIF), Article Influence Score (AIS) and h-index as proxies for quality and impact.
4. Avoid the use of rankings of research organisations in research assessment
Recognising that the international rankings most often referred to by research organisations are currently not ‘fair and responsible’ , the criteria these rankings use should not trickle down to the evaluation of individual researchers, research teams and research units. Research organisations should regain the autonomy to shape assessment practices, rather than having to abide by criteria and methodologies set by external commercial companies.
Supporting commitments
5. Commit resources to reforming research assessment as is needed to achieve organisational changes.
6. Review and develop research assessment criteria, tools and processes.
7. Raise awareness of research assessment reform and provide transparent communication, guidance, and training on assessment criteria and processes as well as their use.
8. Exchange practices and experiences to enable mutual learning within and beyond the Coalition.
9. Communicate progress made on adherence to the Principles and implementation of the Commitments.
10. Evaluate practices, criteria and tools based on solid evidence and the state-of-the-art in research on research, and make data openly available for evidence gathering and research.
Have a look at other recommendations on research and researcher assessment
National recommendations for responsible researcher evaluation
Researcher evaluation shapes and directs research. The entire research community should take responsibility for the principles and practices of researcher evaluation. This recommendation for the responsible evaluation of a researcher provides the basis for a functioning, diverse and flourishing research community.
Declaration of Research Assessment, DORA
There is a need to improve the ways in which the output of scientific research is evaluated by funding agencies, academic institutions, and other parties. Declaration of Research Assessment, DORA, was one of the first initiatives to improve the ways in which researchers and the outputs of scholarly research are evaluated
Leiden Manifest
Leiden Manifest offers best practice and principles on metrics-based research assessment
Go to Aalto University Research assessments page
Research assessments
Aalto University regularly assess the quality of its research and artistic activities, and its societal impact. The assessment is based on internationally agreed best practices and peer review.
- Published:
- Updated: