The usage of open infrastructures for alternative assessing the quality of research outputs
Decision-makers are not able to follow, as fast as needed, all the changes in the OS (Open Science) on the policy level. This affects the way how research outputs are assessed. Due to this, not all research outputs are recognised, and there are no mechanisms to trace and appraise all outputs. This paper presents open infrastructures and supporting methods which are capable of overcoming the mentioned deficiencies. The Scopus and Web of Science citation indexes have a significant impact on the quality assessment of research outputs, especially when it comes to research promotion and career development. At the same time, researchers produce scientific outputs that cannot be assessed using traditional approaches[1].
Open infrastructures can support research communities by providing methods which are verifiable and available for integration in institutional infrastructures (repositories, CRIS systems, etc.). There are also some services that do not fall into the category of open infrastructure, but they support the concept by providing technical solution that can be open accessed. Also, there is tendency to make open infrastructures as much as possible compliant with OS principles. Some of these infrastructures implement other closely related principles such as FAIR data principles[2]. When an institutional infrastructure adopts the mentioned principles and integrates services provided by open infrastructures, then benefits are seen by both researches and decision-makers.
Dimensions is an alternative to traditional citation database. Thanks to functionalities available through various open APIs, Dimensions has played an important part in the adoption of OS principles on the technical in Serbia. The Dimension database is considered to be more open and inclusive than traditional database. Also, Dimensions is integrated with other infrastructures that also follow the concept of openness[3].
Alternative metrics, like Altmetric, could serve as a possible solution[4]. Like Dimensons, Altmetric is not an open infrastructure but it contributes to open infrastructures through open APIs that can track the impact of research outputs across the Internet. The main focus is on following the presence of research papers on social media (Facebook, Twitter), multimedia platforms (YouTube) or just tracking mentions in Internet communities. An important issue for each country is how to assess research outputs in humanities. Knowing that research outputs in this area focus mainly on locally relevant topics, it is difficult to track the impact of such results by relying on traditional citation databases.
Unlike Dimension and Altmetric, OpenCitations can be defined as an open infrastructure which fully follows and adopts the concept of OS. Also, it is compliant to the FAIR data principles. Accordingly, the collected data are findable, accessible, interoperable and re-usable. Furthermore, OpenCitations adopts POSI (the Principles of Open Scholarly Infrastructure) [5]. The infrastructure has been upgraded with a system that provides persistent identifiers (PIDs)[6].
All mentioned solutions are mostly dealing with citations. There are some research outputs, like peer-review reports and editorial work, that could not be assessed only by the number of citations. Open infrastructure should provide metrics for those research outputs. Publons is far from an open infrastructure but it provides at least some openly accessible services for researchers.
The recently launched OpenAlex is highly relevant in this context. Beside potential uses in assessment, OpenAlex can serve as a search engine, research output monitor, etc. OpenAlex aggregates data from many sources, like Crossref and ORCID, and seeks to present data in a standardised way.
Creating a new methodology in assessing research outputs is not an easy task[7], but communities recognise the need to improve traditional methods[8]. The main goal of this paper is to show new possibilities in assessing research outputs and to point out alternatives to traditional assessment methods.
Vladimir Otašević
University of Belgrade, School of Electrical Engineering
Department of Computer Science and Information Technology
Belgrade, Serbia
ORCID ID: 0000-0002-9408-3454
