Due to its quantitative nature, bibliometrics is becoming increasingly popular among policy makers for academic hiring and career promotions. In this article, we quantitatively assess the impact that the granularity level in the classification of scientific areas would entail on research evaluation based on bibliometric indicators. We use as a case study the Italian national habilitation system (ASN), which classifies faculty members according to their academic discipline and relies on journal counts, citations, and h-indices as a basis for promoting tenure track researchers to associate professors and associate to full professors. The assessment checks whether the individual indicators of a researcher are above a certain threshold, e.g., the median over the population of researchers working in the same discipline. Our investigation focuses on two related, rather broad disciplines: computer science and computer engineering. We show that the ASN practice of using the same thresholds for all members of a scientific discipline can favor certain sub-communities that are characterized by higher bibliometric indicators, and disfavor others. We report evidence that up to 30% of Italian faculty members of certain sub-communities would see their indicators drop below the threshold, thus becoming not eligible for promotion, if the ASN were conducted on a more accurate, fine-grained classification. Conversely, in the same scenario, up to 11% of faculty members, in different sub-communities, would see their indicators rise above the threshold, granting them eligibility. Our data set includes 1685 authors, 89,185 distinct publications, and 262,286 author-publication pairs.

On bibliometrics in academic promotions: a case study in computer science and engineering in Italy / Demetrescu, C.; Finocchi, Irene; Ribichini, A.; Schaerf, M.. - In: SCIENTOMETRICS. - ISSN 0138-9130. - 124:3(2020), pp. 2207-2228. [10.1007/s11192-020-03548-9]

On bibliometrics in academic promotions: a case study in computer science and engineering in Italy

Finocchi I.
;
2020

Abstract

Due to its quantitative nature, bibliometrics is becoming increasingly popular among policy makers for academic hiring and career promotions. In this article, we quantitatively assess the impact that the granularity level in the classification of scientific areas would entail on research evaluation based on bibliometric indicators. We use as a case study the Italian national habilitation system (ASN), which classifies faculty members according to their academic discipline and relies on journal counts, citations, and h-indices as a basis for promoting tenure track researchers to associate professors and associate to full professors. The assessment checks whether the individual indicators of a researcher are above a certain threshold, e.g., the median over the population of researchers working in the same discipline. Our investigation focuses on two related, rather broad disciplines: computer science and computer engineering. We show that the ASN practice of using the same thresholds for all members of a scientific discipline can favor certain sub-communities that are characterized by higher bibliometric indicators, and disfavor others. We report evidence that up to 30% of Italian faculty members of certain sub-communities would see their indicators drop below the threshold, thus becoming not eligible for promotion, if the ASN were conducted on a more accurate, fine-grained classification. Conversely, in the same scenario, up to 11% of faculty members, in different sub-communities, would see their indicators rise above the threshold, granting them eligibility. Our data set includes 1685 authors, 89,185 distinct publications, and 262,286 author-publication pairs.
2020
Academic recruitment; Bibliometrics; Citations; H-index; Research productivity
On bibliometrics in academic promotions: a case study in computer science and engineering in Italy / Demetrescu, C.; Finocchi, Irene; Ribichini, A.; Schaerf, M.. - In: SCIENTOMETRICS. - ISSN 0138-9130. - 124:3(2020), pp. 2207-2228. [10.1007/s11192-020-03548-9]
File in questo prodotto:
File Dimensione Formato  
Demetrescu2020_Article_OnBibliometricsInAcademicPromo.pdf

Solo gestori archivio

Tipologia: Versione dell'editore
Licenza: DRM (Digital rights management) non definiti
Dimensione 2.07 MB
Formato Adobe PDF
2.07 MB Adobe PDF   Visualizza/Apri
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11385/200615
Citazioni
  • Scopus 10
  • ???jsp.display-item.citation.isi??? 10
social impact