Recent developments in the social sciences have demonstrated that we cannot uncritically aggregate the published research on a particular effect to conclude about its presence or absence. Instead, questionable research practices such as p-hacking (conducting additional analyses or collecting new data to obtain significant results) and selective publication of significant results can produce a body of published research that misleads readers even if it contains many significant results. It is, therefore, necessary to assess the evidential value of the research on a certain effect, i.e., one must rule out that it is the result of questionable research practices. We introduce the p-curve method to public administration research and apply it to the research on the relationship between public service motivation (PSM) and individual performance, in order to demonstrate how the evidential value of a body of published research can be assessed. We find that this particular literature contains evidential value.

P-Hacking, P-Curves, and the PSM–Performance Relationship: Is there evidential value? / Vogel, Dominik; Homberg, Fabian Kurt Falk. - In: PUBLIC ADMINISTRATION REVIEW. - ISSN 1540-6210. - 81:2(2021), pp. 191-204. [10.1111/puar.13273]

P-Hacking, P-Curves, and the PSM–Performance Relationship: Is there evidential value?

Fabian Homberg
2021

Abstract

Recent developments in the social sciences have demonstrated that we cannot uncritically aggregate the published research on a particular effect to conclude about its presence or absence. Instead, questionable research practices such as p-hacking (conducting additional analyses or collecting new data to obtain significant results) and selective publication of significant results can produce a body of published research that misleads readers even if it contains many significant results. It is, therefore, necessary to assess the evidential value of the research on a certain effect, i.e., one must rule out that it is the result of questionable research practices. We introduce the p-curve method to public administration research and apply it to the research on the relationship between public service motivation (PSM) and individual performance, in order to demonstrate how the evidential value of a body of published research can be assessed. We find that this particular literature contains evidential value.
p-curve
File in questo prodotto:
File Dimensione Formato  
Vogel & Homberg 2020 PAR - AAM version puar.13273.pdf

Open Access

Descrizione: AAM version
Tipologia: Documento in Pre-print
Licenza: Creative commons
Dimensione 3.81 MB
Formato Adobe PDF
3.81 MB Adobe PDF Visualizza/Apri
Vogel D. & Homberg - P-Hacking PAR- early view.pdf

Open Access

Descrizione: online first version
Tipologia: Documento in Pre-print
Licenza: Creative commons
Dimensione 1.34 MB
Formato Adobe PDF
1.34 MB Adobe PDF Visualizza/Apri
Vogel D. & Homberg - P-Hacking PAR- published - open access.pdf

Open Access

Descrizione: p-curve - final version OA
Tipologia: Versione dell'editore
Licenza: Creative commons
Dimensione 934.52 kB
Formato Adobe PDF
934.52 kB Adobe PDF Visualizza/Apri
Pubblicazioni consigliate

Caricamento pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: http://hdl.handle.net/11385/196064
Citazioni
  • Scopus 7
  • ???jsp.display-item.citation.isi??? 8
social impact