Discussion Paper Details

Please find the details for DP16751 in an easy to copy and paste format below:

Full Details   |   Bibliographic Reference

Full Details

Title: Non-Standard Errors

Author(s): Anna Dreber, Felix Holzmeister, Juergen Huber, Magnus Johannesson, Michael Kirchler, Albert Menkveld, Sebastian NeusŁess, Michael Razen, Utz Weitzel and Christian C Wolff

Publication Date: November 2021


Programme Area(s): Financial Economics

Abstract: In statistics, samples are drawn from a population in a data generating process (DGP). Standard errors measure the uncertainty in sample estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence generating process (EGP). We claim that EGP variation across researchers adds uncertainty: non-standard errors. To study them, we let 164 teams test six hypotheses on the same sample. We find that non-standard errors are sizeable, on par with standard errors. Their size (i) co-varies only weakly with team merits, reproducibility, or peer rating, (ii) declines significantly after peer-feedback, and (iii) is underestimated by participants.

For full details and related downloads, please visit:

Bibliographic Reference

Dreber, A, Holzmeister, F, Huber, J, Johannesson, M, Kirchler, M, Menkveld, A, NeusŁess, S, Razen, M, Weitzel, U and Wolff, C. 2021. 'Non-Standard Errors'. London, Centre for Economic Policy Research.