Discussion Paper Details

Please find the details for DP16099 in an easy to copy and paste format below:

Full Details   |   Bibliographic Reference

Full Details

Title: Understanding Performance in Test Taking: The Role of Question Difficulty Order

Author(s): Lina Anaya, Nagore Iriberri, Pedro Rey Biel and Gema Zamarro

Publication Date: May 2021


Programme Area(s): Public Economics

Abstract: Standardized assessments are widely used to determine access to educational resources with important consequences for later economic outcomes in life. However, many design features of the tests themselves may lead to psychological reactions influencing performance. In particular, the level of difficulty of the earlier questions in a test may affect performance in later questions. How should we order test questions according to their level of difficulty such that test performance offers an accurate assessment of the test taker's aptitudes and knowledge? We conduct a field experiment with about 19,000 participants in collaboration with an online teaching platform where we randomly assign participants to different orders of difficulty and we find that ordering the questions from easiest to most difficult yields the lowest probability to abandon the test, as well as the highest number of correct answers. Consistent results are found exploiting the random variation of difficulty across test booklets in the Programme for International Student Assessment (PISA), a triannual international test, for the years of 2009, 2012, and 2015, providing additional external validity. We conclude that the order of the difficulty of the questions in tests should be considered carefully, in particular when comparing performance between test-takers who have faced different order of questions

For full details and related downloads, please visit:

Bibliographic Reference

Anaya, L, Iriberri, N, Rey Biel, P and Zamarro, G. 2021. 'Understanding Performance in Test Taking: The Role of Question Difficulty Order'. London, Centre for Economic Policy Research.