The World Bank Group’s Doing Business data collection and ranking exercise is again in the headlines, allegedly following China’s protest against it being ranked 91. Using the occasion of the 10th anniversary of Doing Business, the World Bank’s new president has formed a commission in October 2012 to assess impact, methodology and effectiveness of the Doing Business project.1 The commission is scheduled to issue its report in the next few days. It’s not the first time the project has come under intense scrutiny: the World Bank’s Independent Evaluation Group issued a report in 2008 criticising specific aspects of measurement and making recommendations for improvement.
The debate on Doing Business has been largely driven by its impact on policymaking in developing countries. It has also been used extensively by researchers interested in whether and, if so, which institutions help economies grow faster, a critical input into the policymaking process across both the developed and developing world. Doing Business indicators are tools for empirical researchers and as such they are subject to shortcomings and criticisms.
How did we get here?
15 years ago, researchers trying to study the importance and effectiveness of ‘institutions’ in developing countries were mostly limited to survey-based measures from Business Environmental Risk Intelligence (BERI) and International Country Risk Guide (ICRG). These typically captured very broad concepts, such as rule of law or political stability.
Enter the Doing Business project. Focusing either on specific institutions (e.g. credit registries) or specific cases (enforcement of bounced check, registering a property of specific value), this effort provided a benchmarking tool for cross-country analysis of specific dimensions of the business environment. The database allowed researchers to use consistent indicators of specific institutions across countries to both gauge the determinants of institutional frameworks and their effects.
The indicators are objective, constructed in a transparent manner, and can be linked to specific policy actions. Capturing a variety of different dimensions, they also allow ressearchers to disentangle different dimensions of the institutional framework. While initially only cross-sectional, the updating of the database over time creates and adds another useful dimension that can be used for cross-country panel analysis.
The icing on the cake
Beyond providing a cross-country benchmarking tool, the Doing Business project went a step further by publishing rankings, both for individual indicators as well as an aggregate indicator of the business environment in a country. In its annual report, it highlighted countries that improved their ranking significantly due to policy reforms. Some of these countries, such as Georgia, in turn, used the ‘Reformer of the Year’ award to attract foreign direct investment.
This move from data collection to country rankings and their use in policy debate has given the Doing Business project more prominence, but has also triggered a fierce debate. Advocates point to the sound academic foundation for their indicators and rankings and to have the use of rankings in fostering policy debate and reform in many developing countries. Rankings attracted the attention of the non-academic public and policymakers alike. And few economists would argue against cutting red tape, at least as a general principle.
Critics point to several pitfalls in these rankings:
- The rankings are relative.
If a country improves its business environment, it will only move up the ranking if it improves more than other countries. Similarly, a country can keep its rank or even improve it if other countries fall behind in their business environment.
- Such rankings assume a cardinal relationship where there is at most an ordinal one.
Is the difference between rank 10 and 20 the same as that between rank 100 and 110? Is the difference between rank 40 and 60 twice that of the difference between rank 40 and 50 or 50 and 60? And is there really a significant difference between a country ranked 59 and a country ranked 60?
There are also more fundamental questions about the relevance of these indicators:
- Are the dimensions captured by the Doing Business ranking really the relevant ones for a large share of the enterprises in a country?
Just to quote one example, Hallward-Driemeier, Khun-Jush and Pritchett (2010) show for a sample of 34 countries in sub-Saharan Africa, the de facto implementation of rules and policies varies more within countries and across firms than across countries.
Linking improvements in Doing Business to economic growth is tricky as it is hard to control for omitted variables. Can we really claim that high growth in Rwanda and in Mauritius is exclusively due to the Doing Business reforms these countries undertook (Rohac and Tupic 2013)? A broad cross-country comparison shows that general indicators of the risk of government expropriation are more robustly related to levels of economic development than specific measures of the business environment, such as contract enforcement (Acemoglu and Johnson 2005).
Back to business – the indicators
Most critiques of the Doing Business project refer to the ranking part. However, when policymakers favour or condemn a specific indicator – most prominently labour market regulations and business taxation – often based on certain priors, empirical researchers see a chance for hypothesis testing:
- Are less restrictive labour regulations welfare improving or not?
- Is their effect conditional on other country circumstances?
- Does the effect vary with the level of development and across the business cycle?
Having detailed cross-country data on labour-market regulation is an important starting point for such hypothesis testing. Researchers do not primarily care about rankings, but accurate data. Their findings, however, can be an important input for country-tailored policy reform!
Cross-country indicators, such as the Doing Business variables, are open to the usual critique against cross-country work. As recognised explicitly by the project, the variables capture the letter of the law (ie. institutions and rules), not how they function in practice. As already discussed, they might not capture the most relevant dimensions of the business environment in specific countries. If 60% of enterprises do not have access to formal credit, the efficiency of the credit registry might not be relevant. If the majority of enterprises are not formal, they do not have access to the formal court system for contract enforcement. But there is a fallacy in this argument, as it refers to the existing enterprise population and structure and not to the potential one that might exist under a more effective institutional framework.
The recent literature has therefore moved beyond cross-country work and focused more on the assessment of country-specific institutional reforms, exploiting either sub-national variation in the business environment (e.g. labour-market regulation across Indian states, Besley and Burgess, 2004) or country-specific policy reforms (e.g. Bruhn 2013 for Mexico, or Gine and Love 2010 for Colombia).2 Country-specific analysis allows better to control for omitted variables, such as cultural traits, and measurement biases. On the other hand, the external validity of country-specific analyses might be limited. It is interesting to note that many of the country-specific analyses are consistent with the Doing Business philosophy, notwithstanding country-specific analyses of China and India that point to the effective functioning of alternative business structures (Allen et al. 2005, 2006).
Which brings us back to the usefulness of the Doing Business project as one critical component of the overall assessment of institutional frameworks rather than the exclusive source of assessment. An assessment of the business environment faced by enterprises in a specific country would involve cross-country comparisons of Doing Business indicators, firm-level assessments to thus gauge the relevance of different growth obstacles for firms of different size, ownership and sectors (such as provided by the Enterprise Surveys, another product of the World Bank Group) and country-specific analysis (quantitative and qualitative) of the effective functioning of specific institutions in the country, such as property and credit registries, courts and alternative dispute mechanisms. Only by bringing together these different data sources and levels of analysis can economists gain a proper understanding of a country’s business environment, the bottlenecks it might pose for private-sector growth in the country and the policy reforms that might help overcome these bottlenecks.
Less icing, more cake! As outlined above, the rankings provided by the Doing Business project are questionable. The underlying indicators are, however, more important than the rankings. In spite of all its shortcomings, a characteristic it shares with all other cross-country institutional indicators, Doing Business has become a critical source of information on institutional frameworks across countries. Researchers do not need rankings! But we need data. And we therefore need Doing Business! A database, which will hopefully be expanded and improved over time, both in its accuracy, its relevance and its comprehensiveness. With less focus on rankings, Doing Business might not capture as many newspaper headlines, but it will be just as important an input in to the policymaking process in developing and developed countries that all economists care about.
Acemoglu, Daron, and Simon Johnson (2005), “Unbundling Institutions”, Journal of Political Economy 113, 949–995.
Allen, Franklin, Jun Qian and Meijin Qian (2005), “Law, Finance and Economic Growth in China”, Journal of Financial Economics 77, 57-116.
Allen, Franklin, Rajesh Chakrabarti, Sankar De, Jun Qian and Meijin Qian (2006), “Financing firms in India”, World Bank Policy Research Working Paper 3975.
Beck, Thorsten (2012), “Legal Institutions and Economic Development”, in Dennis Müller (ed.) Oxford Handbook of Capitalism, Oxford University Press.
Besley, T, and R Burgess (2004), “Can Labor Regulation Hinder Economic Performance: Evidence from India”, Quarterly Journal of Economics 119, 91-134.
Bruhn, Miriam (2013), “A Tale of Two Species: Revisiting the Effect of Registration Reform on Informal Business Owners in Mexico”, Journal of Development Economics 103, 275-83.
Gine, Xavi and Inessa Love (2010), “Do Reorganization Costs Matter for Efficiency? Evidence from a Bankruptcy Reform in Colombia”, Journal of Law and Economics 53, 833-64.
Hallward-Driemeier, Mary, Greta Khan-Jush and Lant Pritchett (2010), “Deals versus Rules”, World Bank Policy Research Working Paper 5321
Independent Evaluation Group, The World Bank (2008), “Doing Business: An Independent Evaluation”.
Rohac, Dalibor and Marian Tupy (2013), “It’s not Business as Usual at the World Bank”, Foreign Policy.
World Bank (2008), “Doing Business: An Independent Evaluation”, Washington DC.
1 See http://www.worldbank.org/content/dam/Worldbank/Feature%20Story/Independent%20Panel%20-%20Doing%20Business%20-%20FinalToRs.pdf for details.
2 See Beck (2012) for a more comprehensive survey on the empirical literature on institutions and growth.