Integrated assessment models are now widely used to evaluate the economic implications of climate mitigation policies. Given the wide range of estimates they generate, meta-analysis of modelling results is important for a robust assessment of the costs of climate protection. However, since only a subset of models is able to simulate the most stringent climate stabilisation scenarios, there is a risk that selection bias can plague the estimates of the more difficult cases.
This is best illustrated with the Summary for Policy Makers of the Fourth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC 2007). Emission reduction costs are reported for three alternative targets. While 118 results were available for targets in the range of 3.2 to 4.0°C warming, there were only 6 estimates of the costs of keeping warming in the range of 2.0 to 2.4°C. According to the reported results, moving from a 3.6°C to a 3.0°C target would double the abatement costs. Moving from a 3.0°C to a 2.2°C target would increase costs by 37.5% only. As the underlying models are correctly specified – that is, convexity in abatement costs means that stringent targets have higher and accelerating costs – the IPCC results can only be explained by selection bias; only models with low emission reduction costs reported results for the most stringent targets.
In a recent paper (Tavoni and Tol, 2009) we use data results from a new model comparison led by the Energy Modelling Forum (Clarke et al. 2009) to shed light on the issue of selection bias. In this model exercise, ten models ran ten climate scenarios, though some scenarios could not be run by some models. In particular, the more ambitious the scenarios, the fewer are the observations, since more and more models find them unattainable.
Since the values are not missing at random, a model for filling in the missing observations is needed in order to correct for the bias. Using a simple OLS regression one can predict policy costs for those models that were unable to run the more stringent scenarios, thus addressing the selection bias issue. Figure 1 compares the original dataset with the one augmented with predictions. It shows that correcting for selection bias leads to a significant upward revision of the estimates of macro-economic implications of stringent climate policies, and of their uncertainty. Supplementing the data with our predicted values generates a much wider range of estimates.
Figure 1. Mean policy costs for the original EMF22 data set (‘observed’) and the one where missing values have been predicted (‘predicted and observed’)
Note: The horizontal axis reports progressively more ambitious climate scenarios. Bars show 95% percentiles.
The climate targets set forward by the EU and the G8 require significant mitigation of greenhouse gases. Yet, the economic assessment of stringent climate policies has been partial and potentially biased.
Given the various ways integrated assessment models can differ from one another, comparison exercises are particularly important to identify robust findings across model specification, and are indeed at the heart of the reviewing work of the IPCC. The IPCC AR4 suggested that caution was needed when interpreting the results of the more stringent climate policies, as a slim number of studies had been carried out at the time. However, approaches more formal than general warnings are needed when dealing with policy relevant issues such as the costs of climate protection. This is especially important when communicating uncertainties, which are easily lost in the executive summaries.
Clarke, L., J.Edmonds, V.Krey, R.Richels, S.Rose, and M.Tavoni (2009), 'International climate policy architectures: Overview of the EMF 22 international scenarios', Energy Economics, 31, (S2), p. S64-S81.
IPCC (2007), 'Summary for Policymakers', in Climate Change 2007: Mitigation -- Contribution of Working Group III to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change, B. Metz et al. (eds.), Cambridge University Press, Cambridge.
Tavoni, M., and R.S.J. Tol (2009) “Counting only the hits: the risk of underestimating the costs of stringent climate policies”, ESRI working paper No. 324