Most of our knowledge about how the economy works is based on what we see people, governments, and firms doing in different situations. In order to test economic theories or estimate economic models, researchers have traditionally analysed data on how firms and households actually behave under the relevant circumstances, letting people’s actions ‘reveal’ their underlying preferences. This ‘revealed preference’ approach is natural – the most accurate way to determine how people respond to an economic policy is to observe how they did in fact respond to that policy. However, there are limits on what we can learn from the revealed preference approach. For many policies, the econometric conditions and type of variation necessary for measuring the effects of policy may not exist, and even when they do, the necessary information about people’s behaviour may not.
Increasingly, and particularly in the area of household finance, economists are attempting to circumvent these limitations by employing a ‘reported preference’ approach. Instead of observing households’ actual behaviour, the reported preference approach identifies people’s choices simply by asking them how they behaved, or how they would behave, in alternative hypothetical scenarios. For example, Bunn et al. (2017) uses responses to a Bank of England survey to determine how households would change their consumption in response to an unexpected change in income. Garziani et al. (2016) performs a similar exercise, using a Federal Reserve Bank of New York survey to gauge households’ response to the 2011 payroll tax cut in the US. Other recent studies employing reported preference methods include Sahm et al. (2010), Japelli and Pistaferri (2014), and Crump et al. (2015).
Reported and revealed responses to the 2008 stimulus payments in the US
The use of reported preference surveys expands the range of questions that economists can study, but may introduce problems of accuracy and reliability. If survey respondents do not remember their past behaviour, are incapable of predicting their behaviour in a hypothetical scenario, do not fully understand the question being asked, or for any reason provide dishonest or inaccurate responses, then reported preference results will be misleading. Despite the increasing popularity of reported preference estimation and its dependence on accurate survey responses, very little research exists investigating the relationship between reported and revealed behaviour. In a recent paper, we evaluate the consistency of reported preference and revealed preference estimates in measuring how much households increased their spending in response to the US federal stimulus payments distributed in 2008 (Parker and Souleles 2017).
The Economic Stimulus Act of 2008 called for over $100 billion in stimulus payments to be distributed to eligible taxpayers in the US. These payments were substantial, averaging over $900 per recipient. The funds were disbursed gradually during May, June, and July 2008, with payment dates largely being determined by the (as-good-as randomly assigned) last two digits of taxpayers’ Social Security numbers.
We worked with two pre-existing consumer surveys to add questions about both the payments received by households and households’ perceptions of how much they spent out of them. The Consumer Expenditure survey (CE), administered by the US Bureau of Labor Statistics, and the Nielsen Consumer Panel (NCP) both collect detailed information about households’ consumption expenditures. Working with the survey administrators, we added questions asking about the date of arrival and size of each household’s stimulus payment. This information, combined with the randomised variation in stimulus payment arrival created by the disbursement scheme, allows us to construct revealed-preference estimates of households’ spending propensities by comparing spending patterns across households that received payments at different times.
Additionally, the questions added to both surveys asked respondents to report whether they used their stimulus payment “mostly to increase spending,” “mostly to increase savings”, or “mostly to pay off debt” (questions that were developed by Shapiro and Slemrod 1995, and are now widely used in other research). And in the NCP survey, we also asked people to report the exact dollar amount that they spent out of the stimulus payment. These questions provide qualitative and quantitative reported-preference measures of spending. The key question is whether these reported spending amounts align well with the revealed preference measures of actual consumption behaviour.
Positive results: Qualitative reports and average spending propensities
Our first finding is that the qualitative reported preference measure of spending is highly informative about households’ revealed preference consumption behaviour. We consistently find substantial differences in revealed preference spending propensities between households that reported using their stimulus payment “mostly to increase spending” and those that reported using the payment “mostly to increase savings” or “mostly to pay off debt.” These differences are generally statistically significant and occur in both the CE and NCP surveys.
Figure 1 illustrates the useful information conveyed by the qualitative spending report. The bars show the average percentage increase in revealed preference consumption (during the three-month period of stimulus payment receipt in the CE; during the month after receipt in the NCP). Households that reported using the payment “mostly to increase spending” had an average revealed-preference spending response that was about three percentage points larger in the CE and about eight percentage points larger in the NCP. We continue to find significantly greater revealed preference spending by people reporting that they mostly spent if we measure the consumption response in dollars or if we measure it as a percentage of the received stimulus payment.
Figure 1 Percent increase in spending by reported main use
Note: See Parker and Souleles (2017). Whiskers display one standard error bands. Source: Authors’ analysis of the CE and NCP. The NCP results are calculated based on data from The Nielsen Company (US) LLC and provided by the Marketing Data Center and the University of Chicago, Booth School of Business.
Qualitative spending reports thus reflect actual spending behaviour, but how accurate are quantitative reported preference measures? Our second finding is that in the aggregate, quantitative reports are generally reliable – the reported preference and revealed preference methods produce similar estimates of population average spending propensities. We make this comparison by mapping the qualitative reports of “mostly spend,” “mostly save”, and “mostly pay off debt” into quantitative propensities, using a calibration procedure established in the literature (Shapiro and Slemrod 2003). We are also able to calculate quantitative reported spending propensities directly from the NCP question that asks households for the dollar amount that they spent out of their stimulus payment. In either case, when quantitative spending propensities are averaged across the entire sample, the reported-preference and revealed-preference methods produce similar estimates of the total spending response, and discrepancies are within the bounds of statistical error.
Negative results: Spending differences across households
Our third finding, however, reveals a significant inaccuracy of reported preference spending measures. Basic models of lifecycle consumption under borrowing constraints predict that low-liquidity households will consume a larger proportion of transitory income like stimulus payments, and we indeed find substantial differences in revealed spending behaviour across liquidity and income categories. But despite this clear relationship between liquidity and revealed preference spending propensities, reported preference spending measures do not vary systematically with households’ liquidity or income status.
Figure 2, constructed from the NCP survey, shows the lack of differences in reported behaviour across households with different amounts of income and liquidity. In the left panel, households are sorted into three groups of roughly equal size based on their income. In the right panel, households are sorted into two groups based on their response to a yes/no question gauging their liquidity (“Do you have at least two months of income available in cash, bank accounts, or easily accessible funds?”). The average revealed preference spending propensities, measured as a percentage of stimulus payment amounts, are higher for the low-income and low-liquidity groups. Average reported preference spending propensities (taken from the NCP question asking for the dollar amount of spending out of the stimulus payment) are very similar across categories (and are actually larger for the high-liquidity group). There is also a lack of variation in qualitative spending reports – the share of households reporting that they “mostly spent” their stimulus payment does not differ significantly across income or liquidity groups.
Figure 2 Reported and revealed estimated spending propensities in NCP dataset
Note: See Parker and Souleles (2017). Whiskers display one standard error bands. Source: NCP, results are calculated based on data from The Nielsen Company (US) LLC and provided by the Marketing Data Center and the University of Chicago, Booth School of Business.
Why might reported preference measures fail to capture these differences? First, high-income and high-liquidity households may not distinguish the independent effect of the stimulus payment on their behaviour, leading them to mentally allocate some consumption spending to the payment when they would have made those purchases anyway. A second possibility is that people may report spending relative to their usual behaviour. Low-liquidity households have persistently higher spending propensities, leading to the observed relationship between liquidity and revealed-preference estimates. But households may only report having “mostly spent” their stimulus payment if they exceeded their already different baseline spending propensities, leading qualitative spending reports to be similar across income and liquidity groups.
Can economists or policymakers simply ask households to report what various policies would (or did) cause them to do differently? We find such reported preference estimates of behaviour to be reliable in some dimensions and unreliable in others. We added survey questions to the CE and NCP surveys about tax payments distributed quasi-randomly in the US, and use the answers to compare reported preference and revealed preference estimates of consumption responses of the same households to these payments. Encouragingly, qualitative spending reports are informative about revealed behaviour and average reported spending propensities agree with average revealed spending propensities. But despite clear differences in revealed spending behaviour, reported preference spending measures are not higher for low-liquidity households. We conclude from these results that reported spending data are likely valuable in predicting behaviour and in estimating population aggregates, but are not sufficiently accurate to provide reliable quantitative measurements of household-level spending responses.
Bunn, P, J Le Roux, K Reinold and P Surico (2017), “The consumption response to positive and negative income changes,” Bank of England, Working Paper 645.
Crump, R K, S Eusepi, A Tambalotti and G Topa (2015), “Subjective intertemporal substitution,” Federal Reserve Bank of New York, Staff Reports 734.
Graziani, G, W van der Klaauw and B Zafar (2016), “Workers’ spending response to the 2011 Payroll Tax Cuts,” American Economic Journal: Economic Policy 8(4): 124–59.
Jappelli, T and L Pistaferri (2014), “Fiscal policy and MPC heterogeneity,” American Economic Journal: Macroeconomics 6(4), 107–36.
Parker, J A and N S Souleles (2017), “Reported preference v. revealed preference: Evidence from the propensity to spend tax rebates,” NBER Working Paper w23920.
Sahm, C R, M D Shapiro and J Slemrod (2010), “Household response to the 2008 Tax Rebate: Survey evidence and aggregate implications,” in Tax Policy and the Economy, Volume 24, NBER Chapters: 69–110.
Shapiro, M D and J Slemrod (2003), “Consumer response to tax rebates,” American Economic Review 93(1): 381–396.
Shapiro, M D and J Slemrod (1995), “Consumer response to the timing of income: Evidence from a change in tax withholding,” American Economic Review 85(1): 274-283.