On 5 May 2023, the World Health Organization ended its global health emergency declaration for the COVID-19 pandemic. For 1,191 days since the global health emergency declaration on 30 January 2020, our eating, learning, working, and leisure practices were transformed (e.g. Huber 2022, Camara et al. 2020, Bloom et al. 2020). In particular, in the first half of 2020, panic consumption occurred, focusing on COVID-19 infection prevention goods and paper products. We witnessed these changes in consumer behaviour every day through media and video sources.
At the time, I had a feeling that these unprecedented changes would not be detected through official statistical surveys alone. During the pandemic, purchases of goods and services fluctuated in volume on a daily or weekly basis depending on the infection status and government announcements. Such fluctuations are not expected to be reflected in official statistical surveys, which mainly consist of monthly and annual aggregations. In the initial phase of the pandemic especially, data had to be quickly reported to detect panic consumption and make quick decisions to secure supply routes and inventories. In addition, detailed sales trends by product were required for infection prevention goods, food products, day-to-day necessities and drugs, for which demand fluctuated greatly during the pandemic.
Therefore, we analysed point of sale and household account book application data and continued to record and disseminate consumption behaviour information in Japan throughout the pandemic.
Impetus provided by the METI Big Data Project
From 2016 to 2020, I participated in the Project for the Development of New Indicators Utilizing Big Data at the Research and Statistics Department of the Ministry of Economy, Trade and Industry (METI). The main objective of the project was to reduce the burden on survey targets, cut survey costs, and improve the speed and level of detail of survey reports by utilising private sector big data for official statistical surveys. At the same time, we proactively released new indicators. In November 2019, we developed a METI POS retail sales index (micro) using POS data from INTAGE Inc. (hereinafter referred to as INTAGE) and GFK Marketing Service Japan in order to ascertain the impact of the consumption tax hike that occurred at that time. The BigData-STATS Dashboard (β version) was launched on the METI website to publish the previous week’s sales trend indicators every Friday, allowing viewers to capture the actual status of consumption on a real-time basis (updates were discontinued on 2 April 2022).
During the chaotic period between the beginning of the COVID-19 outbreak and Japan’s first state of emergency declaration, this project became a platform for public-private cooperation, allowing government and other public organisations to share daily sales trends for infection prevention products and day-to-day necessities. In addition, the dashboard was used as a source of information on consumption trends during the pandemic,
displaying conference documents produced by government and other public organisations, reports by private think tanks, and material from media outlets and researchers.
At an international symposium on 23 March 2020,
I reported (1) a dramatic increase in sales of infection prevention products, such as masks, and their shortages; (2) hoarding of food in preparation for school closures and requests to work from home; (3) explosive buying of paper products due to widespread misinformation being spread through social media; and (4) a fall in cosmetics sales because people covered their faces with masks and stayed home more, reducing the overall demand for cosmetics. These findings were a result of detailed, quick reporting and swift collection of big data. I continued to report consumption trends throughout the pandemic.
Decomposing periods of the COVID-19 pandemic based on changes in sales trends for different goods
INTAGE's nationwide retail store panel survey (SRI+) collects POS data from about 6,000 stores in Japan, including supermarkets, convenience stores, home centres, discount stores, drug stores and speciality shops. The data were classified into 344 product items, including food, beverages, day-to-day miscellaneous goods, cosmetics and pharmaceuticals, and ranked by sales value for each year.
Figure 1 is a scatterplot of rankings for a year (vertical axis) and the previous year (horizontal axis). The size of the bubble is the square of the ranking difference. Product items for which rankings rose or fell by at least 30 places are named. The upper right edge of the 45-degree line represents the top-ranked product for both years. The lower left end indicates the 344th place for both years. Rankings remained unchanged from the previous year for product items on the 45-degree line. Rankings rose from the previous year for those items located above the 45-degree line and fell for those located below the line.
Figure 1 Scatterplot of sales rankings comparing one year to the previous year
Note: bubble size is the square of the difference in ranking from the previous year
Source: Author’s creation based on INTAGE SRI+ data
In 2019, many product items were located on the 45-degree line. No category rose or fell by 30 or more places. Many of the 344 product items are day-to-day necessities whose rankings were stable before the pandemic.
In 2020 when the pandemic started, however, rankings rose by 30 places or more for infection prevention goods, such as masks, hand sanitisers, disinfectants, gargle medicines and thermometers, as well as soaps and wet tissue that were consumed as substitutes for infection prevention goods when such goods were in short supply. On the other hand, rankings fell by 30 places or more for lipsticks, blushes, other lip makeup products, wet paper facial masks, sunscreen and gifts (mid-year, year-end, and other gifts). The ranking also dropped steeply for cardiotonic drugs that had sold well to inbound foreign tourists before the pandemic.
In 2021, no products rose by 30 or more places. However, soaps, bactericidal disinfectants, hand sanitisers and gargle medicines among items that had sold well in 2020, fell by 30 or more places.
In 2022, almost all items were located close to the 45-degree line. The concentration around the line was even stronger than in 2019 before the pandemic. For most of the product items, rankings in 2022 were similar to those in 2021. In 2022, test reagents emerged as a distinctive product item.
Overall, Figure 1 shows that the consumption trend under the pandemic changed from the panic consumption period of 2020 to the new normal period of 2021-2022.
Graphing our consumption behavior, a flower emerged
Next, I turn to visually examining weekly ranking changes with a rank clock. Since it is difficult for human eyes to identify changes for all 344 product items over 209 weeks, I have selected items that ranked in the top 20 in terms of sales in at least one of the 209 weeks since 2019 and compiled the data in Table 1.
Table 1 List of items that ranked in the top 20 in sales in at least one of 209 weeks from 2019 to 2022
Note: I selected items that ranked in the top 20 in sales in at least one of 209 weeks from 2019 to 2022. The highest-ranked product is the highest for the period and the lowest-ranked product is the lowest for the period.
Source: Author’s creation based on INTAGE SRI+ data
Figure 2 is a graph of weekly rankings for product items in Table 1, with the top-ranking items being at the centre, and lower places deviating from the centre. Table 1 shows that most of the product items in the top 20 rankings are food and beverage products. In particular, tobacco, beer, pastries, liquid tea, milk, ice cream, and coffee drinks have stayed at or around the centre.
Figure 2 Rank clock for items that ranked in the top 20 in sales in at least one of 209 weeks from 2019 to 2022
Source: Author’s creation based on INTAGE SRI+ data
As data are like living creatures, the graph shape changes depending on the product items, locations and times. Figure 2 forms a unique flower shape, indicating that there are items that sell well for the year-end and New Year holidays and in some seasons or some events, and poorly for others. For example, demand increases for kamaboko and other fish paste products in January and other winter months, for chocolate towards Valentine's Day, for rhinitis treatments during spring and autumn hay fever periods, for batteries just before typhoons, and for insecticides in summer. Gifts are ranked low except during mid-year and year-end gift seasons. Using weekly data on detailed items, we can see clear seasonal changes.
As shown by Table 1, the items that ranked highly because of the pandemic were masks and paper products. Masks ranked 200th in the summer before the pandemic and peaked in second place in the fourth week of January 2020 in the early days of the pandemic. Figure 1 shows that masks ranked high in annual sales in 2020. On the other hand, personal care products that were selling well just before the consumption tax hike in 2019 and paper products subjected to panic buying at the beginning of the pandemic ranked high in Table 1 but failed to continue to sell well or be among the items that sold prominently in Figure 1. In addition to seasonal changes, weekly data indicate short-term shocks.
What will happen to mask consumption? What will day-to-day life be like after the pandemic?
Finally, let us look at the impact of the pandemic on the seasonality of consumer behaviour. Figure 3 is a cumulative density function of mask sales volume, accumulating weekly sales’ shares of annual sales. The 45-degree line represents the function of a constant sales volume, meaning that there was actually a sales increase of approximately 1.92% (1/52≒ 0.0192).
Figure 3 Cumulative density function of mask sales, 2018-2022
Source: Author’s creation based on INTAGE SRI+ data
In 2018 and 2019, before the pandemic, sales concentrated in winter and in the spring hay fever season. Sales in the second and third weeks of April accounted for half of annual sales. Sales stagnated in summer before some 30% of annual sales occurred in autumn and winter. Mask sales in the two years thus indicate seasonal changes.
In 2020, cumulative sales topped 30% of annual sales in the week including the 30 January WHO global health emergency declaration and stagnated amid mask shortages later. Cumulative sales exceeded 50% at the end of the first state of emergency, before increasing at a constant pace later as masks returned to the market.
Note the trends in 2021 and 2022. In both years, trend lines mostly overlapped the 45-degree line, meaning that weekly sales remained unchanged. Panic buying and mask supply shortages seen due to the sudden increase in demand in 2020 disappeared in those two years. At the same time, seasonal changes seen before the pandemic also disappeared in the two years, resulting in a new mask consumption pattern.
On May 8, 2023, COVID-19 was downgraded to a Category 5 infectious disease. Will mask consumption return to the pre-COVID pattern or create a new pattern? Either way, we can say that masks are the most representative goods of the pandemic, as the mask market was affected first and to the largest extent under the pandemic in Japan.
Towards a society where official statistical surveys and data platforms are both active
The use of consumption-related big data, including credit card information, home scan data, household account book application information and electronic money information, as well as POS data, progressed under the pandemic. I used POS data, which for example demonstrated a decrease in makeup product sales and an increase in mask sales, to analyse voluntary restrictions on going out and mask-wearing. More directly, however, human traffic data can be used for such purposes. Artificial intelligence image recognition is capable of measuring mask-wearing rates in public.
In the current age, where it is possible to learn anything in detail, anywhere and at any time, I feel that the most difficult thing is to decide what to measure and to continue measuring those things. The existence and use of official statistical surveys that continue to measure the status of a country based on long-term hypotheses is very encouraging, because findings from big data can be checked against survey data that will be made available by government agencies one or two months or one year after the big data is available, revealing trends. This intellectual foundation has allowed me to freely utilise big data without hesitation.
Human resources and budget constraints have been significant constraints on the production of statistics for quite a long time now, but I hope that official statistical surveys will continue to develop by mobilising digital technologies, private sector data and administrative records, and by employing statistics experts. The fact that the government has the means to learn about business conditions and people’s lives and to produce statistics means that its policies can be more persuasive and that their policies can have an intellectual basis. With this in mind, collecting multiple types of data from multiple companies, instead of data from one company in one field, and being prepared to create platforms that are useful for quick political decisions during peacetime are both beneficial and important.
Authors’ note: This column was reproduced with permission from the Research Institute of Economy, Trade and Industry (RIETI).
References
Bloom, N, S Davis and J M Barrero (2020), "60 million fewer commuting hours per day: How Americans use time saved by working from home", VoxEU.org, 23 September.
Camara, Y, J W Galbraith and D Bounie (2020), "The COVID-19 containment seen through French consumer transaction data: Expenditures, mobility and online substitution", VoxEU.org, 26 May.
Huber, S (2022), "SHE can’t afford it and HE doesn’t want it.The gender gap in the COVID-19 consumption response", VoxEU.org, 30 April.
