In the United States, Thanksgiving is around the corner, marking the start of the holiday shopping season. If you’re one of the 174 million Americans who go shopping on Black Friday, this day might bring back memories of looking through mailers at the discounts and sales and making plans for shopping. In the past year, the heavy discounts and early start of retail holiday shopping made for a frantic experience —  shoppers hunted the best deals and contended with crowds while ultimately spending about $400 on average per person ^1,2. 

The impact of this shopping surge on retail business cannot be understated: 30% of all retail sales occur in the holiday shopping season starting at Thanksgiving and running through Christmas, and 30% of those sales happen on Black Friday¹. Indeed, even the name of the shopping holiday reflects this.

The term ‘Black Friday’ has connections to the impact this shopping had on business, pushing their accounts from the red into the black (losses to profits).

But Black Friday has been changing, and retailers must change with it if they want to thrive in the new landscape.

The effects of online shopping on the retail industry have been drastic, with numerous legacy retailers falling to the ranks of bankruptcy or bargain-sale acquisition. These impacts continue to be seen within Black Friday shopping. Last year, 40% of Black Friday shopping occured online. And the more recent creation of Cyber Monday has extended this sale beyond just one day and into an extended four-day weekend. 

The shift to online retail has many drivers. Online retailers have bargains similar to brick-and-mortar retail stores, and the fact that 37% is the average discount helps explain the motive behind the shopping frenzy¹, but consumer preferences around online shopping are clear. In the last five years, not only has the share of online shopping grown (up 14% year-over-year), the concentration of shopping has also been stretched over longer periods of time. There’s a nearly 50% reduction in the number of holiday shoppers who plan to do most of their shopping on Black Friday — down from 59% in 2015 and to 36% in 2020 — all of this while overall shopping on Black Friday has increased⁴.  More recent additions like Amazon’s Prime Day and other pre-Black Friday sales specials are contributing to this as well, stretching the holiday shopping window further. And these changes in shopping have also led to another change: the data.

A New Data-Rich Reality

As retail shopping has shifted online, the amount and quality of data available on it has also grown rapidly. There are vast quantities of information available through a combination of web-search trends, advertising, and social media interactions. Changes in consumer preferences and buying behaviors are clear from web-search data⁵, where you can find highly descriptive data about customer buying plans about specific items from different retailers. This is to say nothing about the data available to the retailers themselves from previous customer sales. This influx of data has allowed the largest online retailers to have unprecedented information about their current and potential customers. 

In contrast, consider for a moment how much this differs from brick-and-mortar retail sales, and how much less information is known about individual customers in that setting. Seldom will brick-and-mortar retailers be able to track customer browsing behavior, know specific sales history, or have any idea on buying plans for any customer in the store. Estimates of foot traffic are subject to the variation caused by weather, traffic, and any regional differences at the best of times. Meanwhile, all of this type of insight is available to online retailers. The value of this insight has, in turn, driven brick-and-mortar retailers to increase data collection and try to better understand their customer’s preferences and future demand for products.

Enter Artificial Intelligence

The growth in the breadth, depth, and quality of data on retailer shoppers, driven initially by growth in online shopping, has allowed for another change: the introduction of artificial intelligence (AI) into holiday shopping. Long have sales forecasts been a mainstay of the retail industry. Staffing, inventory, and logistics chains are dependent on planning estimates. Historically, these forecasts have been based almost exclusively on historical sales data: how much of each item sold, at which store, on which day. Forecasts built on this type of univariate time series data, using only historical sales history to predict future sales, have long been based on moving averages, decompositions, smoothing, and various dynamic and autoregressive models. Now, there are numerous new techniques that have improved upon this univariate forecasting, with major advances through deep learning with recurrent neural networks, sophisticated smoothing approaches, and advanced generalized additive models ⁶. 

But driven by the variety of data, there are even more advancements in AI systems that utilize all of the available data, including: web-search trends, marketing and advertising campaigns, discounts, weather, previous sales history, information about holidays, and even granular geolocation data or images.

Developments and innovation on multi-modal modeling approaches has led to the ability to include virtually any type of data to improve forecasts. And the incorporation of these more diverse data types has led to improved accuracy within forecasts, as demonstrated in the results from the most sophisticated forecasting competitions using sales data from the retailer Walmart.

Now the diverse, rich data available can be used to better understand optimal pricing for items, predict demand for items, and provide actionable insight to make decisions for staffing and logistics operations. And on the customer side, there is a tailored shopping experience with ease of finding products, timely recommendations, and tempting discounts. All of this is driven by the wealth of data and the ability to put it to good use through artificial intelligence, with the goal of increasing customer spending with a more engaging user experience.

For retailers, these novel approaches have increased the accuracy of many forecasts, leading to cost savings for retailers. Beyond forecasting, there has been a tremendous surge in the use of AI with marketing and advertising, as experienced by anyone who has seen targeted advertisements related to recent web-searches or purchases. All of these different components of AI systems are weapons that the retailer can use in the war for customer attention and customer dollars, even as increasing competition causes tighter margins. Getting the right item, at the right price, in front of the right customer has never been as feasible or so critical.

Black Friday in 2020 and Beyond

Changing shopping trends around Black Friday and Cyber Monday, and the shifts toward online retailers have been happening for over a decade. But the impact of 2020 on retail is looking to be dramatic. In the wake of the COVID-19 pandemic, some of the largest brick-and-mortar retailers, giants like Target and Walmart, will close their stores on Thanksgiving and limit the number of shoppers in stores in an effort to reduce the risk of shopping in stores and push more shoppers online⁸.

Estimates are that online shopping will be up nearly 50%, growing to 61% on Black Friday this year.

Supporting this effort, retailers have also started the sales season early. Major brick-and-mortar cum online retailers like Best Buy, Target, Walmart started the sales in mid-October, following suit behind the online behemoth Amazon in hosting its annual Prime Day in mid-October. Why have these retailers pushed for earlier and earlier sales? Some of this is certainly driven by the quest to capture wallet share of holiday spending: as a retailer, if other retailers hold early sales and you don’t, then you risk missing out to competitors that ran those early deals. 

But competition for consumer dollars isn’t the whole story. The logistics of shipping fulfillment for online orders also needs to be considered. 

There is only so much shipping capacity available, especially with staffing reductions and other restrictions due to the COVID-19 pandemic. As a result, retailers are pushing to distribute that shopping over a longer period of time to reduce the surge-stress on staff and processes. And because there is the risk of running out of supply of popular items, stretching the holiday shopping season also reduces surge-stress on inventory. 

Finally, retailers are hoping to capture not just a larger share of spend, but also drive up total spend. In other words, that bargain TV you buy in early October may not exclude other shopping later in the holiday season. If you see these great bargains over a longer period of time, spending is expected to rise overall. Extending the shopping season also helps toward countering the suppressive effects that the COVID-19 pandemic, which has had brutal effects on many retailers this year and is certain to affect in-person shopping this year.

With these changes in holiday shopping this year, how will retailers  make predictions for staffing, item demand, and fulfillment logistics? Much of this decision-making is largely guesswork, because the 2020 holiday shopping season will be like nothing in recent history. 

If retailers build more sophisticated models that include data from the past that describe these sales events, including marketing and advertising spend and special discounts, then those models built on data from previous years will extrapolate toward holiday shopping this year with the marketing and sales starting earlier.  

The future is clearly going to be both data-rich and influenced by AI systems, as some retailers continue to expand their competitive advantage, supported by better insight into customer demand, while others see a widening gap with their competition. All retailers would be wise to move toward adoption of AI throughout their organization, lest they become the next Sears, JCPenney, Toys ’R’ Us or other former retail giants who were unable to adapt to changes in the retail space.

The future in retail shopping is unclear, and we don’t yet know how permanent many of the changes brought by 2020 will be. Those companies that have already integrated advanced analytics powered by AI into their decision- making and planning are looking to come out on top this year, and carry that momentum into 2021 with whatever new disruption that brings. 

References

1. Lynkova, D. 25 Black Friday Sales Statistics to Know in 2020. (2020).

2. Danziger, P. Thanksgiving Weekend Retail Results In: Shoppers Will Have Merry Christmas, Retailers Get Coal. (2017).

3. Black Friday History and Statistics. (2020).

4. Share of consumers who plan to do most of their shopping on Black Friday in the United States from 2015 to 2020. (2020).

5. Eberhard, E. Holiday shoppers are in the driver’s seat now—here’s what they’re looking for. (2017).

6. Makridakis, S., Spiliotis, E. & Assimakopoulos, V. The M4 Competition: Results, findings, conclusion and way forward. (2018).

7. Makridakis, S., Spiliotis, E. & Assimakopoulous, V. The M5 Accuracy competition: Results, findings and conclusions.(2020).

8. Kern, M. Black Friday is going to be weird this year — and that’s a good thing. (2020).

About the author

Jarred Bultema, PhD

Data Scientist, Time Series at DataRobot

Jarred is a multidisciplinary scientist with a diverse background. A trained Biochemist, Immunologist, and former University Professor prior to becoming a Data Scientist, Jarred also worked in the Biotech and Pharma industries. Jarred now works with clients across industries and governments to integrate sophisticated forecasting into their organizational DNA.

Meet Jarred Bultema, PhD