Indian retailers who have opened their stores on and off post the various lockdowns of different degrees of stringency in different states have been facing an interesting problem. Invariably, whenever and wherever they have opened their stores, sales have indeed gone up, but predicting where the growth will come from is increasingly becoming a guessing game for most of them.

Retailers came to the painful realisation that traditional forecasting models based on time series techniques, using historical sales data, was inadequate in predicting sales during the COVID-19 pandemic. Traditional forecasting models leveraging two or three years of history to capture seasonality did not help when demand for many items doubled or tripled, while for others it plummeted. Rapidly fluctuating demand meant that retailers needed to shift their focus from solely predicting future store sales in the mid- to long-term planning horizons, to more accurate short-term planning. In addition, they found that there was a treasure trove of external market data such as COVID-19 infection rates, the lockdown and re-opening status in various districts, mobility indices (Google, Apple), demographics, weather patterns and macroeconomic information, that could be made use of as drivers to explain demand patterns and improve the forecast accuracy.

Market knowledge can improve forecast accuracy and explainability

Retail forecasting competencies and systems are rapidly changing to incorporate market knowledge through publicly available data on consumer demographics, macroeconomic indicators such as Gross Domestic Product (GDP) and interest rates, social media buzz, and global trade. Other leading indicators of demand like news, product reviews, search engine statistics, and website glance views are becoming more and more important as demand sensing levers.

Retailers can get very granular, down to the store and pin code level, by bringing in local weather, events near their stores and road conditions, which affect store footfall and consumer purchases. Forecasting techniques are now shifting from traditional time series methods and moving towards intelligent forecasting through AI/ML (Machine Learning) and cloud computing, that can leverage a myriad of external market drivers and scale to retail volumes. Local weather, where increasingly, even different localities within the same city seem to have different weather patterns, stocking milk or fruit juices/cold, local brands or multinational brands, especially for food items, are important short-term decisions that retailers need to take in real time.

These next generation technologies can take leading indicator data and create a view of the forecast that is free of human bias or manipulation. All while constantly learning what leading indicator data best predicts changes for a more accurate forecast, right down to granular detail, such as store, item, day/hour, and the specific consumer fulfilment option – purchase at store, ship from store or click-and-collect.

No longer a secret sauce - AI/ML with robust feature engineering

Feature engineering is perhaps the most important part of the process and critical to robust results. Features can be created from any of the internal or external drivers. Features are also created from the historical sales stream such as seasonality, causal lags, lifecycle characteristics, and trends. An example of a causal lag feature could be an event (e.g. markdown discounting, promotion) that influences consumer purchases a few days or weeks after initiation. A key analytic in modern forecasting systems is a visual and interactive view on feature importance, which is critical to understanding and explaining the forecast. For example, the system may point out that price discounts have a greater impact on demand than holiday periods or weekends.

ML has the ability to iterate over multiple combinations of features to create models with superior forecast accuracy at more granular levels. ML with robust feature engineering delivers robust forecasts for different demand patterns at varying levels of granularity and consumer channels:

Omnichannel demand: Separate forecasts for click-and-collect, ship from store, ship from DC and in-store purchases.

Slow movers: This is where the volume of sales for items may be too small to generate a robust forecast. Hierarchical ML algorithms have to be leveraged to forecast at an aggregate level and then intelligently disaggregate to lowers levels (e.g., at store/daily).

Day of week variations: Typically, retail forecasting is done at the weekly level, but for food items with a short shelf-life and replenishment done many times in a week, a daily forecast is necessary, and day of week variations must be taken into account.

Intraday forecasting: Grocers may have items that get replenished several times in a day such as in the bakery section. This requires more granular level forecasts, which can be by the hour or shift.

In summary, while the historical way of forecasting is still useful to plan the broad numbers, categories and products that a retailer will sell using these AI and ML techniques, along with market knowledge and the understanding of various demand drivers will help accurately predict what will be sold in the next hour, next day, week, and so on. We live in a dynamic world, our customers are bombarded with a plethora of choices, both online and offline. Retailers need to be ready to anticipate their behaviour and react to them instantaneously.

The writer is Vice-President of Industry Strategy at o9 Solutions

Related Topics