Why retailers are upping their investments in data infrastructure and advanced analytics

Why retailers are upping their investments in data infrastructure and advanced analytics

Were you unable to attend Transform 2022? Check out all of the summit sessions in our on-demand library now! Watch here.



As organizations continue their economic recovery efforts from the woes of the pandemic — and many look for new ways to gain competitive advantage — there is growing interest in advanced analytics and data infrastructure tools.

Most in demand are data tools that improve predictive and behavioral analysis, and that enable real-time data analysis.

One industry that is investing heavily in data infrastructure and analytics is the retail sector, including the convenience store segment. If that sounds surprising, consider this: As the country moves toward eliminating fossil fuel-based vehicles, that will eliminate a significant portion of the industry’s revenue stream. A large percentage of convenient stores sell fuel, and that is typically the biggest money generator.

To get a better sense of where retailers are investing, VentureBeat spoke with David Thompson, founder and CEO of 3 Leaps LLC, a company that helps businesses accelerate and scale automation using a data-driven approach.

Doesn’t everyone have their own data scientist?

While it is difficult to generalize, Thompson said the primary drivers of data infrastructure investments have been to increase retail channel performance through higher trip frequency and higher basket rates. As the name implies, the term “basket rates” refers to the number of items that a customer places into their carts, whether an actual shopping cart or a digital one.

“In certain subsectors, there has also been a large investment in live chat or other customer engagement tools to increase responsiveness and lower cost-of-presence,” Thompson said. 

The first question, Thompson said, his organization is typically asked by potential customers is, “How can these technologies help us better understand our customer base?” Or questions about how the technologies “drive investments in customer segmentation, promotional planning and pricing.” 

“Most retailers with whom we work are looking for a degree of ‘measured automation,’ where routine decisions can be made by a system and outlier cases can be brought to an expert’s attention for personal review,” he said. “Today, we are seeing retailers in many sectors hire their own data scientists, setting up initiatives either on their own or to extend solutions from third parties. The challenges of static ‘rules-only’ forecasting models have become painfully clear with the supply chain interruptions caused by the pandemic.”

He added that the company is now “ … seeing more investments in what we call ‘classification’ and ‘interpretive’ technologies, where we use NLP [natural language processing] and advanced multimedia recognition in support of live chat and transcript ‘sentiment analysis’ to extend and improve our customer outreach.”

Using data infrastructure improvements to each supply chain disruption

The largest impact of strengthening data infrastructure for many retail sectors has been seen in supply chain optimization. That can cover anything from replenishment to assortment planning, depending on the retail vertical.

For retailers with a multichannel strategy, the priority may be to help the retailer understand better the benefits and costs of complex fulfillment options such as ‘order online, pick up in store’ or to consider multiple delivery strategies. 

“Finally, we see e-channel retailers in particular having invested in tools to automate very rapid competitive responses — what we sometimes call ‘dynamic pricing,’” Thompson said.

While the basics of such competitive indexing are rules-based, the approach often requires weights or strategy inputs built from various artificial intelligence (AI) or machine learning (ML) processes to finalize responses.

The best programs, from Thompson’s experience, focus on measurable success criteria that include specific measures of error as well as procedures to handle the “unknown” cases that inevitably arise.

“Conversely, a lack of attention to these areas will almost certainly result in a failed implementation,” Thompson said. “User confidence, once lost, is incredibly difficult to regain. Starting with a subset of the business and dedicating extra time to measuring the results will help instill confidence that the benefits will scale with the program.”

Cashing in on the benefits of advanced analytics tools

There are two primary areas where Thompson said retailers hope to benefit from investments in data infrastructure and advanced analytics tools: In supporting growth and increasing productivity.

“First and foremost, AI/ML tools and applications can help us understand our environment and customer base more quickly and more thoroughly,” Thompson said. “This knowledge can then be used to evaluate potential strategies more effectively. With the economies of computing these days, we can also consider a wider range of possible strategies than we could in the past, with much less manual work.”

“Second, we can lower costs and improve retention through increased quality of service. Eliminating unnecessary paper processing makes people happier,” Thompson said. “Being able to evaluate every single interaction helps us improve our training and responsiveness. Knowing more about what a particular day will help us improve the labor positioning we bring to a particular situation.”

Consultants at 3 Leaps LLC focus heavily on predictive analytics when discussing advanced technologies within retail, and for good reason. 

“Digital workflows and RPA [robotic process automation] can deliver huge benefits in accuracy, data security and lower overhead costs. These solutions typically leverage AI/ML solutions for image, text and even speech recognition,” he said.

Going paperless has become something of a cliché, but Thompson stressed that it really should be the goal of every organization. ‘Smart forms,’ digital identification methods and other tools can enable employees to complete complex workflows containing sensitive information from almost anywhere, saving money and boosting productivity.

“Multiformat chat and NLP tools have advanced dramatically in the past few years. Properly deployed, such technologies can assist both customers and employees in directed search [such as] ‘Where do I find … ?’, ‘How do I …?’ and training,” Thompson said.

New applications are emerging for training and coaching employees as well, whether by similar transcript analysis or by live simulated interactions.

“Look for this area to grow significantly in the next few years across industries such as ours with high training requirements and a need for regulatory or statutory compliance checking,” Thompson said.

Growing ‘comfort’ with advanced technology tools

Thompson’s organization is seeing the use cases expand as more companies become comfortable with an increased role for both classifying and predictive technologies.

“What we would highlight is the importance of building sound processes for data validation and testing,” Thompson said. “Think about the real-world examples we saw arise from the pandemic. Forecast models broke — badly, in some cases — due to a radical shift in shopper demand, a breakdown in the supply chain, or both. Successful use of the technologies requires periodic review and specific checkpoints built throughout the processes to abort [or at least warn users] when the data vary too much from expected norms.” 

Just as organizations have “A/B” testing for assessing the impact of price or assortment changes, they also need “A/B” testing for model quality, Thompson believes.

“We recommend asking your design teams, partners or suppliers to deliver and use [regularly] such a harness.  By running known historical data against the current system and a planned upgrade, we see the actual differences in output that arise from the changes,” Thompson says. “With such techniques, we build confidence in both the quality of our outputs and in the handling procedures for unknown or unexpected results. Unstable models will be rapidly rejected by our business users for good reason — it is not helpful to be right occasionally and wildly wrong most other times.”


VentureBeat's mission is to be a digital town square for technical decision-makers to gain knowledge about transformative enterprise technology and transact. Learn more about membership.