How AI increase drive-through sales for Starbucks (Data+A.I. summit 2022)

This post summarize a great sharing on Data+A.I. summit 2022 on how Starbucks implements a Reinforcement Learning solution to improve its sales in drive-through scenario.

Source: Lessons Learned Running RL Recommendation at Scale in Physical Retail Setting at Starbucks – by Ed Daroza, Sulbha Jain

Sharing video: link

Glossary

• Plus Cart: A cart with items.
• Zero Cart: A cart with no items.
• DQN: A DQN, or Deep Q-Network, approximates a state-value function in a Q-Learning framework with a neural network.

TL;DR – Starbucks’s Learnings

1. Using SOTA ML platform (Azure Databricks) make it easy to complete ML E2E lifecycle.
2. Reinforcement Learning shines at its explore-exploit property in the face of uncertainties (in Starbucks case, catalog exploration). Simple epsilon-greedy DQN perform well at scale.
3. Feature engineering plays a key role of success (100+ features)
4. Taste profile, NRT features like weather, improves the quality of recommendation of the agent.
5. Caching is effectively used to meet the latency and throughput SLA.
6. Explainable AI to decipher RL model for algo improvements, partner trust and reasoning.
7. A separate supervised collaborative filtering model is used for “Plus Cart” case to improve precision.

Reinforcement Learning Basics

Before going into details, here are some references materials about reinforcement learning in case you need to brush up some fundamentals (as I did).

Training – Markov Decision Process

Algorithm: Epsilon Greedy in Deep Q Learning

Reference: https://pylessons.com/Epsilon-Greedy-DQN

Reference: MDP – Markov Decision Process

Reference: https://towardsdatascience.com/understanding-the-markov-decision-process-mdp-8f838510f150

Starbucks Scenario

Create “in-person” sales recommendation experiences in the digital ordering process (e.g. the order screen at drive-through lane) to increase sales numbers and customer satisfaction for ~600 SKU.

The Challenges

• Business challenges – in-store employee training, hardware, infrastructure.
• Scale – billions transactions a year; terabytes data daily;
• Latency – 12ms avg response time with <50ms @95th percentile
• QPS – ~10k/min recommendation queries at peak
• Cold start the algo
• Store-based personalization – demographics, taste pattern, seasonality and so on

The Solution – Reinforcement Learning for SKU ranking

According to Sulbha, the RL solution solves below issues for them:

1. Sales likelihood estimation of each products.
2. Increase sales conversion rate of the order board.
3. Cold start and new SKU: store-personalization recommendation that adapts to the seasonality, demography and geolocation.
4. Enable product exploration – get customer try new product.

The Tech Stack

• Cloud: Azure
• Training collaboration: Databricks
• Learning frameworks: Keras, Tensorflow, pytorch
• Data analytics and processing: pyspark, python, scala, R, SQL
• Report and visualization: Tableau

Monitoring

• Monitor for result drift and retrained as needed.
• Model performance monitoring with intrinsic metric dashboards.