Basis
Uber has large teams vying for their user’s attention, and all operate on a greedy basis— Rides, Eats, Delivery, and more all with their own marketing, promos, and merchandising that serve to create profit for the company. The company only makes money when someone engages it and uses one of its services, or through their subscription-based service. With all of these teams and goals vying for attention, it becomes important that notifications don’t lead someone to disengage, and are a very important part of what drives revenue.
History
In the outset of when the Uber team overrode my friend and co-worker Casey Edgeton’s rule of not using notifications for marketing (March 2020), Uber’s notification system and experiments had some issues around scheduling—namely there were quality issues with the content (dead links, expired codes being sent) and timing issues (notifications stacking within minutes of each other, possibly after hours). The marketing team owned these notifications and were manually checking messages being sent. Today’s system came about to resolve these issues as different orgs scaled their contact with users.
System Design
Uber’s teams engage with all users through an internal queuing and scheduling system that intakes requests for pushes from all teams. The system is designed to answer this question: What is the best “schedule” with which to send the pushes currently buffered for a user?
The system takes in pushes that have associated meta-data, if relevant, across the following constructs:
-
Push expiration time (e.g., must send the push before a promotion expires)
-
Push send window (e.g., only morning hours)
-
Daily frequency cap (e.g., at most 2 pushes per day)
-
Minimum time difference between push notifications (e.g., 8 hours between pushes)
-
Restaurant open hours
Uber’s system considers possible combinations of messages across the day, with different schedules for N pushes fitting into S slots. Notably, it doesn’t combine messages into a single push. It takes all the constraints above and uses a linear program solver in favor of greedier systems like Multi-Armed bandits, in that it can “notice” that a push notif is expiring soon and prioritizes getting it sent out quickly, even if other pushes in the queue appear more likely to get the user to engage in the near-term. If the size of the queue reaches a larger value than the frequency cap, the system constructs a schedule with the most valuable pushes queued for delivery and drops the remaining ones.
Considerations
-
The system pairs a push and time and evaluates with a success criteria that the user makes an order within 24hrs of receiving that push at its time.
-
The conditions that their model accounts for are:
-
Time features: hour of day, and day of week
-
Push features: category, content identifier, and deeplink
-
User features: order history by day of week and mealtime, and engagement history with push notifications by day of week and mealtime
-
Overall, really interesting implications of how we might queue from competing teams or health apps, along with some parameters that can help us keep our system running smoothly day-in, day-out.