The sharing economy has introduced a new wave of online marketplace apps and business models, where consumers and sellers are matched across a unique differentiated set of product inventory. This contrasts with traditional e-commerce platforms, where inventory is standardized and sellers are few. Online multi-sided marketplaces range from Uber and Lyft in gig-economy transportation of people with unique routes to DoorDash and Postmates in restaurant delivery with unique orders, Instacart and GoPuff in grocery delivery with unique pick-up and drop-off locations, TaskRabbit and Amazon Mechanical Turk with unique ad-hoc jobs, Turo and Boatsetter for individual car and boat rentals, Airbnb with unique one-of-a-kind accommodations, and Rover for dog care.

Each of these businesses has successfully created a marketplace that matches customers with a wide variety of sellers offering differentiated products or services, rather than standardized items. This creates a more complex system than traditional retail as it presents a complex optimization problem where we must balance the multiple needs across a diverse set of users - times/dates, locations, qualifications / features / amenities, availability, regional regulations, and so forth. Additionally, the rules and functioning of the marketplace need to be considered carefully to prevent gaming and ensure a fair and transparent marketplace for all parties.

In this 8-part e-book, we will design a multi-sided online marketplace encompassing multiple domains, subsystems, and applications, with principles which can be applied to a wide variety of marketplace and gig-economy use cases from taxi services (e.g. Uber, Lyft), to restaurant delivery (e.g. DoorDash, Postmates), to third-party deliveries (e.g. Instacart, Shipt, Convoy). We’ll focus on a complex three-sided marketplace use-case involving three distinct sets of users: customers making purchases, third-party vendors, and gig-economy drivers. By the end we will have designed a global transportation platform with multi-sided marketplace where (i) consumers can shop through a variety of vendors (3rd-party merchants or taxi services), (ii) have their goods or themselves delivered quickly – as quick as same-hour, (iii) by gig-economy drivers, (iv) either scheduled-ahead or on-demand, (v) all without merchants or taxi companies managing any of the logistical complexity.

We must balance: speed, low-cost, and accurate-ETA’s for customers; consistent, highly-available taxi & delivery services for third-party merchants and taxi companies; and fairness, transparency, and consistency for gig-economy drivers. The platform will support a wide range of transport speeds starting as quick as same-hour with low latency APIs to optimize the checkout flow for maximum conversion. We will support multiple pick-up and drop-off locations, given products and customers will be distributed. Finally, we will support multiple mechanisms for gig-economy drivers to obtain work, either scheduling work ahead to guarantee sufficient working hours or working on-demand with maximal flexibility, obtaining work in real time.

1. Part 1: Introduction, Requirements, Mobile and Web Applications
2. Part 2: Backend Infrastructure and Service Architecture
3. Part 3: Vendor Management and 3rd Party Customer Shopping & Checkout
4. Part 4: Transport Supply & Demand Management and Forecasting with Machine Learning
5. Part 5: Route Planning, ETAs, and Dynamic Pricing with Machine Learning
6. Part 6: Driver Onboarding, Preferences, Standings, Rewards, and Payments
7. Part 7: Driver-Route Targeting, Pushed On-Demand Routes, and Precomputed Eligibility
8. Part 8: Gig-Economy Route Marketplace for Schedule-Ahead Routes with Deferred Route Matching

Software Architecture Series:

• Software Architecture - Mapping, Navigation, and ETA System
• Software Architecture - Metrics and Alarms Monitoring System
• Software Architecture - Payments System
• Software Architecture - Hotel Reservation Booking System
• Software Architecture - Translations Service & Clients

Part 4: Transport Supply & Demand Management and Forecasting with Machine Learning

Contents:

1. Introduction
2. Supply and Demand Forecasting with Machine Learning for Recruitment and Marketing
   1. Demand Forecasting
      1. Model Inference
      2. Model Training
   2. Supply Forecasting
      1. Total Drivers
      2. Active and Good-Standing Drivers
      3. Supply-Demand Delta
   3. Recruiting, Marketing, and Re-Engagement
      1. Maximizing Driver Experience and Life-Time-Value (LTV)
      2. Driver Delta
      3. Re-Engaging Inactive Drivers
      4. Recruiting and Onboarding New Drivers
      5. Funnel Outputs and Actions
3. Supply Management
   1. Long-Lead Time Windows
      1. Initializing Supply Data
      2. Updating Supply Data
      3. Long-Lead Availability
   2. Low-Lead Time Windows
4. Driver Tracking for Low-Lead Orders
   1. Ingesting Real-Time Signals
   2. Predicting Driver Acceptance
   3. Reserving Driver Availability
   4. Order ETAs
      1. Pick Up and Drop Off Timings
      2. Delivery ETA
5. Order Management
   1. Updating Long-Lead Supply
   2. Updating Low-Lead Driver Tracking
   3. View Order Status
6. Conclusion

Introduction

In Part 1, we envisioned a global transportation software platform enabling third-party vendors to rapidly transport goods to customers – or customers themselves – within an hour, covering requirements, web/mobile apps, and overall flows. In Part 2, we explored the backend infrastructure and service architecture. In Parts 3-8, we are diving into the domains, subsystems, and services required. Here in Part 4, we discuss the Supply & Demand domain, including how we leverage Machine Learning (ML) for forecasting supply availability, driver demand, and ensuring an optimized balance between them.

Supply and Demand Forecasting with Machine Learning for Recruitment and Marketing

Meeting customer demand while optimizing costs requires precise forecasting of future supply and demand months before any orders are placed. This is particularly critical for ensuring that there are enough drivers in the program to handle customer needs efficiently. To achieve this, our platform will employ machine-learning (ML) models to generate time-series forecasts for:

• Supply - Forecasting the number of active and available drivers per order type, time window, and region worldwide.
• Demand - Forecasting the volume of orders customers are likely to place, segmented by order type, time window, and region worldwide.

These forecasts must account for seasonal trends, holiday events, and year-over-year growth. Accurate forecasting will be important to minimize costs as we meet customer demand. Our platform’s elastic gig-economy workforce ensures that short-term demand spikes can be addressed without disrupting customer experiences, while maintaining cost efficiency for schedule-ahead deliveries.

Demand Forecasting

The Demand Forecasting Service will use machine-learning (ML) models to analyze historical vendor data of years past to see how many orders to expect, considering annual cycles, year-over-year growth, events such as holidays, and other marketing data. Each 3rd party vendor while onboarding can initially provide these forecasts themselves or provide us their historical order volume data. Once a vendor has been integrated into the platform for some time our own Order Management Service will contain the vendors historical data.

Model Inference

ML model inference will run offline as part of a weekly batched workflow, parallelized by region. The output of the demand forecast is a per-region time-series dataset segmented by order-type (e.g. grocery delivery, taxi service, ecommerce delivery). Ops managers and vendors will also have the option to override forecasts through the Operations Portal. For example, if an ops manager knows a specific event will be occurring, they may want to proactively increase forecasted demand for that region and time window. Likewise, a vendor may increase forecasted demand in preparation for a vendor sale. Whenever the demand forecast for a given region and order type is updated a domain event signal is broadcast which other components will consume.

Model Training

ML model training will occur offline as part of a fortnightly asynchronous workflow so that the model continuously improves over time. All overrides are fed back to the model during training so the model may learn from them for future forecasts. A pluggable interface where different models and their outputs can be compared against each other will be built to support iterative model improvements. As new model variants are built they can be deployed as experiments and their results compared against each other. Additionally, Ops Managers can run “what if” simulations to see how different configurations will impact the outputs.

Supply Forecasting

Forecasting the supply-side with how many drivers will be available is inherently more challenging, as we have less historical data to begin with and there are harder-to-predict variables at play. Drivers’ activity levels, program status, and engagement vary, requiring nuanced modeling. The Supply Forecasting Service forecasts supply per-region, segmented by the order types drivers are eligible for, while considering the various states drivers on the platform may be in: onboarded but never driven, tenured good-standing driver, in bad standing due to delivery issues or delays, driven in the past and now dormant, etc. The supply forecast integrates with demand forecasts, forming a sequence of machine-learning (ML) models that simulate a supply funnel, ultimately revealing the supply-demand delta and guiding preemptive actions to balance the system and meet customer demand.

The remainder of this chapter is available by purchasing the e-book.

Sign up to the mailing list to read additional free chapters of my book on Software Engineering, Architecture, and Delivery