Learning State-Dependent Policy Parametrizations for Dynamic Technician Routing with Rework

Home repair and installation services require technicians to visit customers and resolve tasks of different complexity. Technicians often have heterogeneous skills and working experiences. The geographical spread of customers makes achieving only perfect matches between technician skills and task requirements impractical. Additionally, technicians are regularly absent due to sickness. With non-perfect assignments regarding task requirement and technician skill, some tasks may remain unresolved and require a revisit and rework. Companies seek to minimize customer inconvenience due to delay. We model the problem as a sequential decision process where, over a number of service days, customers request service while heterogeneously skilled technicians are routed to serve customers in the system. Each day, our policy iteratively builds tours by adding "important" customers. The importance bases on analytical considerations and is measured by respecting routing efficiency, urgency of service, and risk of rework in an integrated fashion. We propose a state-dependent balance of these factors via reinforcement learning. A comprehensive study shows that taking a few non-perfect assignments can be quite beneficial for the overall service quality. We further demonstrate the value provided by a state-dependent parametrization.

The Restaurant Meal Delivery Problem with Ghost Kitchens

Restaurant meal delivery has been rapidly growing in the last few years. The main challenges in operating it are the temporally and spatially dispersed stochastic demand that arrives from customers all over town as well as the customers' expectation of timely and fresh delivery. To overcome these challenges a new business concept emerged, "Ghost kitchens". This concept proposes synchronized food preparation of several restaurants in a central complex, exploiting consolidation benefits. However, dynamically scheduling food preparation and delivery is challenging and we propose operational strategies for the effective operations of ghost kitchens. We model the problem as a sequential decision process. For the complex, combinatorial decision space of scheduling order preparations, consolidating orders to trips, and scheduling trip departures, we propose a large neighborhood search procedure based on partial decisions and driven by analytical properties. Within the large neighborhood search, decisions are evaluated via a value function approximation, enabling anticipatory and real-time decision making. We show the effectiveness of our method and demonstrate the value of ghost kitchens compared to conventional meal delivery systems. We show that both integrated optimization of cook scheduling and vehicle dispatching, as well as anticipation of future demand and decisions, are essential for successful operations. We further derive several managerial insights, amongst others, that companies should carefully consider the trade-off between fast delivery and fresh food.

Where the Action is: Let's make Reinforcement Learning for Stochastic Dynamic Vehicle Routing Problems work!

There has been a paradigm-shift in urban logistic services in the last years; demand for real-time, instant mobility and delivery services grows. This poses new challenges to logistic service providers as the underlying stochastic dynamic vehicle routing problems (SDVRPs) require anticipatory real-time routing actions. Searching the combinatorial action space for efficient routing actions is by itself a complex task of mixed-integer programming (MIP) well-known by the operations research community. This complexity is now multiplied by the challenge of evaluating such actions with respect to their effectiveness given future dynamism and uncertainty, a potentially ideal case for reinforcement learning (RL) well-known by the computer science community. For solving SDVRPs, joint work of both communities is needed, but as we show, essentially non-existing. Both communities focus on their individual strengths leaving potential for improvement. Our survey paper highlights this potential in research originating from both communities. We point out current obstacles in SDVRPs and guide towards joint approaches to overcome them.