inventory routing for dynamic waste collection from underground containers

Inventory Routing for DynamicWaste Collection fromUnderground Containers

Martijn MesDepartment of Operational Methods for Production and LogisticsUniversity of TwenteThe Netherlands

Monday, November 14, 2011INFORMS Annual Meeting 2011, Charlotte, NC

INFORMS Annual Meeting 2011

OUTLINE

Case introduction The company The underground container project Dynamic collection policies

The Inventory Routing Problem Heuristic approach Optimization approach Conclusions

2/41

THE COMPANY

Twente Milieu: a waste collection company located in the Netherlands.

Main activity: collection and processing of waste. But also: cleaning of streets and sewers, mowing of verges,

road ice control, and the control of plague animals. One of the largest waste collectors in the Netherlands when

it comes to the #households connected to their network. Yearly collection of around 225,000,000 kg of waste from a

population of around 400,000 inhabitants.

INFORMS Annual Meeting 2011 3/41

TYPE OF CONTAINERS

Mini containers Block containers

One per household; have to be put along the side of the road on pre-defined days.

One for multiple households; mostly located at apartment buildings; freely accessible.


UNDERGROUND CONTAINERS


ADVANTAGES UNDERGROUND CONTAINERS

Can be used at all places: apartments, houses, business parks, within the city centre etc. (≠ mini containers)

Don’t have to be emptied on pre-defined days (≠ mini containers)

Much larger then the block containers (typically 5m3 which is 5 times the volume of a block container)

Only accessible with a personal card Avoids illegal waste deposits (≠ block containers) Enables the introduction of ‘Diftar’: charging waste disposal at

different rates per kg depending on the type of garbage Less odour nuisance due to solid locking (≠ block containers) Contributes to an attractive environment (≠ block containers)


USING THE UNDERGROUND CONTAINERS

Between 2009 and 2011, around 700 underground containers have been installed; 800 new containers will be added soon.

Containers are equipped with a motion sensor: the number of lid openings are communicated to Twente Milieu.

There is a static cyclic schedule that states which containers have to be emptied on what day. For example: container X has to be emptied every Tuesday and container Y has to be emptied on Friday once in the two weeks.

Every workday, a planning employee assigns trucks and drivers to the pre-defined containers. On Fridays, the planner uses the sensor information to include some additional urgent containers, thereby slightly deviating from the static cyclic schedule.

Why not using this sensor information for the whole selection process?


DYNAMIC WASTE COLLECTION

Dynamic planning methodology: each day, select the containers to be emptied based on their estimated fill levels (using sensor information).

Research objective:To asses in what way and up to what degree a dynamic planning methodology can be used by Twente Milieu to increase efficiency in the emptying process of underground containers in terms of logistical costs, customer satisfaction, and CO2 emissions.


INVENTORY ROUTING PROBLEM

In the literature, our problem is known as a Inventory Routing Problem (IRP) which combines: The vehicle routing problem (VRP) Inventory Management \ Vendor Managed Inventory (VMI)

Trade-off decisions: When to deliver a customer? How much to deliver a customer? Which delivery routes to use?

The current cyclic planning approach relates to the Periodic Vehicle Routing Problem (PVRP): A multi-period VRP where customers have to be visited a

given number of times within a given planning horizon (decision on visit combinations and routes).


ILLUSTRATION OF THE IRP


Basic question for IRPs: which customers to serve today and how to route our trucks?

Parking

Depot

Enough empty space left

Empty space needs to be delivered soon

13/41

SOLUTION METHODOLOGIES FOR IRPs

ILP\SDP\MDP\Heuristics: Federgruen and Zipkin (1984), A Combined Vehicle Routing and Inventory

Allocation Problem. Campbell et al. (1997), The Inventory Routing Problem. Bard et al. (1998), A Decomposition Approach to the Inventory Routing

Problem with Satellite Facilities. Chan et al. (1998), Probabilistic Analyses and Practical Algorithms for

Inventory-Routing Models. Berman et al. (2001), Deliveries in an inventory/routing problem using

stochastic dynamic programming. Kleywegt et al. (2002), The Stochastic inventory routing problem with direct

deliveries. Adelman (2004), A Price-Directed Approach to Stochastic Inventory/Routing. Campbell et al. (2004), A decomposition approach for the inventory-routing

problem. Kleywegt et al. (2004), Dynamic programming approximations for a

stochastic inventory routing problem. Archetti et al. (2007), A branch-and-cut algorithm for a vendor-managed

inventory-routing problem. Bard et al. (2009), The integrated production–inventory–distribution–routing

problem.


OUR SOLUTION METHODOLOGY

Some characteristics of our problem: Multi-vehicle: up to 7 trucks. Multi-depot: 2 parking areas and 1 waste processing center. Large-scale: expanding to 1500 customers (containers),

which requires > 300 visits per day. Long planning horizon: a short-term planning approach will

postpone deliveries to the next period. Dynamic environment: stochastic travel times and waste

disposals → we have to be able to do replanning. Changing environment: seasonal patters and special days.

To cope with these characteristics, we use a fast heuristic. To anticipate changes in waste disposal, we equip our

heuristic with a number of tunable parameters and optimize over these parameters.


BASIC IDEA OF THE HEURISTIC

Create initial routes based on MustGo’s (seed customers and workload balancing) and extend these routes with MayGo’s.


Parking

Depot

MayGo

MustGo

16/41



Parking

Depot

Seed


17/41



Parking

Depot


18/41



Parking

Depot


19/41



Parking

Depot


20/41



Parking

Depot


21/41



Parking

Depot


22/41



Parking

Depot

Extended with MayGo’s


23/41

ALGORITHM OUTLINE

1. Initial planning in the morning and replanning during the day.2. Empty schedules in a non-preemtive way and keep them feasible.3. Estimate the days left; MustGo’s (days left < MustGoDay); optional

workload balancing (to avoid peaks on Mondays and Fridays); trucks to use; lower bound on the number of routes to use.

4. One seed per truck to (i) spread trucks across the area, (ii) realize container insertions both close and far from the depot, and (iii) balance the workload per route to anticipate later MayGo insertions; seeds based on largest minimum distance from the depot and other seeds; Assign routes to trucks.

5. Optionally, assign MustGo’s to trucks or routes in a balanced way (in anticipation of MayGo insertions).

6. Plan all remaining MustGo’s based on cheapest insertion costs.7. Play MayGo’s: see next sheet.8. Execute planning and perform replanning when needed.

1. Start

2. Initialize schedules

3. Initial computations

4. Plan seeds

5. Balance workload

6. Plan MustGo’s

7. Plan MayGo’s

8. End


ADDING MAYGO CONTAINERS

MayGo’s: days left < MustGoDay+MayGoDay. Planning extremes:

Wait first: MayGoDay=0 Drive first: MayGoDay=∞

The best option would be somewhere in between. Selection of MayGo’s depend on the additional travel time

(insertion costs) as well as the inventory (volume garbage). Options:

Ratio insertion costs / inventory. Relative improvement of this ratio compared to a smoothed

historical ratio. A large positive value indicates an opportunity we should take.

Use (optional) limit on the number of MayGo’s.


WILL IT WORK? A SIMULATION STUDY

Benchmark the current way of working and gain insight in the performance of our heuristic


NUMERICAL RESULTS

Based on current deposit volumes and truck capacity, savings of 14.6% can be achieved, which consists of 40% reduction of penalty costs and 18% less travel distance.

Savings increase with decreasing truck capacities.


OBSERVATIONS

Performance heavily depends on the parameter settings: MustGoDay MayGoDay MaxPerDay (to limit MayGo’s) NrTrucks Slack capacity in trucks (to avoid replanning) Etc.

Moreover, the “right settings” for these parameters heavily depend on the day of the week.

We could learn these parameters Through experimentation in practice (online learning) Through simulation experiments (offline learning)


STOCHASTIC SEARCH

Where is the min\max of some multi-dimensional function when the surface is measured with noise?

In our case: at least a 10 dimensional function (using only the parameters MustGoDay and MayGoDay for 5 workdays).


SIMULATION OPTIMIZATION

The optimization problem:

Simulation optimization: The measurements follow from a simulation run. Hence, these measurements are expensive. Hence, we aim to reduce the required number of

measurements. Approaches: Heuristic methods (genetic algorithms, simulated annealing,

tabu search etc.); Response Surface Methods (RSM); Stochastic Approximation (SA) methods; Bayesian Global Optimization (BGO).

xfXx

min


Vector or parameters to be adjusted (MustGoDay, MayGoDay, NrTrucks, etc., for all working days)

Set of all parameter combinations

• Unknown function (no closed-form formulation)

• We can measure it• Measurement will not be

exact (we measure with noise y=f(x)+ε)

30/41

BAYESIAN GLOBAL OPTIMIZATION

Bayesian optimization involves three stages:1. Designing the prior distribution (belief about f)2. Updating this distribution using Bayes' rule3. Deciding what values to sample next

Often, the belief about f conforms to a Gaussian process. A Gaussian process is a collection of random variables {yx1,

yx2,…} for which any finite subset has a joint multivariate Gaussian (Normal) distribution:


xxkNy xx ,,~ Measurements

Mean

Kernel function (covariance between two variables)

31/41

MORE INFORMATION ON BGO

Daniel Lizotte (2008)Practical Bayesian Optimization, PhD Thesis.

Eric Brochu, Mike Cora and Nando de Freitas (2009)A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning.

INFORMS Tutorial by Peter Frazier today from 16:30-18:00Bayesian Methods for Global and Simulation Optimization.


OPTIMIZATION POLICIES WE CONSIDER

Sequential Kriging Optimization (SKO) by Huang et al. (2006) which is an extension of Efficient global optimization (EGO) by Jones et al. (1998) for noisy measurements. EGO: new points to be measured are selected based on “expected improvement” which strikes a balance between exploitation and exploration.

Knowledge Gradient for Correlated Beliefs (KGCB) by Frazier et al. (2009). KG: best we can do given we if there is only one measurement left to make.

Hierarchical Knowledge Gradient (HKG) by Mes et al. (2011). HKG: hierarchical aggregation technique that uses the common features shared by alternatives to learn about many alternatives from even a single measurement.


ILLUSTRATION OF EGO: N=2


Source: Brochu et al. (2009)

34/41




35/41




36/41




37/41

ILLUSTRATION OF HKG [EXCEL DEMO]


APPLICABILITY OF THESE POLICIES


EXPERIMENTS WITH SKO

Experiment 1: 378 containers with 3 trucks:

with a maximum of 113 emptying's per day. Experiment 2: 700 containers, 50% higher deposit volumes

and 2 trucks:

with a maximum of 672 emptying’s per day. Results are counterintuitive at first sight. Still, they result in

additional savings of around 10%.


Mon Tue Wed Thu Fri

MustGoDay 4.0 0.0 0.0 1.2 0.0

MayGoDay 4.0 X X 3.5 X

Mon Tue Wed Thu Fri

MustGoDay 1.0 1.1 1.5 2.7 2.1

MayGoDay 0.0 0.0 4.0 4.0 4.0

40/41

CONCLUSIONS

We proposed a fast heuristic suitable for Inventory Routing Problems involving a large number of customers.

Application of this heuristic to the waste collection problem is expected to result in a reduction of 18% in travel costs and 40% in penalty costs (due to waste overflow).

An optimization approach is preferred to anticipate changes in waste disposals. To enable this, we equipped our heuristic with several tunable parameters.

To optimize over these parameters we used techniques from Simulation Optimization and Bayesian Global Optimization (SKO, KGCB, HKG).

For our waste collection problem, this will result in additional savings of 10% in total costs (travel costs and penalty costs).


QUESTIONS?

Martijn MesAssistant professorUniversity of TwenteSchool of Management and GovernanceOperational Methods for Production and LogisticsThe Netherlands

ContactPhone: +31-534894062Email: [email protected]: http://www.utwente.nl/mb/ompl/staff/Mes/

inventory routing for dynamic waste collection from underground containers

Documents

ncinforms annual meeting

predefined containers

new containers

waste collection company

waste disposal

processing of waste

additional urgent containers

yearly collection