sandia national laboratories 1 modeling banks’ payment submittal decisions walt beyeler 1 kimmo...

SandiaNationalLaboratories

1

Modeling Banks’ Payment Submittal Decisions

Walt Beyeler1

Kimmo Soramäki2

Morten L. Bech3

Robert J. Glass1

1Sandia National Laboratories

2European Central Bank

3Federal Reserve Bank of New York

PAYMENT AND SETTLEMENT SIMULATION SEMINAR AND WORKSHOP

Helsinki August 2005

The views expressed in this presentation do not necessarily reflect those of the Federal Reserve Bank of New York or the Federal Reserve System


2

Orientation

• NISAC is a core partnership of Sandia National Laboratories (SNL) and Los Alamos National Laboratory (LANL), and is sponsored by the Department of Homeland Security's (DHS) Information Analysis and Infrastructure Protection Directorate.

• NISAC program is charged with understanding 14 critical infrastructures and their interactions for U.S. DHS

• We depend on engaging experts who design and operate infrastructures. We've been especially fortunate in developing contacts in banking and finance

• We look for models that capture common features of many infrastructures, and are therefore more abstract than industry models


3

Outline

• Goal

• Model Design

• Formulations of Payment and Funding Decision Rules

• Results

• Future Work


4

Project Goals

• Understand possible responses to unusual conditions

• Try to capture the complex dynamics as adaptive responses to constraints– Does the ability to adapt make systems more robust?– Are adapted states especially dependent on specific

constraints or regularities?– Is adaptation itself a source of novel conditions?


5

Polynet Model Features

• Designed to support models of diverse systems characterized by network interactions

• Defines supporting classes which can be extended and specialized

• Draws on other open libraries

Polynet implementation classes

Domain packagesinfect power

jaxb

polynet

jung.graph

payment

uchicago.src.sim.*java.*

bound.nw.*


6

Components of Payment System Model

• Federal Reserve (RTGS)

• Funds Market

• Banks

• “Treasuries”– Implement specific decision rules– May learn via interaction with …

• Treasury Adaptor


7

Structure Supports Diverse Models of Decision Making

uchicago.src.sim.*

payment

polynet

payment.classifier payment.genetic

jung.graph

Polynet implementation classes

Classes defining the common featuresof payment system operations and decision-making adaptation

Classes implementing specificdecision-making approaches

java.*jaxb

bound.nw.*

payment.heuristic payment.dumb


8

Strategies• Adaptive strategies (learning takes place)

– GENETICBANK is a bank learning through the process of a genetic algorithm

– CLASSIFIERBANK is a bank learning through a classifier system– HEURISTICBANK is a bank that follows the heuristic rules described

• Static reference strategies (no learning)– DELAYBANK is a bank following a pure strategy of delaying all

payments and settling them at the end of the day (with end-of-day funding/defunding)

– ODBANK is a bank that follows the pure strategy of settling all payments immediately (with end-of-day funding/defunding)

– TITFORTATBANK is a bank that sends its first payment immediately and always delays subsequent payments until the time it receives funds (with end-of-day funding/defunding).


9

Hypotheses

• Adaptive banks become better over time– i.e. learning actually takes place. Successive iterations reduce

total costs of settlement for a system consisting of adaptive banks of a type

• Adaptive banks become good in a homogenous environment– a system consisting of trained adaptive banks of a type has

lower average total costs than systems consisting of reference banks

• Adaptive banks become good in a mixed environment– in a system consisting of adaptive banks of a type and reference

banks of any type, adaptive banks become better over time and better than the reference banks


10

Heuristic Bank Decision making

time

Balance / expected payments

borrow

delay

lend

settle

L1

D1

B1

L2

B2

D2


11

Rules for settlement

• Banks settle arriving payments immediately if balance is above line D1-D2 and no payments are in queue

• Banks settle queued payments in FIFO order if balance is above line D1-D2

• Banks place arriving payments at the end of the queue if balance is below line D1-D2


12

Borrowing and lending• Rules for borrowing and lending

– banks post a bid to borrow if balance is below line B1-B2– banks post an offer to lend if balance is above line L1-L2– the amount posted is |balance-threshold| rounded up to the next million– once a bid or offer is made, the bank cannot participate in the market for a given

time-interval* – banks withdraw all unmatched bids and offers if a payment arrives first (and

make a new decision as above)

• Initially bids and offers are given on a fixed interest rate• Subsequently

– The price will be something the banks learn and adapt to– Bids and offers will be matched to form a payment or a series of payments– Unmatched bids and offers will stay on the board until matched or withdrawn

* to prevent too many transactions and at the same time allow for continuous decision making


13

Cost Components

• Delay - proportional to time between arrival and execution using an implicit interest rate that reflects customer displeasure

• Intraday Overdraft - charged continuously at a specified rate• Failure - charged at a specified rate for all payments remaining at

the close• Overnight Overdraft - charged at a specified rate for any negative

balance• Borrowing - paid at a specified funds rate plus a spread and a fixed

transaction cost• Lending - received at a specified funds rate minus a spread plus a

fixed transaction cost


14

Costs and remedies

L1 L2 B1 B2 D1 D2intraday

overnight CB

overnight CB

overnight

funds transaction

Cost Parameter

reduce value increase value

Only effective if lending occurredOnly effective if borrowing occurredOnly effective if payments were delayed

The direction a parameter should be moved in order to decrease a cost

delays

borrowingintraday CB

overnight market

overnight marketlending


15

Adaptations - Parameters

• Banks adjust decision parameters based on the expected effect on the various costs that the parameter influences

• Directions are not sufficient: the expected magnitude of cost changes are needed as well

• Magnitudes are a linear approximation of the cost surface, which should be valid near the operating point

• Decomposing total cost and defining conditional parameter relevance in the costs-remedies table helps deal with some of the more pronounced non-linearities


16

Adaptations – Coefficients

• Weights in the remedies table estimate the expected cost change for a unit change in the parameter

• Parameters are adjusted by:– Finding the estimated change in total cost associated with

increasing and decreasing each parameter– Selecting the parameter and direction that appears to maximize

cost reduction– Updating the weights using the actual cost change observed for

each component

• Uncertainty in the cost change can be included by– Tracking the variance of the prediction error, and simulating a

possible gradient value for each cost component, or– Occasionally taking a random step


17

Adaptive Process

Ranges of

Cost

Reduction

Possible

Cost

Reduction

Best

Parameter

Move

Observe

Effect On

Cost

Change

Parameter

Calculate Total

ChangeSample

Coefficients

Update Coefficients

and Range


18

Results

• Simple system with:– 9 banks– 1500 payments per bank per day– Lognormal payment size, mean = 1, sigma = 1

Funding Mechanism Rate Period

Daylight overdraft 0.36% Duration of overdraft

Overnight overdraft 5% 24 hours

Delay 0.18% Duration of delay

Failure 6% 24 hours

Federal funds 4.5%+transaction fee 24 hours

• Comparison of reflexive strategies with adaptation• Comparison of adapted strategies across banks


19

Performance of Reflexive Strategies

Contributors

Strategy Total Cost Delay Intraday OD Transaction Funding

Delay 0.0018

(0.0022)

21% <1% 7% 72%

Pay 0.0014

(0.0022)

<<1% 2% 8% 90%

Tit-for-Tat 0.0018

(0.0022)

20% <1% 7% 73%

Percentages


20

Adaptation without Gradient Updating

-1.00E-01

-5.00E-02

0.00E+00

5.00E-02

1.00E-01

1.50E-01

2.00E-01

0 20 40 60 80 100 120 140

Delay

OvernightOD

Opp

daylightOD

borrowing

Failure

total

• Example cost trajectories

• Example parameter trajectories

-4

-3

-2

-1

0

1

2

3

0 20 40 60 80 100 120 140

l1

l2

b1

b2

d1

d2

-1.50E-02

-1.00E-02

-5.00E-03

0.00E+00

5.00E-03

1.00E-02

1.50E-02

2.00E-02

2.50E-02

0 20 40 60 80 100 120 140

Delay

OvernightOD

Opp

daylightOD

borrowing

Failure

total

-3

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

0 20 40 60 80 100 120 140

l1

l2

b1

b2

d1

d2


21

Modifications and Refinements

• Heuristic model produced unexpected and counterintuitive behavior, including extensive borrowing and lending by the same bank in the same day, nonconvergent parameter values, and bursts of poor performance after quiescent periods

• We have implemented a succession of refinements to help impose more reasonable behavior, including– Elaborating the cost components to insure monotonicity– Completing the remediation matrix to include side effects– Constraining parameter moves so that cost effects can be better

inferred– Imposing spreads and transaction costs to deter erratic funding

• These refinements improved performance however the behavior is still not completely “rational”


22

Good Results for A Single Learner Costs

-0.06

-0.04

-0.02

0

0.02

0.04

0.06

0.08

0 200 400 600 800 1000 1200 1400

Time (Days)

Co

st (

$)

Daylight Overdraft Cost

TransactionCost

Overnight Overdraft Cost

Net Funding

Failure Cost

Total Cost


23

Good Results for A Single Learner Parameters

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

0 200 400 600 800 1000 1200 1400

Time (Days)

Par

amet

er V

alu

e (F

ract

ion

of

Dai

ly V

olu

me)

l1

l2

b1

b2

d1

d2


24

Good Results for A Single Learner Funding

-800

-600

-400

-200

0

200

400

600

0 200 400 600 800 1000 1200 1400

Time (Days)

Fu

nd

ing

Am

ou

nt

($)

AverageAmountBorrowed

AverageAmountLent


25

Limitations on Adaptation

• Nothing but balance governs decisions

• Response size is fixed and does not depend on cost gradient

• Uncertain environment rich with feedbacks; effects of parameter changes are difficult to discern amid the noise

• Response based on local sensitivities

• Learn on recent experience but forget past


26

Adaptation with Gradient Updating


27

Final Cost Distributions

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.00001 0.0001 0.001 0.01

Cost ($)

Ran

k

Adapted Banks

Delay Total

Prompt Total

Tit for Tat Total


28

Comparison of Adapted Strategies

-0.5

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

0.00001 0.0001 0.001 0.01 0.1

Total Cost ($)

Para

mete

r V

alu

e

l1

l2

b1

b2

d1

d2

-0.5

0

0.5

1

1.5

2

2.5

3

3.5

9 11 13 15 17

Time (Hours)

Pa

ram

ete

r V

alu

e (

Fra

cti

on

)

o

f V

o

Lending

Borrowing

Delay

-0.5

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

9 11 13 15 17

Time (Hours)

Par

amet

er V

alu

e (F

ract

ion

)

o

f V

o Delay

Borrowing

Lending


29

Hypotheses Revisited

• Adaptive banks become better over time– Usually, but usually not permanently.

• Adaptive banks become good in a homogenous environment– Some appear better and some are worse than the

reference reflexive strategies. We are interested in finding out whether some are worse because some are better.

• Adaptive banks become good in a mixed environment– Still looking at this. Against prompt banks, some can

learn to make a profit, but can later forget this skill


30

Preliminary Conclusions

• Cost matrix must be complete and responses should be monotonic, considering all side effects. Deficiencies will be discovered and exploited

• Gradient following is unlikely to lead to a good solution• Simultaneous parameter changes (e.g. raising L2 and lowering B2)

may be needed to reduce costs. The current implementation cannot discover these moves

• Cost function strongly depends on behavior of correspondents• Current balance information alone may not be enough to inform a

cost-minimizing decision• A more robust search is likely to perform better. Neural networks

are appealing because they can shift among modes, and this strategy complements other adaptive methods we have implemented


31

Next Ideas for Heuristic

• Distinguish counterparties and provide for performance awareness

• Reparameterize in terms of average balance and tolerance

• Revisit multiple parameter changes in a single step• Slow parameter adjustment to provide a better estimate

of consequences• Constrain parameter ranges to exclude irrational

combinations, such as funding-dominated solutions• Allow concurrent payment and funding actions when

both may be taken• Adapt parameter change size• Implement robust learning techniques


32

Further Ahead

• Include simple funds market

• Evaluation of less intuitive decision formulations (genetic algorithm, classifier system, etc.)


33

Spares


34

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.00001 0.0001 0.001 0.01

Cost ($)

Ran

k

Comparison with Reflexive Strategies

Pay Delay Tit-for-Tat

sandia national laboratories 1 modeling banks’ payment submittal decisions walt beyeler 1 kimmo...

Documents

bank learning

federal reserve system

hypotheses adaptive

type adaptive banks

european central bank

future work slide

treasury adaptor slide

adaptive responses