algorithmic trading with markov chains

8/6/2019 Algorithmic Trading With Markov Chains

1/29

June 16, 2010

ALGORITHMIC TRADING WITH MARKOV CHAINS

HENRIK HULT AND JONAS KIESSLING

Abstract. An order book consists of a list of all buy and sell o ffers, repre-sented by price and quantity, available to a market agent. The order book

changes rapidly, within fractions of a second, due to new orders being enteredinto the book. The volume at a certain price level may increase due to limitorders, i.e. orders to buy or sell placed at the end of the queue, or decrease

because of market orders or cancellations.In this paper a high-dimensional Markov chain is used to represent the state

and evolution of the entire order book. The design and evaluation of optimalalgorithmic strategies for buying and selling is studied within the theory of

Markov decision processes. General conditions are provided that guaranteethe existence of optimal strategies. Moreover, a value-iteration algorithm ispresented that enables finding optimal strategies numerically.

As an illustration a simple version of the Markov chain model is calibratedto high-frequency observations of the order book in a foreign exchange mar-

ket. In this model, using an optimally designed strategy for buying one unitprovides a significant improvement, in terms of the expected buy price, over anaive buy-one-unit strategy.

1. Introduction

The appearance of high frequency observations of the limit order book in or-der driven markets have radically changed the way traders interact with financial

markets. With trading opportunities existing only for fractions of a second it hasbecome essential to develop effective and robust algorithms that allow for instan-taneous trading decisions.

In order driven markets there are no centralized market makers, rather all par-ticipants have the option to provide liquidity through limit orders. An agent whowants to buy a security is therefore faced with an array of options. One option isto submit a market order, obtaining the security at the best available ask price.Another alternative is to submit a limit order at a price lower than the ask price,hoping that this order will eventually be matched against a market sell order. Whatis the best alternative? The answer will typically depend both on the agents viewof current market conditions as well as on the current state of the order book. Withnew orders being submitted at a very high frequency the optimal choice can changein a matter of seconds or even fractions of a second.

In this paper the limit order book is modelled as a high-dimensional Markov chainwhere each coordinate corresponds to a price level and the state of the Markov chainrepresents the volume of limit orders at every price level. For this model many toolsfrom applied probability are available to design and evaluate the performance ofdifferent trading strategies. Throughout the paper the emphasis will be on what wecall buy-one-unit strategies and making-the-spread strategies. In the first case an

cthe Authors

1


2/29

2 H. HULT AND J. KIESSLING

agent wants to buy one unit of the underlying asset. Here one unit can be thoughtof as an order of one million EUR on the EUR/USD exchange. In the second casean agent is looking to earn the difference between the buy and sell price, the spread,by submitting a limit buy order and a limit sell order simultaneously, hoping thatboth orders will be matched against future market orders.

Consider strategies for buying one unit. A naive buy-one-unit strategy is exe-cuted as follows. The agent submits a limit buy order and waits until either theorder is matched against a market sell order or the best ask level reaches a prede-fined stop-loss level. The probability of the agents limit order to be executed, aswell as the expected buy price, can be computed using standard potential theoryfor Markov chains. It is not optimal to follow the naive buy-one-unit strategy.For instance, if the order book moves to a state where the limit order has a smallprobability of being executed, the agent would typically like to cancel and replacethe limit order either by a market order or a new limit order at a higher level.Similarly, if the market is moving in favor of the agent, it might be profitable to

cancel the limit order and submit a new limit order at a lower price level. Suchmore elaborate strategies are naturally treated within the framework of Markovdecision processes. We show that, under certain mild conditions, optimal strate-gies always exist and that the optimal expected buy price is unique. In addition avalue-iteration algorithm is provided that is well suited to find and evaluate optimalstrategies numerically. Sell-one-unit strategies can of course be treated in preciselythe same way as buy-one-unit strategies, so only the latter will be treated in thispaper.

In the final part of the paper we apply the value-iteration algorithm to find closeto optimal buy strategies in a foreign exchange market. This provides an exampleof the proposed methodology, which consists of the following steps:

(1) parametrize the generator matrix of the Markov chain representing theorder book,

(2) calibrate the model to historical data,(3) compute optimal strategies for each state of the order book,(4) apply the model to make trading decisions.

The practical applicability of the method depends on the possibility to make suf-ficiently fast trading decisions. As the market conditions vary there is a need torecalibrate the model regularly. For this reason it is necessary to have fast calibra-tion and computational algorithms. In the simple model presented in Sections 5and 6 the calibration (step 2) is fast and the speed is largely determined by howfast the optimal strategy is computed. In this example the buy-one-unit strategyis studied and the computation of the optimal strategy (step 3) took roughly tenseconds on an ordinary notebook, using Matlab. Individual trading decision canthen be made in a few milliseconds (step 4).

Today there is an extensive literature on order book dynamics. In this paper theprimary interest is in short-term predictions based on the current state and recenthistory of the order book. The content of this paper is therefore quite similar inspirit to [4] and [2]. This is somewhat different from studies of market impactand its relation to the construction of optimal execution strategies of large marketorders through a series of smaller trades. See for instance [13] and [1].

Statistical properties of the order book is a popular topic in the econophysicsliterature. Several interesting studies have been written over the years, two of


3/29

ALGORITHMIC TRADING WITH MARKOV CHAINS 3

which we mention here. In the enticingly titled paper What really causes largeprice changes?, [7], the authors claim that large changes in share prices are notdue to large market orders. They find that statistical properties of prices dependmore on fluctuations in revealed supply and demand than on their mean behavior,highlighting the importance of models taking the whole order book into account.In [3], the authors study certain static properties of the order book. They findthat limit order prices follow a powerlaw around the current price, suggesting thatmarket participants believe in the possibility of very large price variations withina rather short time horizon. It should be pointed out that the mentioned papersstudy limit order books for stock markets.

Although the theory presented in this paper is quite general the applicationsprovided here concern a particular foreign exchange market. There are many simi-larities between order books for stocks and exchange rates but there are also someimportant differences. For instance, orders of unit size (e.g. one million EUR) keepthe volume at rather small levels in absolute terms compared to stock markets.

In stock market applications of the techniques provided here one would have tobundle shares by selecting an appropriate unit size of orders. We are not awareof empirical studies, similar to those mentioned above, of order books in foreignexchange markets.

Another approach to study the dynamical aspects of limit order books is bymeans of game theory. Each agent is thought to take into account the effect of theorder placement strategy of other agents when deciding between limit or marketorders. Some of the systematic properties of the order book may then be explainedas properties of the resulting equilibrium, see e.g. [10] and [14] and the referencestherein. In contrast, our approach assumes that the transitions of the order bookare given exogeneously as transitions of a Markov chain.

The rest of this paper is organized as follows. Section 2 contains a detaileddescription of a general Markov chain representation of the limit order book. In

Section 3 some discrete potential theory for Markov chains is reviewed and appliedto evaluate a naive buy strategy and an elementary strategy for making the spread.The core of the paper is Section 4, where Markov decision theory is employedto study optimal trading strategies. A proof of existence of optimal strategies ispresented together with an iteration scheme to find them. In Section 5, a simpleparameterization of the Markov chain is presented together with a calibration tech-nique. For this particular choice of model, limit order arrival rates depend onlyon the distance from the opposite best quote, and market order intensities are as-sumed independent of outstanding limit orders. The concluding Section 6 containssome numerical experiments on data from a foreign exchange (EUR/USD) market.The simple model from Section 5 is calibrated on high-frequency data and threedifferent buy-one-unit strategies are compared. It turns out that there is a sub-stantial amount to be gained from using more elaborate strategies than the naive

buy strategy.

2. Markov chain representation of a limit order book

We begin with a brief description of order driven markets. An order drivenmarket is a continuos double auction where agents can submit limit orders. A limitorder, or quote, is a request to to buy or sell a certain quantity together with aworst allowable price, the limit. A limit order is executed immediately if there are


4/29


outstanding quotes of opposite sign with the same (or better) limit. Limit ordersthat are not immediately executed are entered into the limit order book. An agenthaving an outstanding limit order in the order book can at any time cancel thisorder. Limit orders are executed using time priority at a given price and pricepriority across prices.

Following [7], orders are decomposed into two types: an order resulting in animmediate transaction is an effective market order and an order that is not exe-cuted, but stored in the limit order book, an effective limit order. For the restof this paper effective market orders and effective limit orders will be referred tosimply as market orders and limit orders, respectively. As a consequence, the limitof a limit buy (sell) order is always lower (higher) than the best available sell (buy)quote. For simplicity it is assumed that the limit of a market buy (sell) order isprecisely equal to the best available sell (buy) quote. Note that it is not assumedthat the entire market order will be executed immediately. If there are fewer quotesof opposite sign at the level where the market order is entered, then the remaining

part of the order is stored in the limit order book.

2.1. Markov chain representation. A continuous time Markov chain X = (Xt)is used to model the limit order book. It is assumed that there are d N possibleprice levels in the order book, denoted 1 < < d. The Markov chain Xt =(X1t , . . . , X

dt ) represents the volume at time t 0 of buy orders (negative values)

and a sell orders (positive values) at each price level. It is assumed that Xjt Zfor each j = 1, . . . , d. That is, all volumes are integer valued. The state spaceof the Markov chain is denoted S Zd. The generator matrix of X is denotedQ = (Qxy), where Qxy is the transition intensity from state x = (x

1, . . . , xd) tostate y = (y1, . . . , yd). The matrix P = (Pxy) is the transition matrix of the jumpchain of X. Let us already here point out that for most of our results only the

jump chain will be needed and it will also be denoted X = (Xn)n=0, where n is the

number of transitions from time 0. .For each state x S let

jB = jB(x) = max{j : xj < 0},

jA = jA(x) = min{j : xj > 0},

be the highest bid level and the lowest ask level, respectively. For convenience itwill be assumed that xd > 0 for all x S; i.e. there is always someone willing tosell at the highest possible price. Similarly x1 < 0 for all x S; someone is alwayswilling to buy at the lowest possible price. It is further assumed that the highestbid level is always below the lowest ask level, jB < jA. This will be implicit in theconstruction of the generator matrix Q and transition matrix P. The bid price isdefined to be B =

jB and the ask price is A = jA . Since there are no limit

orders at levels between jB and jA, it follows that xj = 0 for jB < j < jA. The

distance jA jB between the best ask level and the best bid level is called thespread. See Figure 1 for an illustration of the state of the order book.

The possible transitions of the Markov chain X defining the order book are givenas follows. Throughout the paper ej = (0, . . . , 0, 1, 0, . . . , 0) denotes a vector in Zd

with 1 in the jth position.

Limit buy order. A limit buy order of size k at level j is an order to buy k unitsat price j . The order is placed last in the queue of orders at price j . It may be


5/29


!" #$ #% #& #! ## #' #( #)!%$

!*

!(

!#

!&

$

&

#

(

*

%$

+,-.,/0112

Figure 1. State of the order book. The negative volumes to theleft indicate limit buy orders and the positive volumes indicatelimit sell orders. In this state jA = 44, jB = 42, and the spread isequal to 2.

interpreted as k orders of unit size arriving instantaneously. Mathematically it isa transition of the Markov chain from state x to x kej where j < jA and k 1.That is, a limit buy order can only be placed at a level lower than the best asklevel jA. See Figure 2.

! " # $ # % # & # ! # # # ' # ( # )!%$

!*

!(

!#

!&

$

&

#

(

*

%$+,-.,/0112

! " # $ # % # & # ! # # # ' # ( # )!%$

!*

!(

!#

!&

$

&

#

(

*

%$+,-.,/01123/45657/089/1,-.,

! " # $ # % # & # ! # # # ' # ( # )!%$

!*

!(

!#

!&

$

&

#

(

*

%$

+,-.,/0112

! " # $ # % # & # ! # # # ' # ( # )!%$

!*

!(

!#

!&

$

&

#

(

*

%$

+,-.,/01123/45657/8.44/1,-.,

Figure 2. Left: Limit buy order of size 1 arrives at level 42.Right: Limit sell order of size 2 arrives at level 45.

Limit sell order. A limit sell order of size k at level j is an order to sell k unitsat price j . The order is placed last in the queue of orders at price j . It may beinterpreted as k orders of unit size arriving instantaneously. Mathematically it isa transition of the Markov chain from state x to x + kej where j > jB and k 1.That is, a limit sell order can only be placed at a level higher than the best bidlevel jB. See Figure 2.

Market buy order. A market buy order of size k is an order to buy k units at the

best available price. It corresponds to a transition from state x to x kejA

. Notethat if k xjA the market order will knock out all the sell quotes at jA, resultingin a new lowest ask level. See Figure 3.

Market sell order. A market sell order of size k is an order to sell k units at thebest available price. It corresponds to a transition from state x to x + kejB . Notethat ifk |xjB | the market order will knock out all the buy quotes at jB, resultingin a new highest bid level. See Figure 3.


6/29


! " # $ # % # & # ! # # # ' # ( # )!%$

!*

!(

!#

!&

$

&

#

(

*

%$+,-.,/0112

! " # $ # % # & # ! # # # ' # ( # )!%$

!*

!(

!#

!&

$

&

#

(

*

%$+,-.,/01123/45,2.6/078/1,-.,

! " # $ # % # & # ! # # # ' # ( # )!%$

!*

!(

!#

!&

$

&

#

(

*

%$

+,-.,/0112

! " # $ # % # & # ! # # # ' # ( # )!%$

!*

!(

!#

!&

$

&

#

(

*

%$

+,-.,/01123/45,2.6/7.88/1,-.,

Figure 3. Left: Market buy order of size 2 arrives and knocksout level 44. Right: Market sell order of size 2 arrives.

Cancellation of a buy order. A cancellation of a buy order of size k at levelj is an order to instantaneously withdraw k limit buy orders at level j from theorder book. It corresponds to a transition from x to x + kej where j jB and1 k |xj |. See Figure 4.

! " # $ # % # & # ! # # # ' # ( # )!%$

!*

!(

!#

!&

$

&

#

(

*

%$+,-.,/0112

! " # $ # % # & # ! # # # ' # ( # )!%$

!*

!(

!#

!&

$

&

#

(

*

%$+,-.,/01123/4564.7758916/1:/0;


7/29


8/29


Define the terminal cost function vT : D R by

vT

(x, y) = 1 if y = 00 otherwiseand let denote the first time (Xn, Yn) hits D. The potential = ((s))sS givenby

(s) = E[vT(X, Y)I{ < } | (X0, Y0) = s],

is precisely the probability of the agents market order being executed before bestask moves to or above J1 conditional on the initial state. To compute the desiredprobability all that remains is to solve (2) with vC = 0.

3.3. Expected price for a naive buy-one-unit strategy. The probability thata limit buy order is executed is all that is needed to compute the expected price ofa naive buy-one-unit strategy. The strategy is implemented as follows:

(1) Place a unit size limit buy order at level J0.(2) If best ask moves to level J1, cancel the limit order and buy at level J1.

This assumes that there will always be limit sell orders available at level J1. Ifp denotes the probability that the limit buy order is executed (from the previoussubsection) then the expected buy price becomes

E[ buy price ] = pJ0 + (1 p)J1 .

Recall that, at the initial state, the agent may select to buy at the best ask pricejA(X0). This suggests that it is better to follow the naive buy-one-unit strategythan to place a market buy order whenever E[ buy price ] < jA(X0). In Section 4more elaborate buy strategies will be evaluated using the theory of Markov decisionprocesses.

3.4. Making the spread. We now proceed to calculate the expected payoff ofanother simple trading strategy. The aim is to earn the difference between the bidand the ask price, the spread. Suppose the order book starts in state X0. Initiallyan agent places a limit buy order at level j0 and a limit sell order at level j1 > j0.In case both are executed the profit is the price difference between the two orders.The orders are placed instantaneously at n = 0 and after the orders are placed thestate of the order book is X0 e

j0 + ej1 . Let J0 and J1 be stop-loss levels such thatJ0 < j0 < j1 < J1. The simple making-the-spread strategy proceeds as follows.

(1) If the buy order is executed first and the best bid moves to J0 before thesell order is executed, cancel the limit sell order and place a market sellorder at J0.

(2) If the sell order is executed first and the best ask moves to J1 before the

buy order is executed, cancel the limit buy order and place a market buyorder at J1.

This strategy assumes that there will always be limit buy orders available at J0and limit sell orders at J1.

It will be necessary to keep track of the positions of the agents limit orders. Forthis purpose two additional variables Y0n and Y

1n are introduced that represent the

number of limit orders at levels j0 and j1 that are in front of and including theagents orders, respectively.


9/29


It follows that Y00 = Xj00 1 and Y

10 = X

j10 + 1, Y

0n is non-decreasing, and Y

1n

is non-increasing. The agents buy (sell) order has been executed when Y0n = 0(Y1

n= 0).

The triplet (Xn, Y0n , Y

1n ) is also a Markov chain with state space S Z

d

{0, 1, 2, . . .} { 0, 1, 2, . . .}. Let P denote the its transition matrix.The state space S is partitioned into two disjoint subsets: S = D D, where

D = {(x, y0, y1) S : y0 = 0, or y1 = 0}.

Let the function pB(x, y0) denote the probability that a limit buy order placed

at j0 is executed before best ask moves to J1. This probability is computed inSection 3.2. If the sell order is executed first so y1 = 0, then there will be a positiveincome of j1 . The expected expense in state (x, y0, y1) for buying one unit is

pB(x, y0)j0 + (1 pB(x, y

0))J1 . Similarly, let the function pA(x, y1) denote the

probability that a limit sell order placed at j1 is executed before best bid movesto J0. This can be computed in a similar manner. If the buy order is executedfirst so y0 = 0, then this will result in an expense ofj0 . The expected income instate (x, y0, y1) for selling one unit is pA(x, y1)j1 + (1 pA(x, y1))J0 . The aboveagument leads us to define the terminal cost function vT : D R by

vT(x, y0, y1) =

j1 pB(x, y

0)j0 (1 pB(x, y0))J1 for y1 = 0

pA(x, y1)j1 + (1 pA(x, y

1))J0 j0 for y0 = 0.

Let denote the first time (Xn, Y0n , Y

1n ) hits D. The potential = ((s))sS

defined by

(s) = E[vT(X, Y0 , Y

1 )I{ < } | (X0, Y

00 , Y

10 ) = s],

is precisely the expected payoff of this strategy. It is a solution to (2) with vC = 0.

4. Optimal strategies and Markov decision processes

The framework laid out in Section 3 is too restrictive for many purposes, as it

does not allow the agent to change the initial position. In this section it will bedemonstrated how Markov decision theory can be used to design and analyze moreflexible trading strategies.

The general results on Markov decision processes are given in Section 4.1 andthe applications to buy-one-unit strategies and strategies for making the spreadare explained in the following sections. The general results that are of greatestrelevance to the applications are the last statement of Theorem 4.3 and Theorem4.5, which lead to Algorithm 4.1.

4.1. Results for Markov decision processes. First the general setup will bedescribed. We refer to [12] for a brief introduction to Markov decision processesand [6] or [9] for more details.

Let (Xn)n=0 be a Markov chain in discrete time on a countable state space S

with transition matrix P. Let A be a finite set of possible actions. Every actioncan be classified as either a continuation action or a termination action. The setof continutation actions is denoted C and the set of termination actions T. ThenA = C T where C and T are disjoint. When a termination action is selected theMarkov chain is terminated.

Every action is not available in every state of the chain. Let A : S 2A bea function associating a non-empty set of actions A(s) to each state s S. Here2A is the power set consisting of all subsets of A. The set of continuation actions


10/29


available in state s is denoted C(s) = A(s) C and the set of termination actionsT(s) = A(s) T. For each s S and a C(s) the transition probability from s tos when selecting action a is denoted P

ss

(a).For every action there are associated costs. The cost of continuation is denoted

vC(s, a), it can be non-zero only when a C(s). The cost of termination is denotedvT(s, a), it can be non-zero only when a T(s). It is assumed that both vC andvT are non-negative and bounded.

A policy = (0,1, . . . ) is a sequence of functions: n : Sn+1 A such that

n(s0, . . . , sn) A(sn) for each n 0 and (s0, . . . , sn) Sn+1. If after n transitions

the Markov chain has visited (X0, . . . , X n), then n(X0, . . . , X n) is the action totake when following policy . In the sequel we often encounter policies where thenth decision n is defined as a function S

k+1 A for some 0 k n. In thatcase the corresponding function from Sn+1 to A is understood as function of thelast k + 1 coordinates; (s0, . . . , sn) n(snk, . . . , sn).

The expected total cost starting in X0 = s and following a policy until termi-

nation is denoted by V(s,). In the applications to come it could be intepreted asthe expected buy price. The purpose of Markov decision theory is to analyze op-timal policies and optimal (minimal) expected costs. A policy is called optimalif, for all states s S and policies ,

V(s,) V(s,).

The optimal expected cost V is defined by

V(s) = inf

V(s,).

Clearly, if an optimal policy exists, then V(s) = V(s,). It is proved inTheorem 4.3 below that, if all policies terminate in finite time with probability 1,an optimal policy exists and the optimal expected cost is the unique solution to

a Bellman equation. Furthermore, the optimal policy is stationary. A stationarypolicy is a policy that does not change with time. That is = (,, . . . ), with : S A, where denotes both the policy as well as each individual decisionfunction.

The termination time of a policy is the first time an action is taken fromthe termination set. That is, = inf{n 0 : n(X0, . . . , X n) T(Xn)}. Theexpected total cost V(s,) is given by

V(s,) = E 1n=0

vC(Xn,n(X0, . . . , X n))

+ vT(X ,(X0, . . . , X ))

X0 = s

.

Given a policy = (0,1, . . . ) and a state s S let s be the shifted policy s =(0,

1, . . . ), where

n : S

n A with n(s0, . . . , sn1) = n(s, s0, . . . , sn1).

Lemma 4.1. The expected total cost of a policy satisfies

V(s,) = I{0(s) C(s)}

vC(s,0(s)) +

sS

Pss(0(s))V(s, s0(s))

+ I{0(s) T(s)a}vT(s,0(s)). (3)


11/29


Proof. The claim follows from a straightforward calculation:

V(s,) = aC(s)vC(s, a) + E

1

n=1 vC(Xn,n(X0, . . . , X n))+ vT(X ,(X0, . . . , X ))

X0 = sI{0(s) = a}+

aT(s)

vT(s,0(s))I{0(s) = a}

=

aC(s)

vC(s, a) + E[V(X1, s)

X0 = s]I{0(s) = a}+

aT(s)

vT(s,0(s))I{0(s) = a}

=

aC(s)vC(s, a) + sS

Pss(a)V(s, s)

I{0(s) = a}

+

aT(s)

vT(s, a)I{0(s) = a}.

A central role is played by the function Vn; the minimal incurred cost beforetime n with termination at n. It is defined recursively as

V0(s) = minaT(s)

vT(s, a),

Vn+1(s) = min

minaC(s)

vC(s, a) +sS

Pss(a)Vn(s), min

aT(s)vT(s, a)

, (4)

for n 0. It follows by induction that Vn+1(s) Vn(s) for each s S. To see this,

note first that V1(s) V0(s) for each s S. Suppose Vn(s) Vn1(s) for eachs S. Then

Vn+1(s) min

minaC(s)

vC(s, a) +sS

Pss(a)Vn1(s), min

aT(s)vT(s, a)

= Vn(s),

which proves the induction step.For each s S the sequence (Vn(s))n0 is non-increasing and bounded below by

0, hence convergent. Let V(s) denote its limit.

Lemma 4.2. V satisfies the Bellman equation

V(s) = min

minaC(s)

vC(s, a) +sS

Pss(a)V(s), min

aT(s)vT(s, a)

. (5)

Proof. Follows by taking limits. Indeed,

V(s) = limn

Vn+1(s)

= min

minaC(s)

vC(s, a) + limn

sS

Pss(a)Vn(s), min

aT(s)vT(s, a)

= min

min

aC(s)vC(s, a) +

sS

Pss(a)V(s), min

aT(s)vT(s, a)

,

where the last step follows by monotone convergence.


12/29


The following theorem states that there is a collection of policies for whichV is optimal. Furthermore, if all policies belong to , which is quite natural inapplications, then V

is in fact the expected cost of a stationary policy

.

Theorem 4.3. Let be the collection of policies that terminate in finite time,

i.e. P[ < | X0 = s] = 1 for each s S. Let = (,, . . . ) be astationary policy where (s) is a minimizer to

a min

minaC(s)

vC(s, a) +sS

Pss(a)V(s), min

aT(s)vT(s, a)

. (6)

The following statements hold.

(a) For each , V(s,) V(s).(b) V is the optimal expected cost for . That is, V = inf V(s,).(c) If , then V(s) = V(s,).(d) Suppose thatW is a solution to the Bellman equation (5) and letw denote

the minimizer of (6) with V replaced by W. Ifw, then W = V.In particular, if all policies belong to , then V is the unique solution to the

Bellman equation (5). Moreover, V is the optimal expected cost and is attainedby the stationary policy.

Remark 4.4. It is quite natural that all policies belong to . For instance, supposethat Pss(a) = Pss does not depend on a A and the set {s : C(s) = } is non-empty. Then the chain will terminate as soon as it hits this set. It follows thatall policies belong to if P[ < | X0 = s] = 1 for each s S, where its firsthitting time of {s : C(s) = }.

Proof. (a) Take . Let Tn = (0,1, . . . ,n1,T) be the policy terminatedat n. Here T(s) = minaT(s) vT(s, a) is an optimal termination action. That is,the policy Tn follows until time n 1 and then terminates. In particular,P[Tn n | X0 = s] = 1 for each s S.

We claim that

(i) V(s, Tn) Vn(s) for each policy and each s S, and(ii) limn V(s, Tn) = V(s,).

Then (a) follows since

V(s,) = limn

V(s, Tn) limn

Vn(s) = V(s).

Statement (i) follows by induction. First note that

V(s, T0) = minaT(s)

vT(s, a) = V0(s).

Suppose V(s, Tn) Vn(s) for each policy and each s S. Then

V(s, Tn+1) =

aT(s)

vT(s, a)I{0(s) = a}

+

aC(s)

vC(s, a) +

sS

Pss(a)V(s, sTn+1)

I{0(s) = a}.

Since sTn+1 = (1, . . . ,n,T) = Tns it follows by the induction hypothesisthat V(s, sTn+1) Vn(s

). The expression in the last display is then greater or


13/29


equal to

aT(s) vT(s, a)I{0(s) = a} + aC(s)vC(s, a) + sSPss(a)Vn(s)I{0(s) = a} min

min

aC(s)vC(s, a) +

sS

Pss(a)Vn(s), min

aT(s)vT(s, a)

= Vn+1(s).

Proof of (ii). Note that one can write

V(s,) = E 1t=1

vC(Xt,t(X0, . . . , X t)) X0 = s

+ E

vT(X ,(X0, . . . , X )) X0 = s,

and

V(s, Tn) = E

n1

t=1 vC(Xt,t(X0, . . . , X t)) X0 = s+ E

vT(Xn,n(X0, . . . , X n))

X0 = s.From monotone convergence it follows that

E 1t=1

vC(Xt,t(X0, . . . , X t)) X0 = s

= E

limn

n1t=1

vC(Xt,t(X0, . . . , X t)) X0 = s

= limn

E

n1

t=1vC(Xt,t(X0, . . . , X t))

X0 = s

.

Let C (0, ) be an upper bound to vT. It follows thatEvT(X,) vT(Xn,n(X0, . . . , X n)) X0 = s 2CE[I{ > n} | X0 = s]

2CP[ > n | X0 = s].

By assumption so P[ n | X0 = s] 0 as n , for each s S. Thisshows that limn V(s, Tn) = V(s,), as claimed.

(b) Note that by (a) inf V(s,) V(s). From the first part of Theorem 4.5below it follows that there is a sequence of policies denoted 0:n with V(s,0:n) =Vn(s). Since Vn(s) V(s) it follows that inf V(s,) V(s). This proves(b).

(c) Take s S. Suppose first (s) T(s). ThenV(s) = min

aT(s)vT(s, a) = V(s,(s)).

It follows that V(s) = V(s,) for each s {s : (s) T(s)}.Take another s S such that (s) C(s). Then

V(s) = vC(s,(s)) +sS

Pss((s))V(s),


14/29


and

V(s,) = vC(s,(s)) + sSPss((s))V(s,).

It follows that

|V(s) V(s,)| sS

Pss((s))|V(s) V(s,)|

=

s:(s)C(s)

Pss((s))|V(s) V(s,)|

= E[|V(X1) V(X1,)|I{ > 1} | X0 = s]

= E[E[|V(X2) V(X2,)|I{ > 2} | X1] | X0 = s]

. . .

E[|V(Xn) V(Xn,)|I{ > n} | X0 = s]

2CP[ > n | X0 = s],

where n 1 is arbitrary. Since P[ < | X0 = s] = 1 the last expressionconverges to 0 as n . This completes the proof of (c).

Finally to prove (d), let W be a solution to (5). That is, W satisfies

W(s) = min

minaC(s)

vC(s, a) +sS

Pss(a)W(s), min

aT(s)vT(s, a)

.

Proceeding as in the proof of (c) it follows directly that W(s) = V(s,w). By (a)it follows that W(s) V(s). Consider the termination regions {s : w(s) T(s)}and {s : (s) T(s)} ofw and . Since W(s) V(s), and both are solutionsto (5) it follows that

{s : (s) T(s)} { s : w(s) T(s)},

and V(s) = minaT(s) vT(s, a) = W(s) on {s : (s) T(s)}. To show equalityfor all s S it remain to consider the continuation region of . Take s {s :(s) C(s)}. As in the proof of (c) one writes

W(s) minaC(s)

vC(s, a) +sS

Pss(a)W(s)

= minaC(s)

vC(s, a) +sS

Pss(a)(W(s) V(s

)) +sS

Pss(a)V(s)

= V(s) +sS

Pss(a)(W(s) V(s

))

= V(s) + s:(s)C(s)Pss(a)(W(s

) V(s))

= V(s) + E

(W(X1) V(X1))I{ > 1} X0 = s

= V(s) + E

(W(Xn) V(Xn))I{ > n} X0 = s.

Since E[(W(Xn) V(Xn))I{ > n} | X0 = s] 0 as n 0 it follows thatW(s) V(s) on {s : (s) C(s)}. This implies W(s) = V(s) for all s Sand the proof is complete.


15/29


In practice the optimal expected total cost V may be difficult to find, and hencealso the policy that attains V. However, it is easy to come close. Since Vn(s)converges to V

(s) a close to optimal policy is obtained by finding one that attains

the expected cost at most Vn(s) for large n.For s S. Let 0(s) be a minimizer of a vT(s, a) and for n 1, n(s) is a

minimizer of

a min

minaC(s)

vC(s, a) +sS

Pss(a)Vn1(s), min

aT(s)vT(s, a)

. (7)

Theorem 4.5. The policy n:0 = (n,n1, . . . ,0) has expected total cost givenbyV(s,n:0) = Vn(s). Moreover, if the stationary policyn = (n,n, . . . ) satisfiesn , then the expected total cost ofn satisfies

Vn(s) V(s,n) V(s).

Proof. Note that 0 is a termination action and V(s,0) = V0(s). The first claimthen follows by induction. Suppose V(s,n:0) = Vn(s). Then

V(s,n+1:0) =

aC(s)

vC(s, a) +

sS

Pss(a)V(s,n:0)

I{n+1(s) = a}

+

aT(s)

vT(s, a)I{n+1(s) = a}

=

aC(s)

vC(s, a) +

sS

Pss(a)Vn(s)

I{n+1(s) = a}

+

aT(s)

vT(s, a)I{n+1(s) = a}

= min

min

aC(s)vC(s, a) +

sSPss(a)Vn(s

), minaT(s)

vT(s, a)

,

= Vn+1(s),

and the induction proceeds.The proof of the second statement proceeds as follows. For n 0 and k 0 let

kn:0 = (n, . . . ,n k times

,n1, . . . ,0).

Then 0n:0 = n1:0. By induction it follows that V(s,kn:0) V(s,

k+1n:0 ). Indeed,

note first that

V(s,0n:0) V(s,1n:0) = Vn1(s) Vn(s) 0.

Suppose V(s,k1n:0 ) V(s,kn:0) 0. Ifs is such that n(s) T(s), then

V(s,kn:0) V(s,k+1n:0 ) = vT(s,n(s)) vT(s,n(s)) = 0.

Ifn(s) C(s), then

V(s,kn:0) V(s,k+1n:0 ) =

sS

Pss(n(s))

V(s,k1n:0 ) V(s,kn:0)

0.

This completes the induction step and the induction proceeds. Since n itfollows that V(s,n) = limk V(s,

kn:0). Indeed,

|V(s,n) V(s,kn:0)| CP(n > k ) 0,


16/29


as k . Finally, by Theorem 4.3,

V(s) V(s,n) = limk

V(s,kn:0) V(s,1n:0) = Vn(s),

and the proof is complete.

From the above discussion it is clear that the stationary policy n converges toan optimal policy and that Vn provides an upper bound for the final expected costfollowing this strategy. In light of the above discussion it is clear that Algorithm4.1 in the limit determines the optimal cost and an optimal policy.

Algorithm 4.1 Optimal trading strategies

Input: Tolerance TOL, transition matrix P, state space S, continuation actionsC, termination actions T, continuation cost vC, termination cost vT.Output: Upper bound Vn of optimal cost and almost optimal policy n.

Let

V0(s) = minaT(s)vT(s, a), for s S.Let n = 1 and d > TOL.while d > TOL do

Put

Vn(s) = min

minaC(s)

vC(s, a) +sS

Pss(a)Vn1(s), min

aT(s)vT(s, a)

,

and

d = maxsS

Vn1(s) Vn(s), for s S

n = n + 1.

end while

Define : S C T as a minimizer to

min minaC(s)

vC(s, a) + sS

Pss(a)Vn1(s), min

aT(s)vT(s, a).

Algorithm 4.1 is an example of a valueiteration algorithm. There are othermethods that can be used to solve Markov decision problems, such as policyiteration algorithms. See Chapter 3 in [16] for an interesting discussion on algo-rithms and Markov decision theory. Typically valueiteration algorithms are wellsuited to solve Markov decision problems when the state space of the Markov chainis large.

4.2. The keep-or-cancel strategy for buying one unit. In this section a buy-one-unit strategy is considered. It is similar to the buy strategy outlined in Section

3.3 except that the agent has the additional optionality of early cancellation andsubmission of a market order. Only the jump-chain of (Xt) is considered. Recallthat the jump chain, denoted (Xn)

n=0, is a discrete Markov chain where Xn is the

state of the order book after n transitions.Suppose the initial state is X0. An agent wants to buy one unit and places a

limit buy order at level j0 < jA(X0). After each market transition, the agent hastwo choices. Either to keep the limit order or to cancel it and submit a market buyorder at the best available ask level jA(Xn). It is assumed that the cancellation


17/29


and submission of the market order is processed instantaneously. It will also beassumed that the agent has decided upon a maximum price level J > jA(X0). Ifthe agents limit buy order has not been processed and j

A(X

n) = J, then the agent

will immediately cancel the buy order and place a market buy order at level J. Itwill be implicitly assumed that there always are limit sell orders available at levelJ. Buying at level J can be thought of as a stop-loss.

From a theoretical point of view assuming an upper bound J for the price levelis not a serious restriction as it can be chosen very high. From a practical point ofview, though, it is convenient not to take J very high because it will significantlyslow down the numerical computation of the solution. This deficiency may becompensated by defining J appropriately large, say larger than J1 plus oneprice tick.

Recall the Markov chain (Xn, Yn) defined in Section 3.2. Here Xn represents theorder book after n transitions and Yn is negative with |Yn| being the number ofquotes in front and including the agents order, at level j0. The state space in this

case is S Zd

{. . . , 2, 1, 0}.Let s = (x, y) S. Suppose y < 0 and jA(x) < J so the agents order hasnot been executed and the stop-loss has not been reached. Then there is onecontinuation action C(s) = {0} representing waiting for the next market transition.The continuation cost vC(s) is always 0. If jA(x) = J (stop-loss is hit) or y = 0(limit order executed) then C(s) = so it is only possible to terminate.

There are are two termination actions T = {2, 1}. If y < 0 the only termi-nation action available is 1 T, representing cancellation of the limit order andsubmission of a market order at the ask price. If y = 0 the Markov chain alwaysterminates since the limit order has been executed. This action is represented by2 T. The termination costs are

vT(s, 1) = jA(x)

vT(s, 2) = j0 .

The expected total cost may, in this case, be interpreted as the expected buyprice. In a state s = (x, y) with jA(x) < J following a stationary policy =(,, . . . ) it is given by (see Lemma 4.1)

V(s,) =

s PssV(s,) for (s) = 0,

jA(x) for (s) = 1,j0 for (s) = 2.

(8)

When s = (x, y) is such that jA(x) = J, then V(s,) = J. It follows immediately

that

j0 V(s,) J,

for all s S and all policies . The motivation of the expression (8) for the expectedbuy price is as follows. If the limit order is not processed, so y < 0, there is no costof waiting. This is the case (s) = 0. The cost of cancelling and placing the marketbuy order is jA(x); the current best ask price. When the limit order is processedy = 0 the incurred cost is j0 ; the price level of the limit order.

The collection of policies with P[ < | X0 = s] = 1 for each s S arethe only reasonable policies. It does not seem desirable to risk having to wait aninfinite amount of time to buy one unit.


18/29


By Theorem 4.3 an optimal keep-or-cancel strategy for buying one unit is thestationary policy , with expected buy price V satisfying, see Lemma 4.2,

V(s) = min minaC(s)

sS

PssV(s), minaT(s)

vT(s)=

min

sS P

ssV(s

),jA(x)

, for jA(x) < J, y < 0,

J, for jA(x) = J, y < 0,j0 , for y = 0.

The stationary policy n in Theorem 4.5 provides a useful numerical approximationof an optimal policy, and Vn(s) in (4) provides an upper bound of the expected buyprice. Both Vn and n can be computed by Algorithm 4.1.

4.3. The ultimate buy-one-unit strategy. In this section the keep-or-cancelstrategy considered above is extended so that the agent may at any time cancel

and replace the limit order.Suppose the initial state of the order book is X0. An agent wants to buy oneunit. After n transitions of the order book, if the agents limit order is locatedat a level j, then jn = j represents the level of the limit order, and Yn representsthe outstanding orders in front of and including the agents order at level jn. Thisdefines the discrete Markov chain (Xn, Yn, jn).

It will be assumed that the agent has decided upon a best price level J0 anda worst price level J1 where J0 < jA(X0) < J1. The agent is willing to buy atlevel J0 and will not place limit orders at levels lower than J0. The level J1 is theworst case buy price or stop-loss. If jA(Xn) = J1 the agent is committed to cancelthe limit buy order immediately and place a market order at level J1. It will beassumed that it is always possible to buy at level J1. The state space in this caseis S Zd {. . . , 2, 1, 0} { J0, . . . , J 1 1}.

The set of possible actions depend on the current state (x,y,j). In each statewhere y < 0 the agent has three options:

(1) Do nothing and wait for a market transition.(2) Cancel the limit order and place a market buy order at the best ask level

jA(x).(3) Cancel the existing limit buy order and place a new limit buy order at

any level j with J0 j < jA(x). This action results in the transition to

jn = j, Xn = x + e

j ej

and Yn = xj 1.

In a given state s = (x,y,j) with y < 0 and jA(x) < J the set of continuationactions is

C(x,y,j) = {0, J0, . . . , jA(x) 1},

Here a = 0 represents the agent being inactive and awaits the next market transitionand the actions j, where J0 j

< jA(x), corresponds to cancelling the outstandingorder and submitting a new limit buy order at level j. The cost of continuation isalways 0, vC(s, 0) = vC(s, j

) = 0. Ify = 0 or jA(x) = J1, then C(s) = and onlytermination is possible.

As in the keep-or-cancel strategy there are are two termination actions T ={2, 1}. If y < 0 the only termination action available is 1, representing can-cellation of the limit order and submission of a market order at the ask price. If


19/29


y = 0 the Markov chain always terminates since the limit order has been executed.This action is represented by 2.

The expected buy price V(s,) from a state s = (x,y,j) with jA

(x) < J follow-ing a stationary policy = (,, . . . ) is

V(s,) =

s PssV(s,) for (s) = 0,

V(sj ,), for (s) = j, J0 j

< jA(x),jA(x) for (s) = 1,jB(X0) for (s) = 2.

In the second line sj refers to the state (x, y, j) where x = x+ejej

, y = xj

1.If s = (x,y,j) with jA(x) = J1, then V(s,) =

J1 . Since the agent is committedto buy at level J0 and it is assumed that it is always possible to buy at level J1 itfollows immediately that

J0 V(s,) J1 ,

for all s S and all policies .By Theorem 4.3 an optimal buy strategy is the stationary policy , with

expected buy price V satisfying, see Lemma 4.2,

V(s) = min

minaC(s)

sS

PssV(s), min

aT(s)vT(s, a)

which implies that

V(s) = min sS

PssV(s), V(sJ0), . . . , V (sjA(x)1),

jA(x)

,

for jA(x) < J1, y < 0, and

V(s) = J, for jA(x) = J1, y < 0,j , for y = 0.

The stationary policy n in Theorem 4.5 provides a useful numerical approximationof an optimal policy, and Vn(s) in (4) provides an upper bound of its expected buyprice. Both n and Vn can be computed by Algorithm 4.1.

4.4. Making the spread. In this section a strategy aimed at earning the differencebetween the bid and the ask price, the spread, is considered. An agent submits twolimit orders, one buy and one sell. In case both are executed the profit is theprice difference between the two orders. For simplicity it is assumed at first thatbefore one of the orders has been executed the agent only has two options aftereach market transition: cancel both orders or wait until next market transition.The extension which allows for cancellation and resubmission of both limit orderswith new limits is presented at the end of this section.

Suppose X0 is the initial state of the order book. The agent places the limitbuy order at level j0 and the limit sell order at level j1 > j0. The orders areplaced instantaneously and after the orders are placed the state of the order bookis X0 e

j0 + ej1 .Consider the extended Markov chain (Xn, Y

0n , Y

1n , j

0n, j

1n). Here Xn represents the

order book after n transitions and Y0n and Y1n represent the limit buy (negative)

and sell (positive) orders at levels j0n and j1n that are in front of and including the

agents orders, respectively. It follows that Y00 = Xj0n

0 1 and Y1

0 = Xj1n

0 + 1, where


20/29


Y0n is non-decreasing and Y1n is non-increasing. The agents buy (sell) order has

been processed when Y0n = 0 (Y1n = 0).

Suppose the agent has decided on a best buy level JB0

< jA

(X0

) and a worst buylevel JB1 > jA(X0). The agent will never place a limit buy order at a level lowerthan JB0 and will not buy at a level higher than JB1, and it is assumed to alwaysbe possible to buy at level JB1. Similarly, the agent has decided on a best sell priceJA1 > jB(X0) and a worst sell price JA0 < jB(X0). The agent will never place alimit sell order at a level higher than JA1 and will not sell at a level lower thanJA0, and it is assumed to always be possible to sell at level JA0. The state space ofthis Markov chain is S Zd {. . . , 2, 1, 0} { 0, 1, 2, . . . } { JB0, . . . , J B11} {JA0+1, . . . , J A1}.

The possible actions are:

(1) Before any of the orders has been processed the agent can wait for the nextmarket transition or cancel both orders.

(2) When one of the orders has been processed, say the sell order, the agenthas an outstanding limit buy order. Then the agent proceeds according tothe ultimate buy-one-unit strategy presented in Section 4.3.

Given a state s = (x, y0, y1, j0, j1) of the Markov chain the optimal value functionV is interpreted as the optimal expected payoff. Note, that for making-the-spreadstrategies it is more natural to have V as a payoff than as a cost and this is howit will be interpreted. The general results in Section 4.1 still hold since the valuefunctions are bounded from below and above. The optimal expected payoff can becomputed as follows. Let VB(x,y,j) denote the optimal (minimal) expected buyprice in state (x,y,j) for buying one unit, with best buy level JB0 and worst buylevel JB1. Similarly, V

A(x,y,j) denotes the optimal (maximal) expected sell price

in state (x,y,j) for selling one unit, with best sell level JA1 and worst sell level JA0.

The optimal expected payoff is then given by

V(s) =

max

sS PssV(s

), 0

, for y0 < 0, y1 > 0,

j1

VB(x, y0, j0), for y1 = 0, y0 < 0,

VA(x, y1, j1) j

0

, for y0 = 0, y1 > 0.

The term

sS PssV(s) is the value of waiting and 0 is the value of cancelling

both orders.In the extended version of the making-the-spread strategy it is also possible to

replace the two limit orders before the first has been executed. Then the possible

actions are as follows.

(1) Before any of the orders has been processed the agent can wait for the nextmarket transition, cancel both orders or cancel both orders and resubmitat new levels k0 and k1.

(2) When one of the orders have been processed, say the sell order, the agenthas an outstanding limit buy order. Then the agent proceeds according tothe ultimate buy-one-unit strategy presented in Section 4.3.


21/29


It is assumed that JB0, JB1, JA0, and JA1 are the upper and lower limits, as above.Then the optimal expected payoff is given by

V(s) =

maxsS PssV(s), max V(sk0k1), 0, for y0 < 0, y1 > 0,j

1

VB(x, y0, j0), for y1 = 0, y0 < 0,

VA(x, y1, j1) j

0

, for y0 = 0, y1 > 0.

In the first line the max V(sk0k1) is taken over all states sk0k1 = (x, y0, y1, k0, k1)

where JB0 k0 < JB1, JA0 < k

1 JA1, x = x + ej0 ek

0

ej1

+ ek1

, y0 = xk0

1,

and y1 = xk1

+ 1. Here k0 and k1 represent the levels of the new limit orders.

5. Implementation of a simple model

In this section a simple parameterization of the Markov chain for the order bookis presented. The aim of the model presented here is not to be very sophisticated

but rather to allow for simple calibration.Recall that a Markov chain is specified by its initial state and generator matrixQ (see for instance Norris [12], Chapter 2). Given two different states x, y S, Qxydenotes the transition intensity from x to y. The waiting time until next transitionis exponentially distributed with parameter

Qx =y=x

Qxy. (9)

The transition matrix of the jump chain, Pxy, denotes the probability that a tran-sition in state x will take the Markov chain to state y. It is obtained from Qvia:

Pxy =Qxy

y=x Qxy.

Recall from Section 2 the different order types (limit order, market order, can-cellation) that dictate the possible transitions of the order book. The model isthen completely determined by the initial state and the non-zero intensities fortransitions rates for (1).

In this secton the limit, market and cancellation order intensities are specifiedas follows.

Limit buy (sell) orders arrive at a distance of i levels from best ask (bid)level with intensity BL (i) (

SL(i)).

Market buy (sell) orders arrive with intensity BM (SM).

The size of limit and market orders follow discrete exponential distributionswith parameters L and M respectively. That is, the distributions (pk)k1and (qk)k1 of limit and market order sizes are given by

pk = (eL 1)eLk, qk = (eM 1)eMk.

The size of cancellation orders is assumed to be 1. Each individual unitsize buy (sell) order located at a distance ofi levels from the best ask (bid)level is cancelled with a rate BC(i) (

SC(i)). At the cumulative level the

cancellations of buy (sell) orders at a distance of i levels from opposite bestask (bid) level arrive with a rate proportional to the volume at the level:BC(i)|x

jAi| (SC(i)|xjB+i|).


22/29


In mathematical terms the transition rates are given as follows.Limit orders:

x x + kej

, j > jB(x), k 1, with rate pkS

L(j jB(x)),x x kej , j < jA(x), k 1, with rate pkBL (jA(x) j),

Cancellation orders except at best ask/bid level:

x x ej , j > jA(x), with rate SC(j jB(x))|x

j |,x x + ej , j < jB(x), with rate

BC(jA(x) j)|x

j |.

Market orders of unit size and cancellations at best ask/bid level:

x x + ejB(x), with rate q1SM +

BC(jA(x) jB(x))|x

jB(x)|x x ejA(x), with rate q1

BM +

SC(jA(x) jB(x))|x

jA(x)|,

Market orders of size at least 2:

x x + kejB , k 2, with rate qkSM.

x x kejA , k 2, with rate qkBM.

The model described above is an example of a zero intelligence model: Transitionprobabilities are state independent except for their dependence on the location ofthe best bid and ask. Zero intelligence models of the markets micro structurewere considered already in 1993, in the work of Gode and Sunder [11]. Despitethe simplicity of such models, they capture many important aspects of the orderdriven market. Based on a mean field theory analysis, the authors of [15] and [5]derive laws relating the mean spread and the short term price diffusion rate to theorder arrival rates. In [8], the validity of these laws are tested on high frequencydata on eleven stocks traded at the London Stock Exchange. The authors find thatthe model does a good job of predicting the average spread and a decent job ofpredicting the price diffusion rate.

It remains to estimate the model parameters from historical data. This is thenext topic.

5.1. Model calibration. Calibration of the Markov chain for the order bookamounts to determining the order intensities and the order size parameters

(L,C,BM,

SM,L,M).

Suppose an historical sample of an order book containing all limit, market andcancellation orders during a period of time T is given. IfNBL (i) denotes the numberof limit buy orders arrived at a distance of i level from best ask, then

BL (i) =

NBL (i)

T

is an estimate of the arrival rate of limit buy orders at that level. The rates SL

,BM, and

SM are estimated similarly.

The cancellation rate is proportional to the number of orders at each level. Toestimate BC(i) (

SC(i)) one first calculates the average number of outstanding buy

(sell) orders at a distance of i levels from the opposite best quote bi. If bit denotes

the number of buy orders i levels from best ask at time t, then

bi =1

T

T0

bitdt.


23/29


24/29


Table 1. Data sample quotes

TIME BID/ASK LEVEL PRICE VOLUME

04:44:20.800 B 1 1.4342 404:44:20.800 B 2 1.4341 804:44:20.800 B 3 1.4340 804:44:20.800 B 4 1.4339 604:44:20.800 B 5 1.4338 6

......

......

...04:44:21.500 A 1 1.4344 204:44:21.500 A 2 1.4345 504:44:21.500 A 3 1.4346 904:44:21.500 A 4 1.4347 604:44:21.500 A 5 1.4348 3

Table 2. Data sample deals

TIME BUY/SELL - ORDER PRICE VOLUME04:44:20.600 B 1.4343 104:44:29.700 B 1.4344 104:44:29.800 S 1.4344 9

6.2. Calibration result. Extracting all limit, market and cancellation orders fromthe data sample enabled calibration according to the procedure explained in Section5.1. In Table 3 we show the result of the calibration. These particular parameterswhere obtained using orders submitted during the 120 minute period on a singleday, from 02:44:20 am to 04:44:20 am. During this period there were a total of14 294 observed units entered into the order book via 10 890 limit orders. In all,there were 13 110 cancelled orders and 1 029 traded units, distributed over 660market orders.

Table 3. Example of market parameters calibrated to a datasample of 120 minutes.

i: 1 2 3 4 5

BL (i) 0.1330 0.1811 0.2085 0.1477 0.0541

SL(i) 0.1442 0.1734 0.2404 0.1391 0.0584

B

C

(i) 0.1287 0.1057 0.0541 0.0493 0.0408

SC(i) 0.1308 0.1154 0.0531 0.0492 0.0437

SM 0.0467 L 0.5667

BM 0.0467 M 0.4955


25/29


26/29


27/29


Initial level choice and expected payoff

! " # $ %& %! %"

!%!

!%&

!$

!#

!"

!!

&

'()*+,-./-%0"1""

'()*+,./%0"

1"!

!

"

#

$

%&

%!

%"

!%!

!%&!$

!#!"

!!&

"01%

"01!

"011

"01"

"012

"01#

"013

"01$

"014

"0"

5-%&!1

'()*+,-./-%0"1"!'()*+,-./-%0"1""

657,8/,97:;8,!-%0"

1

Figure 5. Expected buy price and optimal initial choice for buyorder placement in the ultimate buy-one-unit strategy. The rightplot shows expected buy price given different volumes at price lev-els 1.4342 and 1.4344. Level 1.4343 contains no quotes. The darkgrey in the choice matrix to the left shows that it is always optimalto place a limit order at 1.4342.

Buy order at 1.4342, different volumes at 1.4343 and 1.4344

! " # $ %& %! %"

!%&

!'

&

'

%&

()*+,-./0.%1"2""

()*+,-/0%1"

2"2

!"

#$

%&%!

%"

!%&

!'

&

'

%&

"1!'

"12

"12'

"1"

3.%&!2

()*+,-./0.%1"2""()*+,-./0.%1"2"2

435-60-75896-!.%1"

2

Figure 6. Expected buy price and optimal choice of buy orderplacement in the ultimate buy-one-unit strategy. The buy ordercurrently has place 2 out of a total of 10 limit buy orders at pricelevel 1.4342. The choice matrix to the left shows optimal choicegiven different volumes at levels 1.4343 and 1.4344. Dark greyindicates that the buy order should be kept in place. Light greyindicates that the buy order should be cancelled and resubmitted atlevel 1.4343. In the white region, the buy order should be cancelledand replaced by a market order. The plot to the right shows theoptimal expected buy price.


28/29


Buy order at different positions at 1.4342

! " # $ %& %! %"

!

"

#

$

%&

%!

'()*+,-./-%0"1""

2*(/,3.4,.5(6(75,7

!"

#$

%&%!

%"!

"#

$%&

%!

"018

"01#

"019

"01$

"01:

"0"

;-%&!1

2*(/,3-.4,.5-(6-(75,7'()*+,-./-%0"1""

?,!-%0"

1

Figure 7. Expected buy price and optimal choice of buy order

placement in the ultimate buy-one-unit strategy. In these plots,the total volume at level 1.4342 is 13 and the volume at 1.4343is 1. The plot to the left shows the optimal choice given differentnumber of quotes ahead of the buy order at 1.4342. That is, ifthere are 4 quotes ahead of the buy order, then these 4 orders needto be either cancelled or matched against market orders before thebuy order can be executed. In the dark grey region the buy ordershould be kept at its current position. In the lighter grey region,the order should be moved to level 1.4343, where it is behind only1 order. In the white region finally, the limit order should becancelled and replaced by a market order. The plot to the rightshows the corresponding expected buy price.

(H. Hult) Department of Mathematics, KTH, 100 44 Stockholm, SwedenE-mail address: [email protected]

(J. Kiessling) Department of Mathematics, KTH, 100 44 Stockholm, Sweden

E-mail address: [email protected]


29/29


Three different buy strategies compared

!

"

#!

#"

!#"

!#!

!"

!

!

#

$

%

&

'(#!!)

*+,-./(01(#2&%&$*+,-./(01(#2&%&&

3'4/51/6

4785/

6899/7/:5

!

"

#!

#"

!#"

!#!

!"

!

!2"

#

#2"

$

$2"

%

%2"

&

&2"

'(#!!"

*+,-./(01(#2&%&$

*+,-./(01(#2&%&&

3'4/51/6

4785/

6899/7/:5/

Figure 8. Difference between expected buy price under different

buy strategies. In both plots, level 1.4343 is empty. The plot tothe left shows the difference between the expected buy price underthe keep-or-cancel policy described in Section 4.2 and the ultimatepolicy, for different volumes at 1.4342 and 1.4344. The plot to theright shows the difference in expected buy price between the naivebuy-one-unit strategy and the ultimate policy. It is clear that theoption of early cancellation has a substantial impact on the finalbuy price.

algorithmic trading with markov chains

Documents