The Problem of Calibrating an Agent-Based Model of High-FrequencyTrading

D. F. Platt ∗1,2 and T. J. Gebbie1,2

1School of Computer Science and Applied Mathematics, University of the Witwatersrand, Johannesburg,South Africa

2QuERILab - Quantifying Emergence, Risk and Information

Abstract

Agent-based models, particularly those applied to financial markets, demonstrate the ability to producerealistic, simulated system dynamics, comparable to those observed in empirical investigations. Despitethis, they remain fairly difficult to calibrate due to their tendency to be computationally expensive, evenwith recent advances in technology. For this reason, financial agent-based models are frequently validatedby demonstrating an ability to reproduce well-known log return time series and central limit order bookstylized facts, as opposed to being rigorously calibrated to transaction data. We thus apply an establishedfinancial agent-based model calibration framework to a simple model of high- and low-frequency traderinteraction and demonstrate possible inadequacies of a stylized fact-centric approach to model validation.We further argue for the centrality of calibration to the validation of financial agent-based models andpossible pitfalls of current approaches to financial agent-based modeling.

Keywords: agent-based modeling, calibration, complexity, high-frequency trading, market microstructure

JEL Classification: C13 · C52 · G10

1. Introduction

Agent-based modeling is a relatively new tech-nique that has shown increased prevalencein recent years, with applications across a di-verse array of fields (Macal and North 2010).Agent-based models (ABMs) are characterizedby a bottom-up development process, in whichthe properties of and interactions between themicro-constituents of a system are modeled asopposed to the overall system dynamics (Macaland North 2010).

The appeal of these models can be stronglylinked to the increased focus on the disciplineof complexity science in contemporary litera-ture, in which it is argued that many systemsexhibit emergent properties which cannot beexplained by the properties of their constituentparts, but rather result from the interactions be-tween these constituent parts (Heylighen 2008).Macal and North (2010) indicate that this focuson representing individual agents and theirinteractions within ABMs leads to emergent

behaviors not explicitly programmed, demon-strating an ability to model phenomena whichcould be described as complex.

One should, however, be skeptical of theidea that all complex behavior in a system canbe replicated using a bottom-up modeling ap-proach. Wilcox and Gebbie (2014) propose theidea of hierarchical causality in the financialsystem, stating that there is in fact a hierar-chy representing different levels of abstraction,each leading to different nuances in overall be-havior. This would imply that there is bothbottom-up and top-down causation within thefinancial system and that replicating complexbehavior within it is more nuanced than simplyreplicating the bottom-up processes that maybe present. Nevertheless, financial ABMs rep-resent a significant attempt at the replication ofcomplex behaviors emerging in financial sys-tems.

Considering the fact that overall system be-havior is not explicitly modeled in most cases,ABMs dispense with most assumptions regard-

∗Corresponding author, donovan.platt@students.wits.ac.za

1

arX

iv:1

606.

0149

5v4

[q-

fin.

CP]

23

Mar

201

7

ing overall system dynamics when comparedto other approaches, particularly relevant infinancial applications where common assump-tions, such as the Gaussianity of log returndistributions, market efficiency and rational ex-pectations have become increasingly criticizeddue to incompatibility with empirical obser-vations (LeBaron 2005). Not surprisingly, theabsence of these assumptions has led to theexistence of a wide variety of financial ABMs,all able to replicate empirically-observed re-turn time series stylized facts, such as volatilityclustering and a fat-tailed distribution (Barde2016).

Despite the apparent successes of financialagent-based modeling, there still exist manycritics of the approach, particularly skeptical ofthe extent to which the models have been vali-dated (Hamill and Gilbert 2016). Fabretti (2013)notes that the literature regarding the applica-tion of quantitative calibration techniques tofinancial ABMs remains relatively sparse, withmuch of the remaining literature relying on lessrigorous qualitative calibration techniques, inwhich parameters are manually selected suchthat well-known log return time series stylizedfacts are recovered by the candidate models. Asimilar approach is also employed in modelsincluding central limit order book (LOB) styl-ized facts, such as in the work conducted byPreis et al. (2006).

While the recovery of stylized facts is agood starting point to indicate that a modelmay be based upon sound principles, rigorousvalidation would require a model to producereturn time series with statistical propertiessimilar to those observed in actual transactiondata, such that a simulated series could be saidto come from the same distribution as a seriesdrawn from empirical data (Fabretti 2013). Wethus require model parameters that allow aparticular ABM to reproduce these statisticalproperties. This, in turn, results in a calibrationproblem in which the model must be calibratedto transaction data to obtain these parameters.There is thus an intrinsic link between calibra-tion and validation.

It should be noted that the vast majority of

financial ABM calibration literature involvesthe application of calibration techniques tomodels involving closed-form solutions forprices applicable at a daily time scale. Inessence, prices are determined as weightedsums or averages of individual trader priceexpectations or order prices and sizes duringeach iteration of a simulation representing a se-ries of trading sessions, with a session typicallyrepresenting a single day of trading. Membersof this class of models, representing earlierwork in the field, include the Farmer and Joshi(2002) and Kirman (1991) models, among manyothers.

This simplification is generally appropriatefor the daily time scale considered by thesemodels, since this mechanism could be seenas analogous to the closing auctions found atthe end of each trading day in real financialmarkets, where trader demands, representedas unexecuted limit orders, are considered anda closing price determined which maximizesthe executable volume in the closing auctionperiod. This would typically involve a formof averaging. It is clear, however, that sucha simplification would not be appropriate forintraday time scales and hence high-frequencytrader activity.

A number of successful strategies haveemerged for the calibration of models withclosed-form solutions for prices, with the twomost prominent being the method of simulatedmoments and maximum likelihood estimation.The work of Gilli and Winker (2003) and Fab-retti (2013), on which we base our investigation,details the application of the method of simu-lated moments. Similar experiments involvingmaximum likelihood estimation are presentedin Alfarano et al. (2005), Alfarano et al. (2006)and Alfarano et al. (2007). Also worth notingis the work of Amilon (2008), which presentsa comparison of maximum likelihood and theefficient method of moments.

Despite the pervasiveness of the above twomethods in ABM calibration literature, newmethods and augmentations to existing ap-proaches have also been proposed in recentyears, with computational constraints remain-

2

ing a key obstacle. Notable examples includethe work of Recchioni et al. (2015), which em-ploys gradient methods, the work of Gueriniand Moneta (2016), which estimates vector au-toregressive models from simulated and realworld data and compares their structure, andthe work of Grazzini and Richiardi (2015),which presents a discussion on approaches ableto estimate ergodic models (Grazzini 2012)and in what circumstances such approachesare successful. A fairly exhaustive survey ofexisting ABM calibration methods is presentedby Kukacka and Barunik (2016).

In contrast to models with closed-form so-lutions for prices, contemporary models, suchas those of Preis et al. (2006), Chiarella et al.(2009) and Jacob Leal et al. (2015), focus in-stead on recreating double auction markets atan intraday time scale in which agent ordersare stored in a central LOB and orders are exe-cuted using realistic matching processes. It isapparent, however, that the addition of realis-tic matching processes severely increases thecomputational expense associated with the sim-ulation process, which may have an adverseeffect on calibration attempts. Furthermore,many of the previously mentioned calibrationmethodologies tend to assume and thereforerequire a closed-form solution for prices. Theseare likely reasons why models of this class stillremain largely uncalibrated in a convincingmanner.

It is important to be aware of what the ideal-istic sufficient and necessary conditions are fora model to be considered identifiable, namelythe objective function having a reasonable andunique extremum at the true parameter values(Canova and Sala 2009). Some caution is re-quired in the context of ABMs of the type weconsider as they can at best be approached viaindirect estimation to provide consistency be-tween moments and data as opposed to beingrigorously estimated; hence we retain the termi-nology of calibration (Cooley 1997) as opposedto estimation, in the sense that we attemptto provide consistency between the data andmodel parameters because we cannot directlytest the structure of the models themselves.

The primary focus of this investigation is toexpand upon the work conducted by Fabretti(2013), who introduces a strategy for the cali-bration of the Farmer and Joshi (2002) model,one of the aforementioned daily models witha closed-form solution for prices. Using thisstrategy, we attempt to calibrate the Jacob Lealet al. (2015) model, a more sophisticated intra-day ABM representing a double auction marketbased around a central LOB, with both high-and low-frequency trader activity present. Akey advantage of the method of Fabretti (2013)is that it does not require the candidate modelto have a closed-form solution for prices, mak-ing it compatible with the Jacob Leal et al.(2015) model.

As per the investigation conducted by Fab-retti (2013), our approach involves the con-struction of an objective function using themethod of simulated moments and a mov-ing block bootstrap (Winker et al 2007), fol-lowed by the application of heuristic optimiza-tion methods, namely the Nelder-Mead sim-plex algorithm with threshold accepting (Gilliand Winker 2003) and genetic algorithms (Hol-land 1975). The need for heuristic optimizationmethods stems from the tendency of objectivefunctions in our context to lack smoothness,which causes traditional optimization methodsto find local as opposed to global minima (Gilliand Winker 2003).

The dataset used in calibration consists ofThomson Reuters Tick History (TRTH) datafor a stock listed on the Johannesburg StockExchange (JSE). Prior to a change in the billingmodel towards the end of 2013, the JSE hadminimum transaction costs that made high-frequency trading largely unprofitable (Harveyet al. 2017). We thus apply the aforementionedcalibration methodology in an attempt to gaina deeper understanding of the emergence ofhigh-frequency trading in a developing market.

2. The Jacob Leal et al. Model

Referring to Figure 1, the model consists oftwo agent types, high-frequency (HF) and low-frequency (LF) traders, where each LF trader

3

may be following either a chartist or fundamen-talist strategy. These agents interact throughthe posting of limit orders to a central LOB,with time progressing through a series of Tone-minute trading sessions.

During each session, each active traderagent places a single limit order, with priceand size determined by the trader’s currentstrategy. Thereafter, limit orders in the orderbook are matched according to price and thentime priority until all possible trades have beenexecuted, with the final trade price definingthe market price for the session.

LOB

LF Traders(Chartist,

Funda-mentalist)

HFTraders

MarketPrice Time

Series

Limit OrdersLimit Orders

Match Orders, Calculate Prices

Results in

Figure 1: An illustration of the agent types and interac-tions within the model

DetermineActive LF

Traders

Active LFTraders

PlaceOrders

DetermineActive HF

Traders

Active HFTraders

PlaceOrders

Match Or-ders andCalculateMarketPrice

CalculateHF Trader

Profits

CalculateLF Trader

Profits

UpdateLF TraderStrategies

OpeningAuction,All LFTraders

PlaceOrders

SimulationStart

SimulationEnd

Enter Standard Sessions

Session 1to T − 1

Session T, Exit Standard Sessions

Figure 2: Flowchart describing a typical simulation of Tone-minute trading sessions

It should be noted that at any point in thesimulation, all traders have access to the his-tory of past market prices and fundamentalvalues as well as the current state of the LOB.

In Figure 2, we summarize the key eventsassociated with Jacob Leal et al. (2015) modeltrading sessions. The opening auction is ourown added initialization process, discussed indetail in Section 2.D.

A. LF Trader Agent Specification

Each simulation contains NL LF traders, wheredifferent LF traders are represented by the in-dex i = 1, ..., NL. Each LF trader has its owntrading frequency, θi, drawn from a truncatedexponential distribution with mean θ and up-per and lower bounds θmax and θmin respec-tively. This is described by Jacob Leal et al.(2015) as representing the chronological timeframe associated with LF traders. This timeframe is discussed at length by Easley et al.(2012). At each activation, active LF traders ran-domly select either the chartist or fundamental-ist strategy for the session, with the probabilityof being chartist given by Φc

i,t.We now elaborate on the simulation event

overview presented in Figure 2 with appropri-ate equations for an arbitrary session t ≤ T.

After activation, the i-th LF trader placesa limit order, which requires the specifica-tion of both an order size and an order price.Should the i-th LF trader agent be followingthe chartist strategy, we calculate the order sizeaccording to

Dci,t = αc(Pt−1 − Pt−2) + εc

t (1)

where Pt is the market price (in units of cur-rency) for session t, 0 < αc < 1 and εc

t ∼N(0, (σc)2).Conversely, in the case of the fundamental-

ist strategy, we calculate the order size accord-ing to

D fi,t = α f (Ft − Pt−1) + ε

ft (2)

where Ft = Ft−1(1 + δ)(1 + yt) is the funda-mental price (in units of currency) for session

4

t, δ > 0, 0 < α f < 1, εft ∼ N

(0, (σ f )2

)and

yt ∼ N(0, (σy)2).

The order price is identical under bothstrategies and is determined according to

Pi,t = Pt−1(1 + δ)(1 + zi,t) (3)

where zi,t ∼ N(0, (σz)2).

Once the order is placed, it remains in theLOB until it is matched or until γL sessionshave passed, resulting in its removal from theorder book. At the end of each session, the i-thLF trader calculates their profits according to

πsi,t = (Pt − Pi,t)Ds

i,t (4)

where s = c for the chartist strategy and s = ffor the fundamentalist strategy.

Finally, the i-th LF trader determines theirprobability of being chartist in the next sessionaccording to

Φci,t =

eπci,t/ζ

eπci,t/ζ + eπ

fi,t/ζ

(5)

where ζ > 0 is the intensity of switching pa-rameter.

B. HF Trader Agent Specification

Each simulation contains NH HF traders,where different HF traders are representedby the index j = 1, ..., NH . In contrast to LFtraders, HF traders in the model follow anevent-based activation procedure as opposedto a chronological trading frequency. This isagain consistent with the work of Easley et al.(2012), in which it is suggested that HF tradersfollow an event time paradigm.

In an arbitrary session, t ≤ T, the j-th HFtrader will activate should the following acti-vation condition be satisfied∣∣∣∣ Pt−1 − Pt−2

Pt−2

∣∣∣∣ > ∆xj (6)

where ∆xj is drawn from a truncated uniformdistribution, with support between ηmin andηmax.

Following activation, active HF traders ei-ther buy or sell, where this choice is made ran-domly with equal probability. In a constructionagain consistent with the empirical findings ofEasley et al. (2012), HF trader agents in themodel exploit order book dynamics created byLF trader agents earlier in each trading sessionby setting their order sizes according to ordervolumes present in the order book, in an at-tempt to absorb the orders of LF traders. Thisis represented by each active HF trader in thesimulation setting their order size randomly ac-cording to an exponential distribution 1, withthe mean being the average order volume inthe opposite side of the order book. For anHF trader who is selling units of the asset, theopposite side of the order book would consistof buy orders and vice-versa.

After the setting of the order size as previ-ously discussed, the j-th HF trader sets theirorder price (provided they are active) accord-ing to

Pj,t = Pbidt (1− κJ) (7)

if selling, or according to

Pj,t = Paskt (1 + κj) (8)

if buying, where Pbidt and Pask

t are the best bidand ask in the order book respectively for ses-sion t and κj is a uniform random variable withsupport between κmin and κmax.

Once the order is placed, it remains in theLOB until it is matched or until γH < γL ses-sions have passed, resulting in its removal fromthe order book. Finally, at the end of each ses-sion, the j-th HF trader calculates their profitaccording to:

πj,t = (Pt − Pj,t)Dj,t (9)

where Dj,t is the j-th HF trader’s order size forthe session.

1In the original paper by Jacob Leal et al. (2015), a Poisson distribution is used, where the mean volume is weighted by0 < λ < 1. We retain this weighting by λ, but discussions on why the distribution was changed can be found in Section2.D.

5

C. Matching Engine and MarketClearing

Jacob Leal et al. (2015) indicate that orders areto be matched by price and then time priority.In our implementation, it is noted that after alltrader agents have placed orders in a particularsession, a crossing of the bid-ask spread is typi-cally observed. In other words, Pask

t − Pbidt < 0

after all orders have been placed. This impliesthat the best bid and best ask can immediatelybe matched, resulting in a trade.

98 99 100 101 102 103 104 105 106 107 108Price

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

Volu

me

Sell OrdersBuy Orders

98 99 100 101 102 103 104 105 106 107 108Price

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

Volu

me

Sell OrdersBuy Orders

Figure 3: Illustration of a typical limit order book con-figuration before (top) and after (bottom) ordermatching for an arbitrary session, showcasingan initial crossing of the bid-ask spread

We set the trade price equal to the averageof the prices of the matched buy and sell or-ders and the trade size equal to the size of thesmallest order in the matched pair. We canthus execute all executable limit orders at theend of each session by continually matchingthe best bid to the best ask until the bid-ask

spread is no longer crossed, with the best bidand best ask changing as orders are depleted.Should there be a tie for the best bid or bestask, the order that was placed earliest is al-ways executed first. The market price for eachsession is simply the final trade price for thatsession.

It is evident that the model’s mechanismof batch order placement by all active tradersbefore matching orders at the end of each ses-sion is not realistic. In a true, continuous dou-ble auction market, attempts to match orderswould be made continuously, as orders areplaced, rather than at the end of a specified pe-riod. Markets of this type would also involveadditional order types, such as market orders.

The order matching processes present in themodel are therefore market clearing processesreminiscent of closing auctions at the end of atrading day rather than the continuous match-ing found in double auction markets duringintraday periods. This is, however, fairly stan-dard practice in many other models, notablythat of Preis et al. (2006).

D. Required Modifications

We implement the model as it was originallypresented, with two minor changes.

Firstly, we implement our own method ofinitialization. This stems from the fact thatmodel initialization procedures are not dis-cussed in detail by Jacob Leal et al. (2015). Inorder to be consistent with the opening auc-tions found at the start of each trading dayin actual double auction markets, we beginwith an analogous event in the first simulatedtrading session. In this session, all LF tradersin the simulation place limit orders to buyor sell according to a random initial strategy,where the probability of each LF trader be-ing either chartist or fundamentalist is initiallyequal. This mimics the large spike in activitythat can be observed at the start of a tradingday, an aspect of intraday seasonality (Lehalle2013; Cartea et al. 2015).

Secondly, for the parameters originally sug-gested by Jacob Leal et al. (2015), αc = 0.04,

6

σc = 0.05, α f = 0.04 and σ f = 0.01, as wellas an initial market price of P0 = 100 and aninitial fundamental value of F0 = 100, it wasfound that the model produced LF trader or-der sizes typically less than 1 under both thechartist and fundamentalist strategies. Refer-ring to Eqn. (1), it is evident that even for rel-atively large price changes over a one-minuteperiod (∼ 10 units of currency), the model willproduce order sizes typically less than 1 forthe given parameters. Even if αc and σc wereto be made larger, the fact that αc is boundedabove by 1 means that typical price changes(∼ 1 units of currency) will still result in thesame problematic order size dynamics. Anal-ogous issues apply to Eqn. (2). This in itselfis somewhat problematic, as real markets onlyallow for whole number order sizes.

Furthermore, the fact that HF trader ordersizes were originally set according to a Poissondistribution, with mean dependent on the av-erage order size on the opposite side of the or-der book, resulted in the majority of active HFtrader order sizes being 0 when combined withthese unusual LF trader order sizes. For thisreason, an exponential distribution was usedinstead, allowing for the unusually small meanorder sizes found in the model to remain as iswithout resulting in 0 values for HF trader or-der sizes. Exponential distributions are used todetermine order sizes in many models through-out existing literature, a notable example beinggiven by Mandes (2015). This provides suitablejustification for this choice of distribution.

It should be noted that when matching or-ders, the only consideration with regards toorder sizes is that they be sensible in relationto one another, with actual magnitudes beingof more importance for studies involving tradevolume effects. Thus, for our purposes, thissimple correction is sufficient and these un-usual order sizes will not affect the dynamicsof the obtained market price time series. Amore comprehensive correction of this ordersize issue should be considered in studies inwhich trade volumes and order sizes are im-portant characteristics.

3. Replication of Stylized Facts

Considering the minor changes made, it is es-sential that we demonstrate the ability of ourimplementation of the model to reproduce well-known stylized facts of return time series2 andprovide some evidence of the similarity be-tween our results and those obtained by JacobLeal et al. (2015).

Table 1: Parameters for Stylized Fact Replication

Parameter Value Parameter ValueT 1200 σy 0.01

NL 10000 δ 0.0001NH 100 σz 0.01θ 20 ζ 1

[θmin, θmax] [10, 40] γL 20αc 0.04 γH 1σc 0.05 [ηmin, ηmax] [0, 0.2]α f 0.04 λ 0.625σ f 0.01 [κmin, κmax] [0, 0.01]

Parameters for simulations used to verify the implementedmodel’s ability to replicate well-known return time seriesstylized facts, set according to those specified in Table 4in the work conducted by Jacob Leal et al. (2015)

We provide this required evidence by simu-lating series of prices using the implementedmodel and using these prices to calculate se-ries of log returns. We generate a total of 50price paths (Monte Carlo replications) usingthe default parameters provided by Jacob Lealet al. (2015). We then proceed to construct a his-togram approximating the distribution of theobtained log returns pooled over all the con-ducted Monte Carlo replications, along with afitted normal density function, as well as a Q-Qplot for the same simulated data. Thereafter,we consider the autocorrelation functions of thereturns and absolute values of the returns andprovide the associated 95% confidence bands.

As is evident in Figures 4 and 5, the distri-bution of the simulated log returns is leptokur-tic and fat-tailed when compared to a normaldistribution.

2A study of stylized facts in the context of the JSE is presented by Wilcox and Gebbie (2008).

7

-0.05 -0.04 -0.03 -0.02 -0.01 0 0.01 0.02 0.03Return

0

0.5

1

1.5

2

2.5

3

Freq

uenc

y

×104

ReturnsFitted Normal Density Function

Figure 4: Histogram representing the distribution of thesimulated log returns

-5 -4 -3 -2 -1 0 1 2 3 4 5Standard Normal Quantiles

-0.05

-0.04

-0.03

-0.02

-0.01

0

0.01

0.02

0.03

Qua

ntile

s of

Sim

ulat

ed D

ata

Figure 5: Q-Q plot comparing the distribution of thesimulated log returns to a normal distribution

Referring to Figure 6, the series of simu-lated log returns demonstrates behaviors con-sistent with those presented in the empirical in-vestigations of Cont (2001), with no significantpattern or deviation from zero being observedin the autocorrelation function for almost alllags. When considering Figure 7, however, weobserve initially significant autocorrelation forshorter lags with a steady decline in autocor-relation as the lag increases in the case of theseries of the absolute values of the simulatedlog returns. The latter of these observationscan be taken as evidence of volatility clustering(Cont et al. 1997).

All of the aforementioned properties arepresented as key stylized facts of empirically-

observed return time series by Cont (2001). It istherefore apparent that our implementation ofthe model is able to reproduce well-known logreturn times series stylized facts and generatesbehaviors consistent with those demonstratedby Jacob Leal et al. (2015).

0 200 400 600 800 1000 1200Lag

-0.1

-0.05

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

Auto

corre

latio

n

95% Confidence BandsAutocorrelation Function

Figure 6: Autocorrelation function for the series of logreturns, with one-minute session lags

0 200 400 600 800 1000 1200Lag

-0.1

0

0.1

0.2

0.3

0.4

0.5

Auto

corre

latio

n

95% Confidence BandsAbsolute Autocorrelation Function

Figure 7: Autocorrelation function for the series of theabsolute values of the log returns, with one-minute session lags

It is worth noting that Jacob Leal et al. (2015)consider only the stylized facts of log returntime series in the process of model validation.In general, the stylized facts of the central LOBhave become increasingly important in recentyears and the exclusion of these characteristicsis problematic if relying on stylized facts formodel validation. A comprehensive survey ofLOB stylized facts is provided by Bouchaud et

8

al. (2002). It may also be worth considering theHurst exponent, as has been done by Preis etal. (2006).

4. Calibration Framework

We consider the approach presented by Fabretti(2013). Considering the framework’s successin existing calibration experiments involvingthe Farmer and Joshi (2002) model, we repro-duce it without alteration where possible inan attempt to ascertain if the framework pro-duces equally successful results when appliedto intraday models.

The calibration problem is formally definedas follows:

minθ∈Θ

f (θ) (10)

where θ is a vector of parameters, Θ is thespace of feasible parameters and f is the objec-tive function.

A. Objective Function Construction

We construct the objective function for the con-sidered dataset using the method proposed byWinker et al (2007).

The transaction data set is obtained in theTRTH (Thomson Reuters 2016) format, pre-senting a series of trades, quotes and auctionquotes. We convert the data set to a series ofone-minute price bars, with each price corre-sponding to the final quote mid price for eachminute, where the mid price of a quote is givenas the average of the level 1 bid price and level1 ask price associated with that quote. Fromthis series of prices, we may obtain a series oflog prices 3, which is the series we attempt tocalibrate the model to.

We make use of a combination of k = 5moments and test statistics to represent the sta-tistical properties of the log price time seriespreviously mentioned, remaining consistentwith Fabretti (2013). We use:

1. The mean, representing the central ten-dency of the measured series.

2. The standard deviation, giving an indica-tion of the spread of the measured series.

3. The kurtosis, acting as a measure of themass contained in the tails of the proba-bility distribution of the measured series.

4. The Kolmogorov-Smirnov (KS) test, com-paring the empirical CDFs of the log pricetime series obtained from the transactiondata and another simulated using themodel.

5. The generalized Hurst exponent (Di Mat-teo 2007), with q = 1, representing thescaling properties of the measured series.

We define the:

1. realized log prices, R = {rt}, t = 1, ..., T, tobe a realization of the log price process,drawn from transaction data.

2. exact moments, m = (m1, ..., mk), to be thetrue values of the k selected momentsof the log price process. We cannot pre-cisely know these true values; we canonly estimate their values according toa particular realization of the log priceprocess.

3. estimated moments, me = (me1, ..., me

k), tobe the estimate of m based on R.

4. simulated moments, msi (θ), to be the esti-

mate of the k selected moments for thesimulated series, generated using param-eter set θ, where the index i correspondsto the estimate associated with a particu-lar simulation run.

Using the method of simulated moments,we require:

E[(ms|θ)] = m (11)

which yields:

E[(ms|θ)−m] = 0 (12)3While Fabretti (2013) chooses log prices for convenience, it was found that the weight matrix required by the method

was extremely ill-conditioned when considering either prices or log returns, motivating our choice to retain this convention.

9

Assuming m is given and calculating the re-quired expectation as the arithmetic average:

1I

I

∑i=1

[(msi |θ)−m] = 0 (13)

Using our estimate, me, we set:

G(θ) =1I

I

∑i=1

[(msi |θ)−me] (14)

For most of our experiments, we considerI = 5, with any deviations from this conven-tion stated. This limited number of MonteCarlo replications is selected primarily due tocomputational constraints, since a single sim-ulation is capable of taking a fairly significantamount of time to complete.

Roughly 7 hours of computational time isrequired by each Nelder-Mead simplex experi-ment and roughly 10 hours of computationaltime is required by each of the genetic algo-rithm experiments, despite being parallelizedand run on a relatively powerful desktop com-puter 4.

While this is not ideal, computational diffi-culties remain a major obstacle in ABM calibra-tion (Grazzini et al. 2015) and are difficult tosolve in practice, especially when simulationsare complex and involve realistic matching pro-cesses. Nevertheless, it is important to note thisrelevant caveat.

Our objective function is finally given by:

f (θ) = G(θ)TWG(θ) (15)

where W is a k× k matrix of weights.W is set to be the inverse of the variance-

covariance matrix of me, denoted Cov[me].This selected weight matrix takes the uncer-tainty of estimation associated with me intoaccount and assigns larger weights to momentsassociated with greater uncertainty and vice-versa.

We can estimate Cov[me] by applying amoving block bootstrap (Kunsch 1989) to R

with a window length of b to create n boot-strapped samples and then use these boot-strapped samples to calculate approximate dis-tributions for each estimated moment or teststatistic. Thereafter, we can use these distribu-tions to calculate a covariance matrix 5 approx-imating Cov[me].

For all data sets considered, we set b = 100and n = 10000, again consistent with Fabretti(2013).

B. Heuristic Optimization Methods

The heuristic optimization methods employedin the minimization of the aforementioned ob-jective function, namely the Nelder-Mead sim-plex algorithm with threshold accepting (Gilliand Winker 2003) and genetic algorithm (Hol-land 1975), are presented in Appendix A andAppendix B respectively.

5. Data

The considered dataset is acquired in the TRTH(Thomson Reuters 2016) format, presenting atick-by-tick series of trades, quotes and auctionquotes.

As previously stated, we convert the datasetto a series of one-minute price bars, with eachprice corresponding to the final quote midprice for each minute, where the mid priceof a quote is given as the average of the level 1bid price and level 1 ask price associated withthat quote. From this series of prices, we obtaina series of log prices, which is the series weattempt to calibrate the model to. The rationalebehind this sampling strategy is as follows:

Trades appear more infrequently in thetransaction data than in the model, wheretrades occur every minute. For this reason, thesampling of trade prices becomes problematic,with a large number of the one-minute periodsconsidered in the data containing no trades.Quotes were therefore chosen instead, as wetypically have at least one quote available dur-ing each minute of trading in the transaction

4HP Z840 workstation with 2 Intel Xeon E5-2683 CPUs, providing 2× 16 = 32 workers.5Specifically, we set Cov[me]i,j = Cov[me

i , mej ], where Cov[a, b] corresponds to the empirical covariance of a and b.

10

data.The consideration of quote mid prices is an

attempt to be consistent with the fact that tradeprices in the Jacob Leal et al. (2015) model areessentially also mid prices and the final quotemid price was considered as the market pricefor each minute since the final trade price wasselected as the market price in the model.

It is evident, however, that reconciling anevent-based intraday transaction dataset and amodel that produces a series of chronologically-sampled prices is by no means intuitive andfor this reason the above assumptions werenecessary.

The transaction dataset often presentsevents occurring outside of standard tradinghours, 9:00 to 17:00, but we consider onlyquotes occurring in the period from 9:10 to16:50 on any particular trading day. This is nec-essary as the opening auction occurring from8:30 to 9:00 tends to produce erroneous dataduring the first 10 minutes of continuous trad-ing and the fact that the period from 16:50 to17:00 represents a closing auction.

In all calibration experiments, we considerone-week periods, corresponding to a totalof 2300 one-minute price bars, representing460 minutes of trading each day from Mondayto Friday. We consider a single stock, AngloAmerican PLC, over the period beginning at9:10 on 1 November 2013 and ending at 16:50on 5 November 2013. The associated weightmatrix required by the method of simulatedmoments is presented in Appendix C.

6. Calibration Results

A. Free and Fixed Parameters

For most of our calibration experiments, weconsider 10 free parameters, with the remain-der of the parameters set to be fixed. Morespecifically, we set αc, α f , σc, σ f , σy, δ, σz, λ,NL and NH , defined in Section 2, to be free pa-rameters and randomly generate initial valuesfor these parameters during each calibrationexperiment. We initialize NH and NL between100 and 10000 for each population member or

simplex vertex and initialize the remaining freeparameters with values between 0 and 0.1. Thisparticular set of free parameters was selecteddue to the fact that it represents a broad rangeof model characteristics, including order size,order price and trader population dynamics,while still remaining computationally tractable.The results related to this free parameter setare presented in Sections 6.B to 6.E.

It must be noted that in the above, as isthe case in the work of Fabretti (2013), we con-sider more free parameters (10) than objectivefunction moments (5). This particular featureof the investigation may be open to criticismand must be acknowledged and dealt with toensure the validity and robustness of our con-clusions. We therefore also perform supple-mentary calibration experiments in which weconsider a smaller free parameter set includingonly the following parameters: δ, σz, NH andNL. In other words, we consider fewer freeparameters than objective function momentsand demonstrate that the results obtained inthese experiments are indeed consistent withthose obtained when considering more free pa-rameters than objective function moments. Theresults related to this free parameter set arepresented in Section 6.F.

In both of the above cases, all other param-eters are constant and their values are set tobe identical to those presented in Table 1, withP0 = F0 and P1 set to be the first two prices ofthe measured price time series from which wecalculate the series of measured log prices.

It is important to ensure that the initialvalue to which the simulated price time seriesis set does not happen to be an outlier of theempirically-observed price time series. In orderto demonstrate this, we apply the well-knownboxplot method proposed by Tukey (1977), inwhich we are required to determine the inter-val

[Q1 − 1.5IQR, Q3 + 1.5IQR],

where Q1 is the lower quartile, Q3 is the upperquartile and IQR is the interquartile range ofthe measured price time series.

Applying this method to our processed

11

data set, we obtain the interval

[217.2638, 278.3337],

with any points outside of this interval pre-dicted to be potential outliers. Therefore, ourchosen value of P0 = F0 = 238.75 is well withinthe interval and we can therefore say with rela-tive confidence that our chosen initial price isnot an outlier.

B. Nelder-Mead Simplex Algorithm

Due to the fact that we consider n = 10 freeparameters in our calibration experiments, webegin with n + 1 = 11 simplex vertices, eachconsisting of randomly generated initial valuesfor the 10 free parameters.

In our experiments, it was noted that 250iterations of the Nelder-Mead simplex algo-rithm with threshold accepting almost alwaysproduced convergent behavior and we thusconducted 20 calibration experiments with thisfixed number of iterations. Despite the pres-ence of convergent behavior, the obtained pa-rameter sets were surprisingly different for var-ious calibration experiments, with a significantdependence on the set of initial vertices. Thisis strongly indicative of convergence to localminima, even with the inclusion of methodsaiming to overcome this problem, namely thethreshold accepting heuristic.

More optimistically, however, a fairly signif-icant number of these experiments (more thanhalf) seemed to converge to parameter setswith objective function values in a relativelysmall range, a similar result to that obtainedby Fabretti (2013), indicating that our imple-mentation has, at the very least, produced sim-ilar behaviors when attempting to calibrate themodel. Furthermore, the obtained parametersets in this region of similar objective functionvalues produced model behaviors that werefairly consistent with those demonstrated bythe empirical data, with the error between thesimulated and measured moments, notably themean, standard deviation and Hurst exponent,

being within similar ranges to those obtainedby Fabretti (2013). An assessment of the qualityof the model fit is presented in Section 6.D.

Table 2: Nelder-Mead Simplex Calibration Results

Parameter 95% Conf Int s√n

αc [0.0249, 0.111] 0.0205σc [0.0322, 0.0939] 0.0147α f [0.0299, 0.0597] 0.00711σ f [0.0389, 0.0747] 0.00853σy [0.00689, 0.0547] 0.0114δ [0.000391, 0.00212] 0.000413

σz [0.0159, 0.0429] 0.00643λ [0.0250, 0.0696] 0.0107

NL [2420, 5693] 782.0283NH [3611, 7746] 987.594

95% confidence intervals for the set of free parameters,obtained from 20 independent calibration experimentsinvolving the Nelder-Mead simplex algorithm combinedwith the threshold accepting heuristic

Referring to Table 2, it is apparent that thecalibration process, while able to reduce thesearch region from [0, 0.1] and [100, 10000] tothe regions roughly specified by the confidenceintervals 6 shown, still produces results wherethe vast majority of these intervals are still fartoo large to identify any true convergence toa unique set of optimal parameters. Despitethis general trend, however, we see an interest-ing irregularity. The parameter δ does indeedseem to demonstrate convergent behavior andthe algorithm has identified a very small confi-dence interval for δ. It is clear that this requiresmore thorough investigation and is discussedin detail in Section 6.E.

Therefore, the obtained results are more orless indicative of an ability to find feasible pa-rameter sets, but we find it problematic that wedo not obtain stable or unique parameters andthat there is evidence of a strong dependenceon the initial simplex. It is evident that themethod converges to reasonable solutions, butwe cannot be sure that the obtained solutionstruly minimize the objective function, which

6The intervals are calculated as: x± t∗ s√n , where x is the sample mean, s is the sample standard deviation, n is the

sample size, and t∗ is the appropriate critical value for the t distribution.

12

can be perceived as fairly problematic.

C. Genetic Algorithm

We found that convergence was observed inalmost all of the genetic algorithm experimentsafter approximately 100 generations. For thisreason, we conducted 10 calibration experi-ments, with random initial populations us-ing the same initial parameter range as in theNelder-Mead simplex calibration experiments,and iterated the populations for 100 genera-tions. This smaller number of calibration ex-periments was selected due to the increasedcomputational cost of the genetic algorithm.

Table 3: Genetic Algorithm Calibration Results

Parameter 95% Conf Int s√n

αc [0.0367, 0.0773] 0.00899σc [0.0459, 0.0840] 0.00843α f [0.0234, 0.0706] 0.0104σ f [0.0187, 0.0708] 0.0115σy [0.0332, 0.0932] 0.0133δ [0.000114, 0.00108] 0.000213

σz [−0.00222, 0.0144] 0.00367λ [0.0468, 0.0924] 0.0101

NL [469, 5337] 1075.947NH [3335, 7881] 1004.866

95% confidence intervals for the set of free parameters,obtained from 10 independent calibration experimentsinvolving the genetic algorithm

Referring to Table 3, it is clear that the ob-tained calibration results point to similar con-clusions as were obtained using the Nelder-Mead simplex algorithm. We obtain similarparameter confidence intervals for each cali-brated parameter and again find that all pa-rameters, with the exception of δ, seem unableto be uniquely determined.

D. Comparison of Calibrated Modeland Empirical Data

It was mentioned in the preceding sectionsthat while no unique parameter set could be

established, a number of the experiments con-verged to different parameter sets with similarobjective function values producing reasonablebehavior when compared to the empirical data.In this section, we consider the best parametersets, presented in Table 4, obtained in both theNelder-Mead simplex and genetic algorithmexperiments, associated with Tables 2 and 3respectively.

Table 4: Best Parameter Sets Obtained Using Each Cali-bration Method

Calibrated Parameter θNM+TA θGAαc 0.0433 0.0025σc 0.0233 0.0728α f 0.0068 0.0222σ f 0.0805 0.0228σy 0.0017 0.0021δ 0.00004 0.00002

σz 0.0309 0.0238λ 0.0657 0.0149

NL 9783 9668NH 5150 5949

Best parameter sets obtained through the implementationof Nelder-Mead simplex algorithm combined with thethreshold accepting heuristic and the genetic algorithm

For each of these parameter sets, we sim-ulate 50 price paths and obtain the 95% con-fidence interval for the estimates of the mean,standard deviation, kurtosis, and Hurst expo-nent of the simulated log prices and comparethese to the empirical moments obtained fromthe transaction data.

Referring to Table 5, we see that the cali-bration strategy has resulted in log price pathswith comparable means and standard devia-tions to that observed in the empirical data,along with a reasonable, but slightly underes-timated generalized Hurst exponent estimatefor the Nelder-Mead simplex parameters anda fairly accurate generalized Hurst exponentestimate for the genetic algorithm parameters.This indicates that the calibration experimentsperformed, though not conclusive in determin-ing a unique parameter set, were able to pro-duce reasonable fits for some of the moments.

13

We also see, as was the case in the investiga-tion of Fabretti (2013), that the genetic algo-rithm tends to perform slightly better than theNelder-Mead simplex algorithm with thresh-old accepting.

Table 5: Comparison of Empirical and Calibrated Mo-ments

θNM+TA θGA Datam1 [5.4931, 5.5436] [5.4450, 5.5273] 5.5143m2 [0.0292, 0.0449] [0.0263, 0.0532] 0.0300m3 [2.1092, 2.6760] [1.9029, 2.6317] 1.2402m4 [0.5110, 0.5239] [0.5551, 0.5826] 0.5658

95% confidence intervals for simulated moments obtainedusing calibrated parameters compared to the empiricallymeasured moments, with m1, ..., m4 corresponding to themean, standard deviation, kurtosis and generalized Hurstexponent respectively

In contrast to this, the kurtosis is more sig-nificantly overestimated. This seems to stemfrom flaws in the calibration methodology. Inboth our own investigation and that of Fab-retti (2013), the kurtosis tended to be the mostpoorly fit moment after model calibration. Thisis likely explained by the fact that both ourown weight matrix and that of Fabretti (2013)tends to assign a very small weight to errors onthe kurtosis. Considering that the method em-ployed was reproduced without alteration, itmay be necessary for the choice of moments orthe construction of the weight matrix to be re-visited in future, since both our weight matrixand that of Fabretti (2013) tends to be signifi-cantly biased towards finding good fits for themean and standard deviation at the expense ofthe other considered moments. A possible al-ternative to the currently constructed objectivefunction is the information theoretic criterionproposed by Lamperti (2015) and applied inLamperti (2016).

This has, however, demonstrated that themethod in question has indeed found reason-able fits for the most significantly weightedmoments in the objective function.

E. Parameter Analysis

In an attempt to illustrate the behavior of theobjective function with respect to changes inparameter values, we generate surfaces rep-resenting the objective function values corre-sponding to various parameter value pairs, ashas been done by Gilli and Winker (2003) andFabretti (2013). The process of generating thesesurfaces is as follows:

1. We choose any pair of parameters drawnfrom the 10 free parameters selected inthe calibration experiments.

2. We then define the space of possible val-ues for each parameter in the pair. Ingeneral, we set the space to be [0, 0.1] onR for all parameters, with the exceptionof NL and NH , where we set the space tobe [100, 10000] on R.

3. This results in a two-dimensional spacewhose x-axis is defined by the possiblevalues of one of these selected parame-ters and whose y-axis is similarly definedby the possible values of the other param-eter.

4. We then populate this two-dimensionalspace by generating 1000 points using anappropriate two-dimensional Sobol se-quence. In the case of NL and NH , weround the values of the obtained points.

5. These randomly generated parameter val-ues are then used to simulate 5 price andhence log price time series (Monte Carloreplications) using the Jacob Leal et al.(2015) model, with all other free param-eter values set according to θNM+TA inTable 4.

6. We can thus evaluate the objective func-tion at each of the generated points andobtain an approximate surface of objec-tive function values corresponding to allpossible values of the parameters in thepair using cubic interpolation.

After the consideration of surfaces for anumber of parameter pairs, we found that

14

there are 3 main types of surface behaviorsthat emerge, with only one of these behaviorsresulting in satisfactory calibration results.

-500

0

8000

500

f(θ)

1000

6000 8000

NH

1500

6000

NL

4000

2000

40002000 2000

Figure 8: Objective function surface demonstrating type1 behavior: NH and NL

Referring to Figure 8, illustrating what wewill call type 1 behavior, we see that changesin NL and NH have a haphazard effect on theobjective function, with very little evidence ofstructure. It is evident that even slight changesin these parameters have the potential to re-sult in fairly large relative differences in objec-tive function values, leading to a surface domi-nated by a series of peaks and valleys, with noclear path to a global minimum or discernabletrend emerging. This is the likely reason thatboth the Nelder-Mead simplex algorithm withthreshold accepting and genetic algorithm hada tendency to converge to local minima, evenwith attempts made in both search methodsto avoid this. It seems very unlikely that themodel can be calibrated in a meaningful waywith respect to parameters demonstrating thistype of behavior, as the observed degeneraciesare fairly severe, with such parameters generat-ing unpredictable, near-random effects on theconsidered objective function. Other param-eters generating similar behaviors include αc,α f , σc, σ f and λ.

Referring to Figure 9, illustrating what wecall type 2 behavior, we see that there is indeeda trend presented by σy, albeit with many localminima and maxima obscuring this trend, re-sulting in a fairly noisy surface with some formof overall path to a global minimum emerging.

While this type of behavior is indeed morepromising than the first, both optimizationmethods failed to demonstrate any real con-vergence with respect to σy. This is most likelydue to the presence of a very large numberof local minima and a lack of smoothness hid-ing the observed trend from the optimizationmethods considered.

-0.5

0

0.5

1

0.08

1.5

2

f(θ)

×106

0.06 0.08

2.5

λ

3

0.06

3.5

σy

0.04

4

0.040.02 0.02

0 0

Figure 9: Objective function surface demonstrating type2 behavior: λ and σy

Finally, referring to Figure 10, illustratingwhat we will call type 3 behavior, we see that δdemonstrates a very well-behaved effect on theobjective function, resulting in a very smoothsurface, far more amenable to calibration. Be-ing the parameter that best demonstrates thiskind of behavior, it is not surprising that itwas the only parameter that demonstrated trueconvergence in the calibration experiments con-ducted.

It should be noted that the effect of σz is farless significant than that of δ, but still appearsto result in well-defined effects on the objectivefunction, observable when δ is close to 0. Inthe calibration results previously presented, σz

typically performed much better than the otherparameters, with the exception of δ, having amuch smaller confidence interval, but likelydid not show true convergence since the effectof δ on the objective function is extremely dom-inant, with σz only producing a noticeable ef-fect when δ is small, but still larger than valuesto which δ tends to converge in the calibrationexperiments.

Calibration experiments involving a fixed

15

δ may result in convincing convergence for σz,but such investigations are beyond the scopeof this investigation.

-5

0

0.08

5

f(θ)

×107

10

0.06 0.08

δ

15

0.06

σz

0.04

20

0.040.02 0.02

0 0

Figure 10: Objective function surface demonstratingtype 3 behavior: δ and σz

A key realization is the fact that the effectsof the other parameters on the objective func-tion are small in comparison to the effect ofδ. Parameters demonstrating type 1 behavior,as shown in Figure 8, tend to result in objec-tive function values of no more than 5000 fortheir worst possible combinations, absolutelyinsignificant in comparison to δ, resulting invalues in excess of 2 × 107 if chosen poorly.Though σy produces a significant effect on theobjective function, the worst values of δ stillproduce objective function values more than10 times that of the worst values of σy.

This is somewhat disconcerting, as it pointsto one of two possible conclusions.

The first, which we tend to lean toward, isthat model itself may be flawed in some re-spects. To substantiate these claims, 7 of the 10free parameters we considered demonstratedwhat we referred to as type 1 behavior, pro-ducing haphazard and predominantly smalleffects on the objective function values. In con-trast to this, δ and to a lesser extent σy andσz, demonstrate far more significant effects onthe objective function, which seem to have re-sulted in δ dominating the calibration process,leading it to be the only successfully calibratedparameter. This seems to indicate that 7 of theparameters considered demonstrate some form

of parameter degeneracy, resulting in a modelthat is primarily driven by a single parameter,δ. This is further illustrated by the fact that sim-ilar objective function values were obtained formany of the calibration experiments conductedand that these in general resulted in a reason-able fit to the empirical data. This suggeststhat choosing δ well may be enough to obtainreasonable calibration, at least with regard tothe moments considered. We suggest possiblereasons for this in Section 7.

Alternatively, it may simply be a case thatthe method of calibration, more specifically theweight matrix, is flawed in some respects, re-sulting in these observed behaviors. While wedo indeed acknowledge that there are flaws inmethod of Fabretti (2013) and hence our own,the successful use of the methodology to cali-brate the Farmer and Joshi (2002) model andthe fact that we were able to obtain reasonablefits to empirical data and obtain convergencein the parameter δ seems to negate these con-cerns. We address these issues in more detailin Section 7.

F. Supplementary Calibration Experi-ments

In this section, we address potential criticismrelating to the approach taken to calibrate themodel, such that we are able to ensure thatour conclusions are robust. Specifically, weaddress the fact that we attempt to calibrate10 free parameters in the preceding calibrationexperiments, but employ only 5 moments inour objective function.

In this context, it is important to realizethat the work of Fabretti (2013) and the bodyof work on which it is based, namely the workof Gilli and Winker (2003), similarly employsfewer moments than free parameters in theconducted estimation procedures.

Nevertheless, not addressing such criticismwould place significant doubt on whether theconclusions reached in this investigation arerobust or simply the result of experimentaldesign. We therefore repeat our calibrationexperiments, as before, but this time consider

16

a smaller free parameter set and apply onlythe Nelder-Mead simplex algorithm combinedwith the threshold accepting heuristic.

In these experiments, we set NH , NL, δ andσz to be the only free parameters and verifythat δ once again emerges as a more easilyidentifiable parameter than the remaining freeparameters and that the associated objectivefunction surface once again demonstrates type3 behavior. In addition to this, we verify thatNH and NL are once again difficult to uniquelyidentify and that the associated parameter sur-face again demonstrates type 1 behavior, in-dicative of parameter degeneracies.

Table 6: Nelder-Mead Simplex Calibration Results (Re-duced Free Parameter Set)

Parameter 95% Conf Int s√n

δ [0.0016, 0.0063] 0.0011σz [0.0238, 0.0452] 0.0051NL [835, 3696] 683.5941NH [2082, 8935] 1637.0516

95% confidence intervals for the reduced set of free pa-rameters, obtained from 20 independent calibration ex-periments involving the Nelder-Mead simplex algorithmcombined with the threshold accepting heuristic

Table 7: Best Parameter Set Obtained Through Calibra-tion (Reduced Free Parameter Set)

Calibrated Parameter Valueδ 0.000008

σz 0.0001NL 1005NH 7299

Best parameter set obtained through the implementationof the Nelder-Mead simplex algorithm combined with thethreshold accepting heuristic in experiments involvingthe reduced free parameter set

As is clearly evident in Table 6, we observethat δ once again emerges as showing someevidence of convergence, with a much smallerconfidence interval and standard error than theremaining free parameters. Furthermore, refer-ring to Figures 11 and 12, we observe that NHand NL once again result in a type 1 objective

function surface, though slightly different tothat observed in Figure 8, while δ and σz onceagain result in a type 3 objective function sur-face that is nearly identical to that presentedin Figure 10. This is in spite of the fact thatthe calibration experiments were repeated withfewer free parameters than objective functionmoments.

-2000

0

2000

4000

8000

6000

8000

f(θ)

6000

10000

8000

NH

12000

6000

14000

NL

4000

16000

40002000 2000

Figure 11: Objective function surface illustrating type 1behavior (NH and NL), generated using theparameter values in Table 7 as the base valuesfor the free parameters, with all other param-eters set according to the values specified inTable 1

-5

0

0.08

5

f(θ)

×107

10

0.06 0.08

δ

15

0.06

σz

0.04

20

0.040.02 0.02

0 0

Figure 12: Objective function surface illustrating type3 behavior (δ and σz), generated using theparameter values in Table 7 as the base valuesfor the free parameters, with all other param-eters set according to the values specified inTable 1

It is therefore evident that our results can-not be explained as a mere consequence of the

17

relative number of moments and parametersemployed in the calibration procedures.

7. Discussion of Results

As was shown in Section 6, αc, σc, α f , σ f , λ,NL and NH demonstrate evidence of parameterdegeneracies and have a haphazard effect onthe objective function, while σy and σz presentevidence of an observable objective functiontrend and δ demonstrates a clear relationshipbetween its values and the objective functionvalue. This dominance of δ on the model dy-namics may be a direct result of the model’sconstruction:

Firstly, referring to Eqn. (3), we see that LFtrader order prices for a particular session areset according to a single iteration of a geomet-ric random walk referencing the market priceof the previous session. Not surprisingly, δ isan important parameter in determining the na-ture of the aforementioned geometric randomwalk and hence δ plays an important role in thesetting of LF trader order prices in the model.Furthermore, HF trader orders expire at theend of each session (γH = 1) and HF traderorder prices are set to be slight perturbationsof existing LF trader orders in the order book,hence we see that δ would have a significantimpact on HF trader prices in each session aswell.

Secondly, referring to Eqn. (2), we see thatboth δ and σy, another parameter shown todemonstrate structure in the calibration exper-iments, drive the equation which determinesthe fundamental value. Because order sizes un-der both the chartist and fundamentalist strate-gies are predominantly determined by the mar-ket price and fundamental value time series,both of which are driven by or heavily influ-enced by geometric random walks including δas a parameter, we see that δ has a significanteffect on both the model’s order size and pricedynamics. This is in contrast to the remain-ing parameters, which typically only influenceone aspect of the model. This leads to a fairlysignificant coupling of dynamics in the model,making δ far more important than the remain-

ing model parameters and possibly leading itto dominate the calibration experiments as ithas. Chakraborti et al. (2011) state that it isoften difficult to identify the role played byand dependencies between the various param-eters present in ABMs, despite their ability toreproduce well-known stylized facts. Our in-vestigation has demonstrated a similar finding.

We see that σy, σz and δ, the only param-eters demonstrating any structured effect onthe objective function in the calibration experi-ments, appear in two geometric random walksincluded in the model’s equations. It may bethat these random walk dynamics dominatethe model, with the parameters not consideredin these random walks degenerating to noise.It was previously shown in Section 6.E that theeffects of σy, σz and δ on the objective func-tion are far more significant than that of theremaining parameters, again pointing to a simi-lar conclusion. This suggests that the inclusionof a random walk may be a far stronger anddominant model assumption than one may per-ceive it to be at first glance.

These observed degeneracies may also findtheir origin in the more realistic matching pro-cesses used to determine market prices in in-traday ABMs of double auction markets, incontrast to the approximating equations usedin daily models, which have been more suc-cessfully calibrated by Fabretti (2013) and Gilliand Winker (2003). Perhaps the indirect calcu-lation of prices through order matching doesnot lead to the same well-defined dynamicsas an approximating equation that considersthe effects of all parameters with simple as-sumptions regarding their relationship? It isalso worth considering why only parametersdealing with order price dynamics, δ and σz,resulted in a smooth objective function surface.

The exact nature and cause of these de-generacies requires further investigation andwould likely require the consideration of othermodels also involving realistic order matchingprocesses, but with a different approach to thatof Jacob Leal et al. (2015), since it is possiblethat degeneracies may emerge due to the cur-rent LF trader order price mechanism.

18

As an example, one might consider thePreis et al. (2006) model, also involving realisticmatching processes in a continuous double auc-tion market, but using a market microstructure-based order placement mechanism as opposedto a random walk determining order prices. Ingeneral, this type of order placement mecha-nism is more desirable, since the use of a ran-dom walk by Jacob Leal et al. (2015) is unintu-itive and results in orders being placed withoutany consideration of the dynamics of the orderbook. Considering the importance of price im-pact in contemporary literature (Lehalle 2013;Cartea et al. 2015), order placement behaviorsmore directly rooted in the dynamics of theorder book are worth investigating.

It is evident through the calibration exper-iments conducted that the model or perhapseven this particular class of continuous doubleauction models present a number of obstaclesthat render them difficult to calibrate defini-tively. This, in essence, presents a number ofchallenging questions. How can we be surethat the parameter NH truly influences the na-ture of HF trading in a simulated market ratherthan introducing noise into a random walkmodel? Furthermore, how can we trust theconclusions drawn by Jacob Leal et al. (2015) re-garding the parameter NH and the prevalenceof flash crashes, when NH seems to be a degen-erate parameter? It should also be noted thatFigure 8, demonstrating the presence of manyvastly different combinations of NL and NHresulting in similar objective function values,suggests that answering the question raised atthe start of this investigation, namely the ex-tent to which HF trading strategies have beenadopted in South Africa, is essentially impos-sible, as the values obtained for NL and NHdemonstrate almost no structure in general.

Being unable to obtain unique calibrationsfor a particular model rigorously is in fact amore serious issue than it is presented as inmuch of the literature. Without a unique setof parameters allowing us to reproduce theproperties of financial time series drawn froma particular market, how can we argue that in-troducing and observing changes in the model

truly reflects the same changes in the market?It is important, however, to remember that

we have attempted to calibrate the model toa single week of data for a particular stocktraded on a particular market. Therefore, theabove are not definitive conclusions, but ratherdiscussions related to potential calibration chal-lenges identified by this particular investiga-tion. In future, it would certainly be worthapplying similar methodologies to other intra-day datasets to verify whether this behavioris common or isolated. It may very well bethat other data sets yield more well-defined be-haviors with respect to important parameters,such as NH .

Of course, it is entirely possible, but in ouropinion unlikely, that the aforementioned con-cerns result from the calibration methodologyand not the model itself. Firstly, it must benoted that the weight matrix, shown in Ap-pendix C, was obtained by inverting a matrixthat is non-singular, but only just. Fabretti(2013) also performs the aforementioned inver-sion of a near-singular matrix, though we in-verted a less ill-conditioned matrix. This couldof course be used to explain the haphazardbehaviors demonstrated by some of the param-eters, but if and only if this was a general trend.As we have shown, this same weight matrixresults in fairly well-defined behaviors with re-gard to the parameter δ, making this argumentharder to justify. It should be noted that the cal-ibration experiments did not fail as such. Whilemost of the parameters could not be uniquelyidentified, δ showed evidence of convergenceand the obtained fits of the model to the em-pirical data, shown in Section 6.D, were in factrelatively good, more or less in line with thatof Fabretti (2013), who was considered to havebeen successful. It is also worth noting thatwe have not deviated from the methodology ofFabretti (2013), regarded as a valid calibrationapproach.

In addition to the above, we have also ad-dressed the criticism that the number of freeparameters considered in the calibration experi-ments exceeds the number of moments used toconstruct the objective function in Section 6.F.

19

If, in the continuous time limit and under theassumptions of ergodicity and stationarity, thenumber of moments in the objective functionis at least the same as the number of parame-ters in model to be calibrated, then there canbe perfect model parameter identification andthe parameter values found are the solutionthat minimizes the objective function (Canovaand Sala 2009). This is not in general possible,in part due to the large number of parameterscharacteristic of the class of models considered.

Under-identified moment estimators mustbe approached with care (Canova and Sala2009), however, the problem of estimation ofan ABM can reside with the model itself ratherthan the estimation method when a subset ofthe parameters can be well-identified – a subsetwith a clear and specific physical interpreta-tion. This may imply that calibration (which isweaker than estimation because of the auxiliaryassumptions made) can be used to determinewhether a model is appropriate or not. Themodel we have investigated demonstrates de-generacies even when considering fewer freeparameters than moments in separate experi-ments (where the same subset of parametersremained identifiable), which we consider suf-ficient to negate concerns related to over- andunder-identification (Bollen 1989; Rigdon 1995;Canova and Sala 2009) and sufficient to ques-tion the model specification, even though themodel is able to recover various stylized facts.

Regardless of one’s stance on whether thebehavior observed in this investigation is iso-lated, or indicative of general trends in rules-based ABMs involving realistic order-matchingprocesses at an intraday time scale, we havedemonstrated a concern. While Section 3demonstrates the ability of our implementa-tion of the model to replicate a number of es-tablished log return time series stylized facts,this did not account for the fact that the modeldemonstrated evidence of parameter degenera-cies and appeared to be driven predominantlyby a single parameter, with much of the re-maining processes tending towards noise. Thisis testament to a problematic oversight thatpermeates the breadth of much of the contem-

porary literature on the subject: the sufficiencyof stylized facts as a form of validation. Theperceived sufficiency of stylized facts as a formof validation is so pervasive that Panayi et al.(2013) suggest it is more or less standard prac-tice in financial agent-based modeling. Whilea model must replicate stylized facts to be vali-dated, the replication of stylized facts can onlybe taken as a first step and successful calibra-tion to transaction data must be achieved totruly validate a model.

Despite a number of shortcomings, ABMsstill have many important contributions to offer.In particular, the effect of HF trader activity onthe stability and dynamics of financial marketsis still being debated (Easley et al. 2012) andABMs may hold the answers to many unan-swered questions. Other modeling approaches,such as those employed by Gerig (2012), havebeen used to show the price synchronizing fea-tures of HF trading, as well as features leadingto improvements in market efficiency duringnormal periods and negative consequences dur-ing times of stress. ABMs are well placed tocontribute similarly to this debate.

In future work, it may very well arise thatrules-based, intraday ABMs involving realis-tic matching processes may all demonstratesimilar flaws when subjected to rigorous cali-bration. For this reason, it would be necessaryto mention possible alternatives that may rep-resent a step up from overly-simplified dailyclosed-form models.

One such alternative is to revisit the math-ematics of trader rules and consider the re-placement of the disjointed order price, ordersize and trader activation rules with a sin-gle structure, namely a Hawkes process, togenerate trade events. In recent years, muchprogress has been made in using Hawkes pro-cesses to generate realistic occurrences of orderbook events (Bacry et al. 2015; Martins andHendricks 2016). Hawkes processes may alsoallow the model to function more appropri-ately in an event-based paradigm (Hendrickset al. 2016a), possibly negating a need to down-sample event-based tick data into a chronologi-cal series, which may have resulted in the prob-

20

lems previously observed. An example of theincorporation of Hawkes process into agent-based order book simulation is presented byToke (2011).

Alternatively, it may be advantageous todispense with rules entirely, focusing insteadon large scale simulations involving many pur-poseful, reinforcement-learning agents with noinitially assumed rules (Hendricks and Wilcox2014; Hendricks 2016; Wilcox and Gebbie 2014)and focus on competing, adaptive agents in agame theoretic framework (Galla and Farmer2013; Lachapelle et al. 2015). This may requirea more effective adoption of a large-scale multi-agent system (MAS) approach for the mod-eling of ABM systems for market simulationapplications7, which would require appropri-ately reactive software platforms, such as Akka(Wampler 2015). This in turn would create newchallenges for model validation.

8. Conclusion

We have provided an example of a financialABM that is capable of recovering a num-ber of well-known stylized facts of returntime series, while also demonstrating param-eter degeneracies and calibration difficulties.This casts doubt on the meaningfulness ofthe stylized fact-centric ABM model valida-tion prevalent in much of the contemporaryliterature as it relates to intraday ABMs ap-proximating continuous double auction marketphenomenology and dynamics. Furthermore,the observed parameter degeneracies make itextremely challenging to accept any insightgained through varying parameter values andobserving changes in model behavior.

This work does not serve to undermine theextensive and predominantly successful workthat has been done in the domain of finan-

cial agent-based modeling, particularly withregards to closed-form daily ABMs, but ratherto caution model developers against the blindacceptance of stylized fact-centric validation,which can, as it has here, fail to identify funda-mental model problems.

We suggest that future model validationprocesses should instead focus on demonstrat-ing the ability of an ABM to reproduce thestatistical properties of empirically measuredtransaction data through calibration, while stillincluding stylized-fact verification as a sepa-rate component of model validation.

Accidental agreements between models andstylized facts need to be guarded against.The pragmatic calibration approach of Fabretti(2013) provides an additional probe of modelstructure in the context of continuous double-auction markets, one that may further help bal-ance the tenants of verisimilitude and modelcorroboration with the conceptual benefits ofABMs.

9. Acknowledgements

T.J. Gebbie acknowledges the financial supportof the National Research Foundation (NRF)of South Africa (grant number 87830). D.F.Platt acknowledges the financial support of theUniversity of the Witwatersrand, Johannesburgand NRF of South Africa. The conclusionsherein are due to the authors and the NRFaccepts no liability in this regard.

T.J. Gebbie would like to thank the INET-Oxford Complexity Economics research groupfor hospitality and stimulating interactions onthe application of MAS and ABM models infinance. T.J. Gebbie and D.F. Platt would liketo thank S. Jacob Leal for helpful conversationsrelating to agent-based modeling.

References

Alfarano S, Lux T, Wagner F (2005) Estimation of agent-based models: the case of an asymmetricherding model. Comput Econ 26(1):19-497The authors benefited from conversations with D. Pugh and J.D. Farmer in relation to the large-scale simulation of

purposeful agents in the financial market setting.

21

Alfarano S, Lux T, Wagner F (2006) Estimation of a simple agent-based model of financial markets:an application to Australian stock and foreign exchange data. Physica A 370(1):38-42

Alfarano S, Lux T, Wagner F (2007) Empirical validation of stochastic models of interacting agents.Eur Phys J B 55(2):183-187

Amilon H (2008) Estimation of an adaptive stock market model with heterogeneous agents. JEmpir Financ 15(2):342-362

Bacry E, Mastromatteo I, Muzy JF (2015) Hawkes processes in finance. arXiv:1502.04592

Barde S (2016) Direct calibration and comparison of agent-based herding models of financialmarkets. J Econ Dyn Control 73:329-353

Bollen KA (1989) Structural Equations with Latent Variables. John Wiley & Sons, Hoboken

Bouchaud JP, Mezard M, Potters M (2002) Statistical properties of stock order books: empiricalresults and models. Quant Financ 2(4):251-256

Canova F, Sala L (2009) Back to square one: Identification issues in DSGE models. J MonetaryEcon 56:431-449

Cartea A, Jaimungal S, Penalva J (2015) Algorithmic and High-Frequency Trading. CambridgeUniversity Press, Cambridge

Chakraborti A, Toke IM, Patriarca M, Abergel F (2011) Econophysics review: II. agent-basedmodels. Quant Financ 11(7):1013-1041

Chiarella C, Iori G, Perello J (2009) The impact of heterogeneous trading rules on the limit orderbook and order flows. J Econ Dyn Control 33:525-537

Cont R (2001) Empirical properties of asset returns: stylized facts and statistical issues. QuantFinanc 1:223-236

Cont R, Potters M, Bouchaud JP (1997) Scaling in stock market data: stable laws and beyond.arXiv:9705087

Cooley T (1997) Calibrated models. Oxf Rev Econ Policy 13(3):55-69

Di Matteo T (2007) Multi-scaling in finance. Quant Financ 7:21-36

Easley D, Lopez de Prado M, O’Hara M (2012) The Volume Clock: Insights into the High-FrequencyParadigm. J Portfolio Manage 39:19-29

Fabretti A (2013) On the problem of calibrating an agent-based model for financial markets. J EconInteract Coord 8:277-293

Farmer JD, Joshi S (2002) The price dynamics of common trading strategies. J Econ Behav Organ49:149-171

Galla T, Farmer JD (2013) Complex dynamics in learning complicated games. Proc Natl Acad SciUSA 110:1232-1236

Gao F, Han L (2010) Implementing the Nelder-Mead simplex algorithm with adaptive parameters.Comput Optim Appl 51:259-277

22

Gerig A (2012) High-Frequency Trading Synchronizes Prices in Financial Markets. arXiv:1211.1919

Gilli M, Winker P (2003) A global optimization heuristic for estimating agent based models.Comput Stat Data Anal 42:299-312

Goldberg DE (1989) Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley Longman, Boston

Grazzini J (2012) Analysis of the Emergent Properties: Stationarity and Ergodicity. J Artif Soc SocSimulat 15(2):7

Grazzini J, Richiardi M (2015) Estimation of ergodic agent-based models by simulated minimumdistance. J Econ Dyn Control 51:148-165

Grazzini J, Richiardi M, Tsionas M (2015) Bayesian estimation of agent-based models. NuffieldCollege, Economic Working Papers Series, 2015-W12

Guerini M, Moneta A (2016) A Method for Agent-Based Models Validation. Working Paper Series,Institute for New Economic Thinking, 42

Hamill L, Gilbert N (2016) Agent-Based Modelling in Economics. John Wiley & Sons, Chichester

Harvey M, Hendricks D, Gebbie T, Wilcox D (2017) Deviations in expected price impact for smalltransaction volumes under fee restructuring. Physica A 471:416-426

Hendricks D (2016) Using real-time cluster configurations of streaming asynchronous features asonline state descriptors in financial markets. arXiv:1603.06805

Hendricks D, Gebbie T, Wilcox D (2016a) Detecting intraday financial market states using temporalclustering. Quant Financ 16(11):1657-1678

Hendricks D, Gebbie T, Wilcox D (2016b) High-speed detection of emergent market clustering viaan unsupervised parallel genetic algorithm. S Afr J Sci 122(1/2)

Hendricks D, Wilcox D (2014) A reinforcement learning extension to the Almgren-Chriss modelfor optimal trade execution. Computational Intelligence for Financial Engineering & Economics,2014 IEEE Conference, pp 457-464

Heylighen F (2008) Complexity and Self-organization. In: Bates MJ, Maack MN (eds) Encyclopediaof Library and Information Sciences. CRC Press, Boca Raton

Holland JH (1975) Adaptation in Natural and Artificial Systems. University of Michigan Press,Ann Arbor

Jacob Leal S, Napoletano M, Roventini A, Fagiolo G (2015) Rock Around the Clock: An Agent-Based Model of Low- and High-frequency Trading. J Evol Econ 25:1-25

Kirman A (1991) Epidemics of opinion and speculative bubbles in financial markets. In: Taylor M(ed) Money and Financial Markets. Blackwell, Oxford, pp 354-368

Kukacka J, Barunik J (2016) Estimation of Financial Agent-Based Models with Simulated MaximumLikelihood. SSRN:2783663

Kunsch HR (1989) The jackknife and the bootstrap for general stationary observations. Ann Stat17:1217-1241

23

Lachapelle A, Lasry JM, Lehalle CA, Lions PL (2015) Efficiency of the Price Formation Process inPresence of High Frequency Participants: a Mean Field Game Analysis. arXiv:1305.6323

Lamperti F (2015) An Information Theoretic Criterion for Empirical Validation of Time SeriesModels. LEM Papers Series, Laboratory of Economics and Management, Sant’Anna School ofAdvanced Studies, 2-2015

Lamperti F (2016) Empirical Validation of Simulated Models through the GSL-div: an IllustrativeApplication. LEM Papers Series, Laboratory of Economics and Management, Sant’Anna Schoolof Advanced Studies, 18-2016

LeBaron B (2005) Agent-based Computational Finance. In: Judd KL, Tesfatsion L (eds) TheHandbook of Computational Economics, Vol. 2. Elsevier, Amsterdam, pp 1187-1233

Lehalle CA (2013) Market Microstructure Knowledge Needed for Controlling an Intra-Day TradingProcess. arXiv:1302.4592

Macal CM, North MJ (2010) Tutorial on agent-based modelling and simulation. J Simulat 4:151-162

Mandes A (2015) Impact of inventory-based electronic liquidity providers with a high frequencyevent and agent-based modeling framework. Joint Discussion Paper Series in Economics,Department of Business Administration and Economics, University of Marburg, 15-2015

Martins R, Hendricks D (2016) The statistical significance of multivariate Hawkes processes fittedto limit order book data. arXiv:1604.01824

Nelder JA, Mead R (1965) A simplex method for function minimization. Comput J 7:308-313

Panayi E, Harman M, Wetherilt A (2013) Agent-based modelling of stock markets using existingorder book data. In: Giardini F, Amblard F (eds) Multi-Agent-Based Simulation XIII. Springer-Verlag, Berlin, pp 101-114

Preis T, Golke S, Paul W, Schneider (2006) Multi-agent-based Order Book Model of financialmarkets. Europhys Lett 75(3):510-516

Recchioni MC, Tedeschi G, Gallegati M (2015) A calibration procedure for analyzing stock pricedynamics in an agent-based framework. J Econ Dyn Control 60:1-25

Rigdon EE (1995). A necessary and sufficient identification rule for structural models estimated inpractice. Multivariate Behav Res 30(3):359-383

Thomson Reuters (2016) Thomson Reuters Tick History. https://tickhistory.thomsonreuters.com.Accessed 23 May 2016

Toke IM (2011) Market Making in an Order Book Model and Its Impact on the Spread. In: Abergel F,Chakrabarti BK, Chakraborti A, Mitra M (eds) Econophysics of Order-driven Markets. Springer,Milan, pp 49-64

Tukey JW (1977) Exploratory Data Analysis. Addison-Wesley, Boston, pp 39-49

Wampler D (2015) Fast Data: Big Data Evolved. https://info.lightbend.com/rs/558-NCX-702/images/COLL-white-paper-fast-data-big-data-evolved.pdf. Accessed 4 June 2016

Whitley D (1994) A genetic algorithm tutorial. Stat Comput 4(2):65-85

24

Wilcox D, Gebbie T (2008) Serial correlation, periodicity and scaling of eigenmodes in an emergingmarket. Int J Theoretical Appl Finance 11(7):739-760

Wilcox D, Gebbie T (2014) Hierarchical causality in financial economics. SSRN:2544327

Winker P, Gilli M, Jeleskovic V (2007) An Objective Function for Simulation Based Inference onExchange Rate Data. SSRN:964131

Appendices

A. Nelder-Mead Simplex Algorithm

The Nelder and Mead (1965) simplex algorithmis based upon the fundamental idea that thecurrent solution consists of the points of a sim-plex with n + 1 vertices in the parameter space,Θ, where n is the number of parameters beingcalibrated in our context. During each iteration,we calculate the value of f , the objective func-tion, at the vertices of the simplex and considerthe vertex with the worst objective functionvalue. We then consider the reflection, expan-sion, in-contraction and out-contraction pointsof this vertex. If this leads to a point where thevalue of the objective function is improved, weupdate the vertex with the worst objective func-tion value, but if none of these points leads toan improvement, the simplex shrinks. We con-sider the version of the algorithm presented byGilli and Winker (2003) and perform 5 MonteCarlo replications when applying the method.

Algorithm 1: Nelder-Mead Simplex Algorithm

1: while stopping criteria not met do2: Rename vertices such that f (θ(1)) ≤ ... ≤

f (θ(n+1))3: if f (θR) < f (θ(1)) then4: if f (θE) < f (θR) then5: θ∗ = θE

6: else7: θ∗ = θR

8: end if9: else

10: if f (θR) < f (θ(n)) then11: θ∗ = θR

12: else13: if f (θR) < f (θ(n+1)) then

14: if f (θO) < f (θ(n+1)) then15: θ∗ = θO

16: else17: shrink18: end if19: else20: if f (θ I) < f (θ(n+1)) then21: θ∗ = θ I

22: else23: shrink24: end if25: end if26: end if27: end if28: if not shrink then29: θ(n+1) = θ∗

30: end if31: end while

Table 8: Nelder-Mead Simplex Vertex Transformations

Operation EquationMean θ = 1

n ∑ni=1 θ(i)

Reflection θR = (1 + ρ)θ − ρθ(n+1)

Expansion θE = (1 + ρξ)θ − ρξθ(n+1)

Out-contraction θO = (1 + ψρ)θ − ψρθ(n+1)

In-contraction θ I = (1− ψρ)θ + ψρθ(n+1)

Shrink simplex θ(i) = θ(1) − σ(θ(i) − θ(1)),toward θ(1) i = 2, ..., n + 1

Computation of transformed points as required by theNelder-Mead simplex algorithm. We set ρ = 1, ξ = 2,ψ = 1

2 and σ = 12 , as is the standard for the algorithm

(Gao and Han 2010)

While the Nelder-Mead simplex algorithmis relatively robust in the case of a smooth ob-

25

jective function, it tends to converge to localminima when the objective function being con-sidered is not smooth. Gilli and Winker (2003)highlight this problem in the context of agent-based modeling, as typical objective functionsassociated with agent-based modeling prob-lems are usually not smooth. For this reason,the Nelder-Mead simplex algorithm is com-bined with the threshold accepting heuristic toaid convergence to a global minimum.

In threshold accepting, we randomly per-turb a set of parameters and accept new param-eter sets which are not worse than our currentbest set, as measured by the objective function,by more than a certain threshold, τ. We typi-cally have a set of nR thresholds, τr, r = 1, ..., nR.When combining this heuristic with the Nelder-Mead simplex algorithm, we begin by shiftingthe current simplex in a random direction, withthe magnitude of this shift also being random.We then accept this new simplex if the vertexwith the best objective function value in thenew simplex has an objective function valueless than the best in the previous simplex, afterthe threshold has been added.

We implement the required random shiftas follows:

1. We begin by randomly selecting one of thefree parameters considered in the opti-mization problem.

2. We then multiply the mean value of this pa-rameter across the simplex vertices by arandom number drawn from U (−0.5, 0.5)and add this quantity to the current valueof the selected parameter at each vertex.

We perform nR = 3 rounds in each appli-cation of threshold accepting, each consistingof ns = 5 steps, with τ1 = 1

5 , τ2 = 110 and

τ3 = 0. We perform 3 Monte Carlo replicationsin round 1, 4 in round 2 and 5 in Round 3.

A single step in the combined algorithm(Gilli and Winker 2003), involving both theNelder-Mead simplex algorithm and thresholdaccepting heuristic, is described as follows:

Algorithm 2: Combined Algorithm (NM + TA)

1: Generate u from U (0, 1)2: if u < ξ then3: for r = 1 to nR do4: for s = 1 to ns do5: Generate θnew ∈ Nθold using a ran-

dom shift6: if f (θnew) < f (θold) + τr then7: Accept θnew8: end if9: end for

10: end for11: else12: Generate θnew ∈ Nθold with a simplex

search and check for acceptance13: end if14: if θnew is accepted then15: θold = θnew16: end if

In the above, we set ξ = 0.15.

B. Genetic Algorithm

We refer to the class of algorithms proposedby Holland (1975). In essence, genetic algo-rithms are parameter selection and perturba-tion strategies, inspired by evolutionary pro-cesses in nature, used to solve optimizationproblems. Like organisms in the natural world,superior solutions survive and pass on char-acteristics, whereas inferior solutions tend tobecome extinct.

The basic principles of genetic algorithms,as summarized by Whitley (1994), are given asfollows:

1. We begin with an initial population, usu-ally randomly generated, consisting ofpossible solutions to the optimizationproblem, which are called chromosomesin the context of the literature.

2. Chromosomes that represent better solu-tions to the target problem (as measuredby a fitness function), are given betteropportunities to reproduce (crossover) ascompared to inferior chromosomes.

3. Reproduction (crossover) ultimately re-sults in a new population that hope-

26

fully contains improved chromosomes,in other words, those that result in lower(higher) objective function values in thecontext of a minimization (maximization)problem.

Genetic algorithms have become increasinglyprevalent in contemporary literature, a resultof the fact that they are both intuitive and rel-atively robust. They do not require gradientinformation and are thus well suited to func-tions with many local minima (or maxima)and non-differentiable functions. For this rea-son, genetic algorithms are typically widely ap-plied, unless an applicable gradient or problem-specific method exists (Whitley 1994). Consid-ering that the constructed objective function inour framework does indeed have many localminima and considering the lack of any special-ized algorithms for ABM calibration, a geneticalgorithm seems to be a viable candidate.

We consider the Simple Genetic Algorithm(SGA), described by Goldberg (1989) as fol-lows:

Algorithm 3: Simple Genetic Algorithm

1: Randomly initialize the population 8

2: Determine the fitness values of the currentpopulation

3: while best individual in the populationdoes not meet the required tolerance do

4: Select parents at random to create anintermediate population

5: Perform a crossover operation on theparents to generate offspring

6: Apply the mutation operation to thepopulation of offspring

7: Calculate the fitness values for the newpopulation

8: end while

The genetic operators highlighted in theabove algorithm are discussed in detail as fol-lows:

1. During a selection operation, chromo-

somes are randomly selected with re-placement and probability proportionalto their current fitness values to create anintermediate population.

2. During a crossover operation, chromo-somes are first randomly paired for re-combination. Since the selection processsufficiently shuffles the population, thepairs are simply set to be the first andsecond chromosomes in the intermedi-ate population, the third and fourth chro-mosomes in the intermediate populationand so on. Thereafter, the strings recom-bine, with probability pc, to form newchromosomes that constitute the newpopulation. The specific recombinationprocess is dependent on the implementa-tion.

3. During the mutation operation, a slightperturbation is applied to the population,with a very small probability, pm. Thistypically involves the slight shifting of asingle parameter value associated with asingle chromosome in the population.

It should be noted that fitness and objec-tive functions need not be the same. Fitnessfunctions typically measure the performanceof a chromosome relative to the populationwhereas objective functions tend to measurethe performance of the chromosome in the con-text of the optimization problem itself. In ourcase, however, we set the fitness function to beidentical to the objective function defined forthe calibration framework.

We make use of MATLAB’s ga function, animplementation of the above algorithm, pack-aged as part of the Global Optimization Tool-box. The problem encoding as well as thecrossover and mutation operators in our con-text are defined as per the defaults for this func-tion. Parallel objective function evaluations areperformed using a master-slave paradigm, sim-ilar to that implemented by Hendricks et al.(2016b).

8We set the population size to be 100 and consider 5 Monte Carlo replications per individual.

27

C. Weight Matrix Required by the Method of Simulated Moments

The weight matrix below corresponds to the data set representing transaction data for AngloAmerican PLC over the period beginning at 9:10 on 1 November 2013 and ending at 16:50 on 5November 2013. The diagonal elements correspond to the mean, standard deviation, kurtosis, KStest statistic and Hurst exponent respectively.

W =

5.0346× 104 −1.2885× 104 −736.6343 3.0220× 103 391.2534−1.2885× 104 7.8957× 105 2.5435× 103 −341.4378 −6.1999× 103

−736.6343 2.5435× 103 28.7473 −88.4746 17.26403.0220× 103 −341.4378 −88.4746 723.4611 56.7301

391.2534 −6.1999× 103 17.2640 56.7301 2.3549× 103

It should be noted that the above weight matrix is similar to that obtained by Fabretti (2013).

Our weight matrix is obtained by inverting Cov[me], which had a condition number of 1.2772× 105.While this may seem problematic, it must be noted that Fabretti (2013) inverts Cov[me] with acondition number of 2.6964× 105 to obtain the weight matrix, indicating that our weight matrix isat least as stable.

28