observability, event sourcing and state machines...observability, event sourcing and state machines...
TRANSCRIPT
![Page 1: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/1.jpg)
Peter Lawrey – Chronicle Software
QCon London - 2017
Observability,EventSourcingandStateMachines
![Page 2: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/2.jpg)
PeterLawrey
JavaDeveloper/Consultantforinvestmentbanksandhedgefunds
for10years.
MostanswersforJavaandJVMonstackoverflow.com
![Page 3: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/3.jpg)
KeypointsInpureJavayoucan• AccessTBsofdatainprocess• DatacanbesharedacrossJVMs• Thiscanspeedupyourapplication
![Page 4: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/4.jpg)
Why?Observability• Reducestimetofix• Reducestimetodeliveraqualitysolution• Improvesperformance
![Page 5: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/5.jpg)
TypicalSolutionsMarketdataprocessinganddistributionOrdergenerationandmanagementPositionnotificationanddistributionRealtimeCompliance
30micro-secondstypical,100micro-seconds,99%ofthetime
![Page 6: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/6.jpg)
128 KB RAM
![Page 7: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/7.jpg)
Howmuchdoesrecordeverythingcost
2TBSSD~£1K
![Page 8: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/8.jpg)
ScaletohighvolumeswithlessmemoryWriting1TBona128GBmachine
![Page 9: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/9.jpg)
ScaletohighvolumeswithlessmemoryWriting1TBona128GBmachine
![Page 10: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/10.jpg)
ScaletohighvolumeswithlessmemoryWriting1TBona128GBmachine
![Page 11: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/11.jpg)
Scaletohighthroughputwithlowlatencies.
![Page 12: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/12.jpg)
HowtoaccessTBsofpersisteddata
MemorymappedfilesDatastructuresonthesefiles
ConcurrentaccessbetweenJVMsUsereplicationinsteadofsync
![Page 13: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/13.jpg)
Eventsourcing
persiststhestateofabusinessentity…asasequenceofstate-changingevents.
Theapplicationreconstructsanentity’scurrentstatebyreplayingtheevents.
http://microservices.io/patterns/data/event-sourcing.html
![Page 14: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/14.jpg)
UsingEventSourcing
Eachoutputistheresultofoneinputmessage.Thisisusefulforgateways,bothinandoutofyour
system.Highlyconcurrent.
![Page 15: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/15.jpg)
Buildinghighlyreproduciblesystems
EachoutputistheresultofALLtheinputs.InsteadofreplyingALLinputmessageeachtime,theFunction
couldsaveanaccumulatedstate.
![Page 16: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/16.jpg)
Yourcriticalpathasaseriesoflowlatency,nonblockingtasks.Thiskeepsyourlatencies
endtoendconsistentlylow.
![Page 17: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/17.jpg)
![Page 18: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/18.jpg)
Recordeverythingmeans
GreaterTransparencyHighReproducibilityFastertimetofix
Fasterdeliveryofaqualitysystem
![Page 19: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/19.jpg)
Togofaster,doless
Perfectionisachieved,notwhenthereisnothingmoretoadd,butwhenthereisnothinglefttotakeaway.
AntoinedeSaint-Exupery
![Page 20: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/20.jpg)
NoFlowControl?
MarketData
Compliance
![Page 21: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/21.jpg)
Reproduceeachcomponentindependently
Whetheryouareenrichingdatafromadatabaseorproductioniscomplex,eachservicecanbetestedin
isolation.
![Page 22: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/22.jpg)
TestingandDebuggingMicroservices
Frameworkscanmaketestinganddebuggingharder.
Youneedtobeabletotestanddebugyourcomponentswithouttheframework,oratransport.
![Page 23: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/23.jpg)
TurningaMonolithintoMicroservices
BusinessComponent+Transport=Service.
![Page 24: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/24.jpg)
Startingwithasimplecontract
Anasynchronousmessagehasatype,apayloadanddoesn’treturnaresult.
public interface SidedMarketDataListener {void onSidedPrice(SidedPrice sidedPrice);
}
public interface MarketDataListener {void onTopOfBookPrice(TopOfBookPrice price);
}
![Page 25: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/25.jpg)
ADataTransferObjectpublic class SidedPrice extends AbstractMarshallable {
String symbol;long timestamp;Side side;double price, quantity;
public SidedPrice(String symbol, long timestamp, Side side, double price, double quantity) {
this.symbol = symbol;this.timestamp = timestamp;this.side = side;this.price = price;this.quantity = quantity;return this;
}}
![Page 26: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/26.jpg)
Deserializable toString()Forittodeserialize thesameobject,noinformationcanbelost,whichusefultocreatingtestobjectsfromproductionlogs.
SidedPrice sp = new SidedPrice("Symbol", 123456789000L, Side.Buy, 1.2345, 1_000_000);
assertEquals("!SidedPrice {\n" +" symbol: Symbol,\n" +" timestamp: 123456789000,\n" +" side: Buy,\n" +" price: 1.2345,\n" +" quantity: 1000000.0\n" +"}\n", sp.toString());
// from stringSidedPrice sp2 = Marshallable.fromString(sp.toString());assertEquals(sp2, sp);assertEquals(sp2.hashCode(), sp.hashCode());
![Page 27: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/27.jpg)
Writingasimplecomponent
Wehaveacomponentwhichimplementsourcontractandinturncallsanotherinterfacewitharesult
public class SidedMarketDataCombinerimplements SidedMarketDataListener {
final MarketDataListener mdListener;
public SidedMarketDataCombiner(MarketDataListener mdListener) {this.mdListener = mdListener;
}
![Page 28: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/28.jpg)
Writingasimplecomponent
Thecomponentcalculatesaresult,usingprivatestate.
final Map<String, TopOfBookPrice> priceMap = new TreeMap<>();
public void onSidedPrice(SidedPrice sidedPrice) {TopOfBookPrice price = priceMap.computeIfAbsent(
sidedPrice.symbol, TopOfBookPrice::new);if (price.combine(sidedPrice))
mdListener.onTopOfBookPrice(price);}
![Page 29: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/29.jpg)
Testingoursimplecomponent
Wecanmocktheoutputlistenerofourcomponent.
MarketDataListener listener = createMock(MarketDataListener.class);listener.onTopOfBookPrice(new TopOfBookPrice("EURUSD", 123456789000L,
1.1167, 1_000_000, Double.NaN, 0));listener.onTopOfBookPrice(new TopOfBookPrice("EURUSD", 123456789100L,
1.1167, 1_000_000, 1.1172, 2_000_000));replay(listener);
SidedMarketDataListener combiner = new SidedMarketDataCombiner(listener);combiner.onSidedPrice(new SidedPrice("EURUSD", 123456789000L,
Side.Buy, 1.1167, 1e6));combiner.onSidedPrice(new SidedPrice("EURUSD", 123456789100L,
Side.Sell, 1.1172, 2e6));
verify(listener);
![Page 30: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/30.jpg)
TestingmultiplecomponentsWecanmocktheoutputlistenerofourcomponent.
// what we expect to happenOrderListener listener = createMock(OrderListener.class);
listener.onOrder(new Order("EURUSD", Side.Buy, 1.1167, 1_000_000));
replay(listener);
// build our scenarioOrderManager orderManager =
new OrderManager(listener);
SidedMarketDataCombiner combiner =new SidedMarketDataCombiner(orderManager);
![Page 31: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/31.jpg)
Testingmultiplecomponents
// events in: not expected to triggerorderManager.onOrderIdea(
new OrderIdea("EURUSD", Side.Buy, 1.1180, 2e6));
combiner.onSidedPrice(new SidedPrice("EURUSD", 123456789000L, Side.Sell, 1.1172, 2e6));
combiner.onSidedPrice(new SidedPrice("EURUSD", 123456789100L, Side.Buy, 1.1160, 2e6));
combiner.onSidedPrice(new SidedPrice("EURUSD", 123456789100L, Side.Buy, 1.1167, 2e6));
// expected to triggerorderManager.onOrderIdea(
new OrderIdea("EURUSD", Side.Buy, 1.1165, 1e6));
verify(listener);
![Page 32: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/32.jpg)
Addingatransport
Anymessagingsystemcanbeusedasatransport.Youcanuse• RESTorHTTP• JMS,Akka,MPI• AeronoraUDPbasedtransport.• RawTCPorUDP.• ChronicleQueue.
![Page 33: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/33.jpg)
Makingmessagestransparent
--- !!data #binaryonOrderIdea: {
symbol: EURUSD,side: Buy,limitPrice: 1.118,quantity: 2000000.0
}
orderManager.onOrderIdea(new OrderIdea("EURUSD", Side.Buy, 1.1180, 2e6));
![Page 34: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/34.jpg)
WhyuseChronicleQueue
ChronicleQueuev4hasanumberofadvantages• Brokerless,onlytheOSneedstobeup.• Lowlatency,lessthan10microseconds99%ofthe
time.• Persisted,givingyourreplayandtransparency.• Canreplaceyourloggingimprovingperformance.• KernelBypass,SharedacrossJVMswithasystemcall
foreachmessage.
![Page 35: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/35.jpg)
--- !!meta-data #binaryheader: !SCQStore { wireType: !WireType BINARY, writePosition: 777, roll: !SCQSRoll { length: 86400000, format: yyyyMMdd, epoch: 0 }, indexing: !SCQSIndexing { indexCount: !int 8192, indexSpacing: 64, index2Index: 0, lastIndex: 0 } }
# position: 227 --- !!data #binaryonOrderIdea: { symbol: EURUSD, side: Buy, limitPrice: 1.118, quantity: 2000000.0 }
# position: 306 --- !!data #binaryonTopOfBookPrice: { symbol: EURUSD, timestamp: 123456789000, buyPrice: NaN, buyQuantity: 0, sellPrice: 1.1172, sellQuantity: 2000000.0 }
# position: 434 --- !!data #binaryonTopOfBookPrice: { symbol: EURUSD, timestamp: 123456789100, buyPrice: 1.116, buyQuantity: 2000000.0, sellPrice: 1.1172, sellQuantity: 2000000.0 }
# position: 566 --- !!data #binaryonTopOfBookPrice: { symbol: EURUSD, timestamp: 123456789100, buyPrice: 1.1167, buyQuantity: 2000000.0, sellPrice: 1.1172, sellQuantity: 2000000.0 }
# position: 698 --- !!data #binaryonOrderIdea: { symbol: EURUSD, side: Buy, limitPrice: 1.1165, quantity: 1000000.0 } ... # 83885299 bytes remaining
![Page 36: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/36.jpg)
Measuringtheperformance?MeasurethewritelatencywithJMH(JavaMicrobenchmarkHarness)
Percentiles, us/op:
p(0.0000) = 2.552 us/op
p(50.0000) = 2.796 us/op
p(90.0000) = 5.600 us/op
p(95.0000) = 5.720 us/op
p(99.0000) = 8.496 us/op
p(99.9000) = 15.232 us/op
p(99.9900) = 19.977 us/op
p(99.9990) = 422.475 us/op
p(99.9999) = 438.784 us/op
p(100.0000) = 438.784 us/op
![Page 37: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/37.jpg)
![Page 38: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/38.jpg)
WherecanItrythisout?
LowLatencyMicroservicesexampleshttps://github.com/Vanilla-Java/Microservices
TheOSSChronicleproductsareavailablehttps://github.com/OpenHFT/
![Page 39: Observability, Event Sourcing and State Machines...Observability, Event Sourcing and State Machines Peter Lawrey Java Developer / Consultant for investment banks and hedge funds for](https://reader030.vdocuments.us/reader030/viewer/2022040307/5ecb5bd4de228e61af6aed6d/html5/thumbnails/39.jpg)
Q&A
Blog:http://vanilla-java.github.io/
http://chronicle.software
@ChronicleUG
https://groups.google.com/forum/#!forum/java-chronicle