antonios giannopoulos percona 2016 wiredtiger configuration variables
Post on 22-Jan-2018
60 Views
Preview:
TRANSCRIPT
WiredTigerconfigura/onvariables-Lookingunderthehood
AntoniosGiannopoulosDatabaseAdministrator–Rackspace
PerconaLiveEurope2016
Agenda
Wearegoingtodiscuss:MongoDBWiredTigerConfigura/onVariables
- Whattheymean?- Howtoconfigure/change?- Benchmarks
BenchmarkspecsRackspaceOnMetalCloudServerv212cores32GBRAM2x800GBSSDs(RAID1)MongoDB3.2.9SysbenchforMongoDBbyTimCallaghanusingdefaultse[ngs
BenchmarkspecsSysbench:1) Creates16collec/ons10milliondocumentseach2) Executesamixedworkloadfor10minutesLoad:Comple/on/mefor(1)TPSCUM:AveragenumberofSysbenchtransac/onsTPSINT:NumberofSysbenchtransac/onsSamplesarecollectedevery10seconds
WTconfigura/on/FilesThreewaystochangeWTvariables:mongod.conf:engineConfig.configStringrun/me:db.adminCommand({setParameter:…})configura/onfile(s):(WiredTiger.basecfg,WiredTiger.config)
Theorderofconfigura/onis:WiredTiger.basecfgfilewired/ger_openconfigura/onstringargument,WiredTiger.configfile,WIREDTIGER_CONFIGenvironmentvariable
WTconfigura/on/mongod.confengineConfig: cacheSizeGB:<number> journalCompressor:<string> directoryForIndexes:<boolean>
configString:<string>,<string>…collec/onConfig: blockCompressor:<string>indexConfig: prefixCompression:<boolean>
setParameter: wiredTigerConcurrentReadTransac/ons:<number> wiredTigerConcurrentWriteTransac/ons:<number>
WTconfigura/on/mongod.confengineConfig: cacheSizeGB:<number> journalCompressor:<string> directoryForIndexes:<boolean>
configString:<string>,<string>…collec/onConfig: blockCompressor:<string>indexConfig: prefixCompression:<boolean>
setParameter: wiredTigerConcurrentReadTransac/ons:<number> wiredTigerConcurrentWriteTransac/ons:<number>
JournalBenchmarkDefault:SnappyLoad:10minutesand58secondsWithoutcompression:Load:10minutesand21secondsWithoutjournal:Load:8minutesand35seconds
Journal(cumtps)
0
200
400
600
800
1000
1200
0 100 200 300 400 500 600 700
default
withoutcompression
withoutjournal
Journal(inttps)
0
200
400
600
800
1000
1200
1400
0 100 200 300 400 500 600 700
default
withoutcompression
withoutjournal
WTconfigura/on/mongod.confengineConfig: cacheSizeGB:<number> journalCompressor:<string> directoryForIndexes:<boolean>
configString:<string>,<string>…collec/onConfig: blockCompressor:<string>indexConfig: prefixCompression:<boolean>
setParameter: wiredTigerConcurrentReadTransac/ons:<number> wiredTigerConcurrentWriteTransac/ons:<number>
CompressionAlgorithmssnappy:itgathersdatauptoamaximumof32KB,compressesit,andifcompressionissuccessful,writestheblockroundeduptothenearest4KB.zlib:itgathersmoredataandcompressenoughtofilla32KBblockondisk.Configurablepercollec/on:db.createCollec/on("foo",{storageEngine:{wiredTiger:{configString:"block_compressor=zlib"}}})
blockCompressorBenchmarkSnappy:Load:10minutesand58secondsZlib:Load:18minutesand21secondsWithoutcompression:Load:10minutesand45seconds
blockCompressor(cumtps)
0
200
400
600
800
1000
1200
0 100 200 300 400 500 600 700
snappy
zlib
nocompression
blockCompressor(inttps)
0
200
400
600
800
1000
1200
1400
1600
0 100 200 300 400 500 600 700
snappy
zlib
none
WTconfigura/on/mongod.confengineConfig: cacheSizeGB:<number> journalCompressor:<string> directoryForIndexes:<boolean>
configString:<string>,<string>…collec/onConfig: blockCompressor:<string>indexConfig: prefixCompression:<boolean>
setParameter: wiredTigerConcurrentReadTransac/ons:<number> wiredTigerConcurrentWriteTransac/ons:<number>
Prefixcompressionuseusedusefulusefullyusefulnessuselessuselesslyuselessness
Prefixcompressionuseusedusefulusefullyusefulnessuselessuselesslyuselessness
0:use1:0d2:0ful3:0fully4:0less5:0lessly6:0lessness
Prefixcompressionuseusedusefulusefullyusefulnessuselessuselesslyuselessness
0:use1:0d2:0ful3:0fully4:0less5:0lessly6:0lessness
0:use1:0d2:0ful3:2ly4:0less5:4ly6:4ness
Prefixcompression(load/mes)
Default:Snappycompression:Load:10minutesand58secondsNone:Withoutcompression:Load:09minutesand51seconds
Prefixcompression(cumtps)
0
200
400
600
800
1000
1200
0 100 200 300 400 500 600 700
default
none
Prefixcompression(inttps)
0
200
400
600
800
1000
1200
1400
0 100 200 300 400 500 600 700
default
none
WTconfigura/on/mongod.confengineConfig: cacheSizeGB:<number> journalCompressor:<string> directoryForIndexes:<boolean>
configString:<string>,<string>…collec/onConfig: blockCompressor:<string>indexConfig: prefixCompression:<boolean>
setParameter: wiredTigerConcurrentReadTransacKons:<number> wiredTigerConcurrentWriteTransacKons:<number>
ConcurrentTransac/onsSpecifythemaximumnumberofconcurrentread/writetransac/ons.Defaultis128mongod.confsetParameter: wiredTigerConcurrentReadTransac/ons:<number> RunKme:db.adminCommand({setParameter:1,wiredTigerConcurrentWriteTransac/ons:<num>})
ConcurrentTransac/onsBenchmark64Trx:64read/writetranscac/onsLoad:11minutesand06seconds128Trx:Default-128read/writetranscac/onsLoad:10minutesand58seconds256Trx:256read/writetranscac/onsLoad:10minutesand46seconds
ConcurrentTransac/ons(cumtps)
0
200
400
600
800
1000
1200
0 100 200 300 400 500 600 700
trx-128
trx-64
trx-256
ConcurrentTransac/ons(inttps)
0
200
400
600
800
1000
1200
1400
0 100 200 300 400 500 600 700
trx-128
trx-64
trx-256
WTconfigura/on/mongod.confengineConfig: cacheSizeGB:<number> journalCompressor:<string> directoryForIndexes:<boolean>
configString:<string>,<string>…collec/onConfig: blockCompressor:<string>indexConfig: prefixCompression:<boolean>
setParameter: wiredTigerConcurrentReadTransac/ons:<number> wiredTigerConcurrentWriteTransac/ons:<number>
WTconfigura/onEveryWTvariablecanbeassignedtoconfigStringWearegoingtoexamineasubset:- evic/on_target- evic/on_trigger- evic/on_dirty_target- evic/on_dirty_trigger- evic/on.threads_min- evic/on.threads_max- direct_IO
Evic/on
ResidentObjects Sharedcache
Evic/on
Cachesavesobjects(workingset)
Evic/on
Evic/onTriggers
Evic/on
Evic/onCandidates
Evic/on
Evic/onTarget
evic/on_triggerevicKon_triggeristheoccupiedpercentageofthetotalcachesizethatcausesevic/ontostart.Defaultvalue:95%Acceptablevalues:Between10and99RunKme:db.adminCommand({setParameter:1,wiredTigerEngineRun/meConfig:"evic/on_trigger=71”})mongod.conf:evic/on_trigger=71WiredTiger.config:evic/on_trigger=71
evic/on_targetevicKon_targetistheoveralltargetforevic/on,expressedasapercentageoftotalcachesize.Defaultvalue:80%Acceptablerange:Between10and99(lowerthantrigger)RunKme:db.adminCommand({setParameter:1,wiredTigerEngineRun/meConfig:"evic/on_target=70"})mongod.conf:evic/on_target=70WiredTiger.config:evic/on_target=70
Evic/on_trigger/targetBenchmarkDefault:trigger/target:95/80Load:11minutesand06seconds95/90:trigger/target:95/90Load:11minutesand41seconds90/89:trigger/target:90/89Load:10minutesand16seconds95/94:trigger/target:95/94Load:12minutesand26seconds
evic/on_trigger/target(cumtps)
0
200
400
600
800
1000
1200
0 100 200 300 400 500 600 700
default
95and90
90and89
95and94
Evic/on_trigger/target(inttps)
0
200
400
600
800
1000
1200
1400
1600
0 100 200 300 400 500 600 700
default
95and90
90and89
95and94
evic/on_dirty_triggerevicKon_dirty_trigger:triggerevic/onwhenthecacheisusingthismuchmemoryfordirtycontent,asapercentageofthetotalcachesize.Defaultvalue:95%Acceptablerange:Between5and99RunKme:db.adminCommand({setParameter:1,wiredTigerEngineRun/meConfig:"evic/on_dirty_trigger=71"})mongod.conf:evic/on_dirty_trigger=71WiredTiger.config:evic/on_dirty_trigger=71
evic/on_dirty_targetevicKon_dirty_target:con/nueevic/ngun/lthecachehaslessdirtypagesthanthis(asapercentage).Default:80%Acceptablerange:Between5and99(lowerthantrigger)RunKme:db.adminCommand({setParameter:1,wiredTigerEngineRun/meConfig:"evic/on_dirty_target=70"})mongod.conf:evic/on_dirty_target=70WiredTiger.config:evic/on_dirty_target=70
dirty_trigger/targetBenchmarkDefault:trigger/target95/80Load:10minutesand58seconds20/5:trigger/target20/5Load:31minutesand20seconds80/70:trigger/target80/70Load:10minutesand06seconds
dirty_trigger/target(cumtps)
0
200
400
600
800
1000
1200
0 100 200 300 400 500 600 700
default
20and5
80and70
dirty_trigger/target(inttps)
0
200
400
600
800
1000
1200
1400
1600
0 100 200 300 400 500 600 700
default
20and5
80and70
evic/onthreadsevicKon.threads_min&evicKon.threads_max:theminimumandmaximumnumberofaddi/onalevic/onthreadsDefault:1and4Acceptablerange:between1and20RunKme:db.adminCommand({setParameter:1,wiredTigerEngineRun/meConfig:"evic/on=(threads_min=2)"})mongod.conf:evic/on=(threads_max=2)WiredTiger.config:evic/on=(threads_max=2)
evict_threads_min/maxBenchmarkDefault:min/max1/4Load:10minutesand58seconds4and6:min/max4/6Load:10minutesand31seconds2and8:min/max2/8Load:10minutesand11seconds4and4:min/max4/4Load:10minutesand21seconds
evict_threads_min/max(cumtps)
0
200
400
600
800
1000
1200
0 100 200 300 400 500 600 700
default
4and6
2and8
4and4
evict_threads_min/max(inttps)
0
200
400
600
800
1000
1200
1400
1600
0 100 200 300 400 500 600 700
default
4and6
2and8
4and4
direct_io
STORAGE STORAGE
FScacheWTcache WTcache FScache
RAM RAM
direct_io• minimizetheopera/ngsystemcacheeffectsofI/Otoand
fromWiredTiger'sbuffercache• avoiddouble-bufferingofblocksinWiredTiger'scacheand
theopera/ngsystembuffercache• avoidstallingunderlyingsolid-statedrivesbywri/ngalarge
numberofdirtyblocks.WiredTiger.config:direct_io=(data)mongod.conf:storage.wiredTiger.engineConfig.configString:direct_io=(data)
DirectIOBenchmark
Default:WithoutDirectIOLoad:10minutesand58secondsDirectIO:WithDirectIOLoad:11minutesand25seconds
DirectIO(cumtps)
0
200
400
600
800
1000
1200
0 100 200 300 400 500 600 700
default
directIO
DirectIO(inttps)
0
200
400
600
800
1000
1200
1400
1600
0 100 200 300 400 500 600 700
default
directIO
Ques/ons?
Thankyou!!!
antonios.giannopoulos@rackspace.co.uk#iamantonios
top related