gain a mongodb advantage with the percona memory engine...• data from the mongodb query profiler...
TRANSCRIPT
©2017Percona1
VadimTkachenko
GainaMongoDBAdvantagewiththePerconaMemoryEngineWebinarDecember6th,2017
CTO,Percona
©2017Percona2
Me• CTOatPercona
• LeadingSoftwareEngineeringatPercona
• Technology&(database,hardware,filesystems)PerformanceEnthusiast
• Currentareasofinterest:MySQL,MongoDBandClickHouse
• @VadimTk
©2017Percona3
MemoryEngine• https://www.percona.com/software/mongo-database/percona-
memory-engine-for-mongodb
• StorageEngineinPerconaServerForMongoDB• BasedonwiredTiger• butwithoutwritestothestorage• HighWriteThroughput• HighReadThroughput• Predictable,lowlatencies
©2017Percona4
MemoryEngineusecases
©2017Percona5
ApplicationcacheReplaceservicessuchasmemcached andcustomapplication-leveldatastructureswiththefullpowerofMongoDBfeatures.
©2017Percona6
Typicalarchitecture
Databaselayer(MongoDB,MySQL,etc)
Cachinglayer(memcached or
Redis)
Key-valuestoreLimitedcapabilities
©2017Percona7
Betterway
Databaselayer(MongoDB,MySQL,etc)
CachinglayerPerconaMemory
Engine
• Documentstore• Flexiblequerylanguage• Replication/sharding• FamiliarMongoDBdeployment
©2017Percona8
SessionmanagementDecreaseapplicationresponsetimesbykeepingactiveusersessionsinmemory.
©2017Percona9
TransientruntimestateStoreapplicationstateful runtimedatathatdoesn'trequireon-diskstorage.
©2017Percona10
Multi-tierobjectsharingFacilitatesharingofdatainmulti-tier/multi-languageapplications.
©2017Percona11
ApplicationtestingReduceturnaroundtimeforautomatedapplicationtests.
©2017Percona12
SophisticateddatamanipulationIncreaseperformancefordatamanipulationoperationssuchasaggregationandmapreduction.
©2017Percona13
Real-timeAnalyticsUsesin-memorycomputinginsituationswhereresponsetimeismorecriticalthanpersistence.
©2017Percona14
MemoryEnginedeployments
©2017Percona15
• Fortestinganddevelopment• Usefulforsizingandquickresets• NoRedundancy• NoPersistence• Limitedtoserversize
Standalone
PSMDB Memory
©2017Percona16
./mongod --storageEngine inMemory -–dbpath /data/db -–inMemorySizeGB 150
Standaloneconfig
PSMDB Memory
©2017Percona17
• Forproduction• Failovercapability• Dataredundancy• NoPersistence• Readscaling
ReplicaSet- Memory
PSMDB Memory
primary secondary
secondary
PSMDB Memory
PSMDB Memory
©2017Percona18
ReplicaSet- config
PSMDB Memory
primary secondary
secondary
PSMDB Memory
PSMDB Memory
./mongod --storageEngine inMemory -–dbpath /data/db -–inMemorySizeGB 150 --replSet repSet1
©2017Percona19
• Forproduction• Failovercapability• Dataredundancy• DataPersistence• Readscaling• Configurepermanentmemberasahiddenmember(i.e.hidden:trueandpriority:0)
ReplicaSet– withPersistentOption
PSMDB Memory
primary secondary
secondary
PSMDB Memory
PSMDB wiredTiger or MongoRocks
Permanentstorage
©2017Percona20
Replicatesetwithpermanentoption• Onlythemongod instancesrunningwiththeMemorystorage
enginecanbecometheprimary.• ClientsconnectonlytotheMemorystorageenginemongod
instances.• Evenifbothmongod instancesrunningMemorystorageengine
crashandrestart,theycansyncfromthememberrunningpermanentstorageengine.
• Thehiddenmongod instancerunningwithpermanentstoragepersiststhedatatodisk,includingtheuserdata,indexes,andreplicationconfigurationinformation.
©2017Percona21
Sharding – scalingformorememory
PSMDB Memory PSMDB Memory
mongos
Shard by id
©2017Percona22
Sharding – scalingmongos
Shard by id
PSMDB Memory
primary secondary
secondary PSMDB Memory
PSMDB wiredTiger or MongoRocks
PSMDB Memory
primary secondary
secondary PSMDB Memory
PSMDB wiredTiger or MongoRocks
• Forproduction• Failovercapability• Dataredundancy• DataPersistence• Write scaling
©2017Percona23
PSMDBextras
©2017Percona24
• Dataretrievalbydocumentfields
PSMDB– secondaryindexes{"_id“ : “Vadim”,“visits“ : 45,”created“ : ISODate("2016-01-07T15:46:32.085Z”)
}
> db.coll.ensureIndex({visits:1});> db.coll.find({visits : {$gte :40}});
©2017Percona25
• Toremovedataautomatically
PSMDB– TTLindexes{"_id“ : “Vadim”,“visits“ : 45,”created“ : ISODate("2016-01-07T15:46:32.085Z”)
}
> db.coll.createIndex({"created": 1},{expireAtferSeconds: 7200 });
©2017Percona26
Morefeatures• Authentication• IncludingExternalSASLAuthentication• Role-BasedAccessControl• Auditlogging• GeospatialIndexes• TextSearch
©2017Percona27
PSMDBMemoryEngine- summary
StoreType DocumentStore
Failover/HA ReplicaSet
Scaling Sharding
Persistence ExtraReplicawithwiredTiger orMongoRocks
CRUD Insert,find,update,remove
Expiredata TTL
Concurrency Multi-threading
©2017Percona28
Scalability– MultiThreading• Intel(R)Xeon(R)[email protected]/56
threads• 256GBRAM• Sysbench 1.0witholtp_mongo.lua• Availableathttps://github.com/Percona-Lab/sysbench-mongodb-lua• 100GBdataset• Userthreadsfrom1to128• Operation:OLTPevent
©2017Percona29
©2017Percona30
Stability– MultiThreading• Intel(R)Xeon(R)[email protected]/56
threads• 256GBRAM• Sysbench 1.0witholtp_mongo.lua• Availableatgithub.com/percona-labs• 100GBdataset• Userthreads256• Operation:OLTPevent
©2017Percona31
©2017Percona32
©2017Percona33
Predictableresponsetime130ms(withpracticallynovariation)256userthreads
©2017Percona34
MonitoringwithPerconaMonitoringandManagement(PMM)
©2017Percona35
PMM• NewsoftwarefromPercona• Freeandopen-sourceplatformformanagingandmonitoring
MySQL® andMongoDB® performance.• YoucanrunPMMinyourownenvironment• Providesthoroughtime-basedanalysisforMySQL,MariaDB® and
MongoDBserverstoensurethatyourdataworksasefficientlyaspossible.
©2017Percona36
PMMbenefits• Providesqueryandmetricinformationthatenables
administratorstooptimizedatabaseperformance• Displayscurrentqueriesandhighlightspotentialqueryissuesto
enablefasterissueresolution• Mapsqueriesagainstmetricstohelpmakeinformeddecisions
aboutcrucialdatabaseresources:platformneeds,systemgrowth,teamfocusandthemostimportantdatabaseactivities
©2017Percona37
PMMfeatures• Point-in-timevisibilityandhistoricaltrendingofdatabase
performance• DatafromtheMongoDBqueryprofiler• SpecializedMongoDBdashboardsforgraphingMMAPv1,InMemory,
MongoRocks• Ahistoricalviewofmetricsthatarecriticaltoadatabaseserver• Querymetrics,includingbytessent,locktime,rowssent,andmore• Best-of-breedtools,includingGrafana,Prometheus,andConsul,as
wellasPercona-developedqueryanalytics,administration,API,agentandexportercomponents
• Asingle,easytomanagevirtualappliance
©2017Percona38
©2017Percona39
©2017Percona40
PMM– QueryAnalytics
©2017Percona41
Questions?