Azure + MONGODBMUG Milan June 2015

Ivan Fioravanti

4wardCTO

co-founderAzure & MongoDB Fan

WHO?

http://www.4ward.it

http://www.4ward365.com

MASSIMILIANO SCOTTI

4wardSoftware Developer Machine Learning

Big Data

@ivanfioravanti @maxsv15

AZURE + MONGODBIt’s a kind of magic

2 PARTS

AZURE + MMSAzure CLI

Service ManagerResource Manager

SIOPEOpen Data

Aggregation FrameworkPython

FIRST MAGIC TRICK: REPLICASET AZURE + MONGODB MMS

Photo  by  Nasa  (http://www.nasa.gov/mission_pages/station/multimedia/gallery/iss030e078095.html)

East Coast... USA...

Virginia... Azure “G”

GODZILLA!

REPLICASET?Group of servers that maintain the same data set.

PRIMARY SECONDARY SECONDARY

20 GB 20 GB 20 GB

REPLICASET?

AVAILABILITY

Group of servers that maintain the same data set.

PRIMARY SECONDARY SECONDARYPRIMARY

20 GB 20 GB 20 GB

DEMO REPLICASET ON AZURE

REPLICASET WITH MMS

1 Azure Subscription

Azure Command Line

MongoDB MMS Account

GitHub:

https://github.com/4ward/easy-azure-opensource

SECOND MAGIC TRICK: SHARDINGAzure + MongoDB MMS

SHARDING?

SCALABILITY

Dataset distributed on more servers or replica sets

SHARD A200GB

SHARD B200GB

CONFIG SERVERS

400GB

Router MONGOS

DEMO SHARDING ON AZURE

1 DEV-TESTSERVER

MMS, Azure

Single Server with

1 config server

4 mongod server

1 mongos

COMING SOON: AZURE RESOURCE MANAGERQuick Demo: MongoDB with MMS template

AZURE TYPE “G” + MONGODB = PERFORMANCE!Up to:32 Core448 GB di RAM6.144 GB di SSD

HOW TO TEST IT?...

IDEA! SIOPE DATA IN REALTIME!

Same used in soldipubblici.gov.it

Siope-MONGODB

TRY YOURSELF!

Download zip from Siope.it

Import RDS data from CSV

Convert to Document

Convert to Time Series

github.com/4ward/siope-mongodb

DEMO AZURE “G” + MONGODB + SIOPE

SIOPE: Aggregation Framework in action!

QUERY on PLAIN VS TIMESERIES: 7 VS 2,5 SECS!queries.uscitePerEnteSlow(2014)queries.uscitePerEnte(2014)

REGIONS & PROVINCES?queries.uscitePerRegione(2014)queries.uscitePerProvincie(2014)

EXPENSES BY MACRO ENTITY?queries.uscitePerSottoComparti(2014)

SIOPE: Aggregation Framework in action!

COMUNE DI MILANO DETAIL?queries.uscitePerEnteDettaglio(2014,"COMUNE DI MILANO”)

TOP SPENDERS FOR EDUCATION ?uscitePerRegioniPerCategoriaGestionale(2014,"Servizi scolastici")queries.uscitePerProvinciePerCategoriaGestionale(2014,"Servizi scolastici")queries.uscitePerEntePerCategoriaGestionale(2014, "Serviziscolastici")

ONE MORE THING... MTOOLS!

https://github.com/rueckstiess/mtools

MTOOLS: 3 QUICK SAMPLES

slow queries Dynamic graphmplotqueries mongodb.log

SLOW QUERIES PORTABLE HTML REPORT?mlogvis mongodb.log

LOG DETAILED ANALYSIS FOR SLOW QUERIES?mlogfilter mongodb.log --slow 5000 –word QUERY

Q&A E LINKSEasy-azure-opensourcehttps://github.com/4ward/easy-azure-opensource

Siope – MongoDB translatorhttps://github.com/4ward/siope-mongodb

Azure Quickstart Templateshttps://github.com/Azure/azure-quickstart-templates

MToolshttps://github.com/rueckstiess/mtools

@ivanfioravanti@maxsv15

AZURE LOVES MONGODB