mongodb evenings dc: mongodb - the new default database for giant ideas

Tweet tonight using #MongoDB to be entered to win a t-shirt!

Welcome to MongoDB Evenings DC!Agenda

6:00pm: Pizza, Beer & Soft Drinks

6:30pm: Welcome

Andrew Lockman, Enterprise Account Executive, MongoDB

6:40pm: MongoDB: The New Default Database for Giant Ideas

Gary Taylor, Senior Solutions Architect, MongoDB

7:30pm: Get MEAN and Lean with Docker and Kubernetes

Vadim Polyakov, Director of Enterprise Architecture, Inovalon

8:15pm: Announcements

Q&A

#MongoDB

@1776

MongoDB World

Pre-conference training: June 27th

Conference: June 28th-29th

Hilton Midtown

1335 6th Avenue

New York, NY 10019

Who’s Speaking (partial list)

Who’s Speaking

Code “AndrewLockman” gets 25% off

Early Bird Registration Ends May 13, 2016

June 29 - 29, 2016

New York, NY

www.mongodbworld.com

MongoDB

The New Default Database for GIANT Ideas

Gary W. Taylor

Senior Solutions Architect

[email protected]

Agenda

• The Changing World

• The Relational Approach

• Challenging the Relational Model

• The MongoDB Way

• When to use MongoDB

• Enterprise Advanced Features

• Use Cases Demo (time permitting)

The Changing World

Remember these?

Or these?

“Internet” in 1973

Internet Today

Software Has Changed

•Higher up-front costs

•Higher TCO

• Lower up-front costs

• Lower TCO

Data Has Changed

• 90% of the world’s data was

created in the last two years

• 80% of enterprise data is

unstructured

• Unstructured data growing 2x

faster than structured

What These Changes Mean

YEARS

ITERATIVE

Database Popularity in Today’s World

http://db-engines.com/en/ranking

2015 Gartner Magic Quadrant

for

Operational Database Management Systems

Technical Skills Trend

http://www.indeed.com/jobtrends

The Relational Approach

Modern Relational Underpinnings

Expressive Query Language

& Secondary Indexes

Strong Consistency

Enterprise Management

& Integrations

Relational In Today’s World

Relational Data Model

This is hard…

Long time to develop

Queries are complex

Difficult to change

Car – A Relational View

How we really want to view the car

Challenging the Relational

Model

Database Timeline

Magnetic tape

“flat” (sequential) files

Pre-computer technologies:

Printing pressDewey decimal systemPunched cards

Magnetic Disk

IMS

Relational Model defined

Indexed-Sequential Access Mechanism (ISAM)

Network Model

IDMS

ADABAS

System R

Oracle V2

Ingres

dBase

DB2

Informix

Sybase

SQL Server

Access

Postgres

MySQL

Cassandra

Hadoop

Vertica

Riak

HBase

Dynamo

MongoDB

Redis

VoltDB

Hana

Neo4J

Aerospike

Hierarchical model

1960-701940-50 1950-60 1970-80 1980-90 1990-2000 2000-2010

Dell Software Group

40 Years Ago Today

Relational (SQL)Non-Relational (NoSQL)

NoSQL Challenger’s Underpinnings

Scalability

& Performance

Always On,

Global Deployments

FlexibilityExpressive Query Language

& Secondary Indexes

Strong Consistency


& Integrations

The MongoDB Way

What’s MongoDB? (short version)

• MongoDB (from “humongous”) is a full-featured scalable, high-performance and developer friendly open source document-oriented database.

• Instead of storing your data in tables and rows as you would with a relational database, in MongoDB you store JSON-like documents with dynamic schemas.

• MongoDB bridges the gap between key-value stores (which are fast and scalable) and relational databases (which have rich functionality).

• Using BSON (binary JSON), developers can easily map to modern object-oriented languages without a complicated ORM layer.

• This new data model simplifies coding significantly, and also improves performance by grouping relevant data together internally.

What’s MongoDB for

•We’re for modern, mission-critical and the previously impossible

•We’re for #GIANTideas

MongoDB’s Nexus Architecture

Scalability

& Performance

Always On,

Global Deployments

FlexibilityExpressive Query Language

& Secondary Indexes

Strong Consistency


& Integrations

{

first_name: "Paul",

surname: "Miller",

cell: 447557505611,

city: "London",

{

"type": "Point”, "location”: [-

0.145280,51.510907]

},

Profession: ["banking", "finance",

"trader"],

cars: [

{ model: "Bentley",

year: 1973,

value: 100000, … },

{ model: "Rolls Royce",

year: 1965,

value: 330000, … }

]

}

MongoDB

Document Model

RDBMS

MongoDB vs. SQL terminology

MongoDB SQL

Database Database

Collection Table/View

Document Record/Tuple

PK: _id Field PK: Any Attribute(s)

Dynamic Schema Fixed Schema

Index Index

Embedded Document Joins

Shard Partition

Documents are Rich Data Structures{

first_name: "Paul",

surname: "Miller",

cell: 447557505611,

city: "London",

loc: {

"type": "Point”, ”coordinates”: [-0.145280,51.510907]

},

Profession: ["banking", "finance", "trader"],

cars: [

{ model: "Bentley",

year: 1973,

value: 100000, … },


year: 1965,

value: 330000, … }

]

}

Fields can contain an array of sub-

documents

Fields

Typed field values

Fields can contain

arrays

Document Model Benefits

{

first_name: "Paul",

surname: "Miller",

cell: 447557505611,

city: "London",

loc: {

"type": "Point”, ”coordinates”: [-

0.145280,51.510907]

},

Profession: ["banking", "finance", "trader"],

cars: [

{ model: "Bentley",

year: 1973,

value: 100000, … },


year: 1965,

value: 330000, … }

]

}

Agility and flexibility

Data model supports business change

Rapidly iterate to meet new requirements

Intuitive, natural data representation

Eliminates ORM layer

Developers are more productive

Reduces the need for joins, disk seeks

Programming is simpler

Performance delivered at scale

Dynamic Schema (polymorphic)

{

policyNum: 123,

type: “auto”,

customerId: “abc”,

payment: 1200,

deductible: 500,

make: “Taurus”,

model: “Ford”,

VIN: “123ABC456”,

miles_driven_yr: 15000,

start: “jan, 2016”,

end: “jul, 2016”

}

Insurance Policy Collection

{

policyNum: 456,

type: “life”,

customerId: “efg”,

payment: 240,

policyValue: 125000,

smoker: “no”,

beneficiaries: [

{spouse: “jane”, percent: 80},

{child: “jack”, percent: 20}

],


end: “jul, 2016”

}

{

policyNum: 789,

type: “home”,

customerId: “hij”,

payment: 2300,

deductible: 1000,

square_footage: 3000,

floodCoverage: “no”,

street: “10 Maple Lane”,

city: “Springfield”,

state: “Maryland”,


end: “jul, 2016”

}

Rich

Queries

• Find Paul’s cars

• Find everybody in London with a car built

between 1970 and 1980

db.vehicles.find({

first_name: “Paul”

})

db.vehicles.find({

city: “London”,

“cars.year” : {

$gte : 1970,

$lte : 1980

}

})

{

first_name: "Paul",

surname: "Miller",

cell: 447557505611,

city: "London",

loc: {

"type": "Point”, ”coordinates”: [-

0.145280,51.510907]

},

Profession: ["banking", "finance",

"trader"],

cars: [

{ model: "Bentley",

year: 1973,

value: 100000, … },


year: 1965,

value: 330000, … }

]

}

Fully Featured .

Geospatial• Find all of the car owners within 5 miles of

Trafalgar Square

{

first_name: "Paul",

surname: "Miller",

cell: 447557505611,

city: "London",

loc: {

"type": "Point”, ”coordinates”:

[-0.145280,51.510907]

},

Profession: ["banking",

"finance", "trader"],

cars: [

{ model: "Bentley",

year: 1973,

value: 100000, … },


year: 1965,

value: 330000, … }

]

}

var milesToRadian = function(miles) {

var earthRadiusInMiles = 3963.2;

return miles / earthRadiusInMiles;

};

var landmark = db.landmarks.findOne({name: ”Trafalgar Square"});

var query = {

"loc" : {

$geoWithin : {

$centerSphere : [landmark.loc.coordinates, milesToRadian(5) ]

}

}

};

// Query points.

db.vehicles.find(query).pretty();

Fully Featured ..

Fully Featured ...

Text Search• Find all the cars with ‘leather seats’ in their

description

Aggregation• Calculate average value of every owner’s car

collection & show the top 10 sorted

descending by average value

{

first_name: "Paul",

surname: "Miller",

cell: 447557505611,

city: "London",

loc: {

"type": "Point”, ”coordinates”:

[-0.145280,51.510907]

},

Profession: ["banking",

"finance", "trader"],

cars: [

{ model: "Bentley",

year: 1973,

value: 100000, … },


year: 1965,

value: 330000, … }

]

}

Native Binary

JSON Support• Add other vehicle classes (boats,

motorcycles) to Paul’s vehicle collection.

Left Outer Join

($lookup)• Query for Paul’s cell # and lookup all calls

made in the last 30 days.

Morphia

MEAN Stack

Java Python PerlRuby

Support for the most popular languages and frameworks

Drivers & Ecosystem

When To Use MongoDB

The important aspect of MongoDB

• MongoDB was not designed for niche use cases

• MongoDB strives to have excellent characteristics applicable to a very broad range of use cases

MongoDB: the most balanced general purpose database for Enterprise applications and performance

MongoDB is good for

• Single View

• Internet of Things – sensor data

• Mobile apps – geospatial

• Real-time analytics

• Catalog

• Personalization

• Content management

• Inventory management

• Personalization engines

• Shopping cart

• Dependent datamarts

• Archiving for fast lookup

• Collaboration tools

• Messaging applications

• Log file aggregation

• Caching

• Adserving

• …

Use Cases where MongoDB shines

Mixture of analytics and archiving

Build information from data as it comes in

Extract from DW for analysis

Large volume, targeted queries

Sharing in near real time

Twitter-like apps

e.g., SPLUNK

Enable massive reads on consolidated data

MongoDB Use Cases

Single View Internet of Things Mobile Real-Time Analytics

Catalog Personalization Content Management

Single ViewInternet of Things /

Sensor DataMobile Engagement Real-Time Analytics

Open Data Personalization Content Management

Department of EnergyCustoms and

Immigration Service

Veterans Affairs

Intelligence

Community

Immigration and

Customs Enforcement

Federal

Communications

Commission

Intelligence

Community

US Geological

Survey

Department of

Homeland Security

Defense

Information

Systems Agency

Federal

Aviation

Administration

Intelligence

Community

National Institutes of HealthConsumer Financial

Protection Bureau

Federal

Law Enforcement

Veterans Affairs

US ArmyOffice of the

Secretary of Defense

MongoDB US Federal Use Cases

Best Fit for MongoDB over RDBMSs

Data

Variably or unstructured

Hierarchical objects

Geo-coordinates

Disparate sources

Schema changes often

Querying

Real-time analytics & aggregations

Location-based

Lowest latency

Performance affects user experience

Known relationships between entities

Local reading/writing globally

Other requirements

Agile development

Fastest time-to-market

Cloud infrastructure

Data will grow quickly

Highest throughput

Always on (~99.999%) availability

Lowest TCO

Challenges today with RDBMS

Best Fit for MongoDB over NoSQL

Data

Hierarchical objects

Geo-coordinates

Disparate sources

Schema changes often

Querying

Secondary indexes useful

Strong consistency desired

In-DB analytics & aggregations

Geospatial (location-based)

SQL-based access & BI

Other requirements

Robust management tools

Highest read/write concurrency

Lowest TCO

Full application DB

Largest ecosystem

Future proofing & recruiting

Want to influence roadmap

Commercial license desired

MongoDB does well

• Straightforward replication

• High performance on mixed workloads of reads,

inserts, and updates

• Scaling on demand

• Location based deployment

• Geospatial queries

• High Availability and auto failover

• Flexible schema & secondary indexing

• Agile development in most programming

languages

• Commodity infrastructure

• Real time analytics

• Text indexing

• Data consistency

• Compression

• …

As a database, where does MongoDB shine?

Easy to initiate

All reads, mixed, and mostly writes

No expensive overprovisioning

One cluster can span the globe

Easy to build relevant mobile apps

Low stress operations

No need for complex data modeling

No need to give up your favorite development language

No vendor lock-in through hardware

Get value from data right away !

Basic search feature

Simpler app design

With new version 3.X

Enterprise Advanced

Features

Only With Enterprise Advanced

Storage Engines

Encrypted Storage Engine

In-Memory Storage Engine

Tools

MongoDB Compass

MongoDB Connector for BI

Ops Manager or Cloud Manager

Premium

Integrations

Kerberos & LDAP Authentication

Auditing

SNMP Support

Red Hat Identity Management

Certification

Platform Certification

Windows,

RedHat/CentOS,

Ubuntu,

Amazon Linux

And …

Private, On-Demand Training

Support SLA 1 hour

Support Availability 24 x 365

Emergency Patches

Commercial License, Warranty,

and Indemnification

Encrypted & In-memory Storage

Engines

In-memory storage engine offers the

benefits of in-memory computing without

sacrificing functionality or durability

Encrypted storage engine natively encrypts

database files on disk – for sensitive data,

i.e., in regulated industries

MongoDB Compass

For fast schema discovery and

visual construction of ad-hoc

queries

• Visualize schema

– Frequency of fields

– Frequency of types

– Determine validator rules

• View Documents

• Graphically build queries

• Authenticated access

MongoDB Connector for BI

Visualize and explore multi-

dimensional documents using SQL-

based BI tools. The connector does

the following:

• Provides the BI tool with the schema of the

MongoDB collection to be visualized

• Translates SQL statements issued by the

BI tool into equivalent MongoDB queries

that are sent to MongoDB for processing

• Converts the results into the tabular format

expected by the BI tool, which can then

visualize the data based on user

requirements

Monitoring & Alerting - Charts,

dashboards & alerts on 100+ metrics

Backup - Backup and restore, with point-in-

time recovery

Automation - Single-click provisioning,

scaling out/back & rolling upgrades without

downtime

MongoDB Ops Manager

The Best Way to Manage MongoDB In Your Data Center

Typically around 95% Reduction in Operational Overhead

Integrates with Existing Infrastructure

MongoDB Enterprise Server

MongoDB Enterprise Advanced2

4 x

7 S

up

po

rtC

om

me

rcia

l Lic

en

sePlatform

Certifications

Emergency

Patches

On-Demand

Online Training

Customer

Success

Program

MongoDB CompassMongoDB Ops Manager

Monitoring &

Alerting

Query

Optimization

Backup &

Recovery

Automation &

Configuration

Schema Visualization

Data Exploration

Ad-Hoc Queries

MongoDB

Connector for BI

Visualization

Analysis

Reporting

LDAP & Kerberos Auditing FIPS 140-2Encryption at Rest

REST API

Technical: Why MongoDB

High performance (1000’s – millions queries / sec) - reads & writes

Need flexible schema, rich querying with any number of secondary indexes

Need for replication across multiple data centers, even globally

Need to deploy rapidly and scale on demand (start small and fast, grow easily)

99.999% availability

Real time analysis in the database, under load

Geospatial querying

Processing in real time, not in batch

Need to promote agile coding methodologies

Deploy over commodity computing and storage architectures

Point in Time recovery

Need strong data consistency

Advanced security

Business: Why MongoDB

Management tooling and services

Ease of hiring

Commercial license

Ease of developer adoption

Global Support

Global Professional Services

IT ecosystem integration

Company stability

De facto standard for next generation database

Summary

• Wide range of use cases – and that’s the core point !• Excellent across many possible use cases, not just a few

• Recognized by Gartner and Forrester

• De facto standard for next generation database

• Enterprise maturity and integration

• Built for GIANT Ideas!

Government DBaaSGovernment agency provides a centralized data store to

manage veterans’ electronic records (VLER DAS)

Problem Why MongoDB ResultsProblem Solution Results

Internal and external systems need to

exchange and store data through

trusted connections to provide a full

range of services to the veteran

Clinicians needs accurate information to

ensure quality patient treatment

Benefits users needs accurate

information for benefits adjudication

Leverage flexible data model to save

all types of electronic records via one

centralized data service

Scales easily using sharding to

manage electronic records for the

lifetime (and beyond) of all veterans

Provides expressive query

capabilities to meet the needs of each

line of business

Succeeded in rolling out system in 9

months, meeting Congressionally

mandated deadline

Common access mechanism to

exchange and store veteran electronic

records

One place to store and manage veteran

electronic records for the lifetime of the

agency

Real-Time Geospatial Platform for

InnovationUsing MongoDB to create a smarter and safer city

Problem Why MongoDB ResultsProblem Solution Results

Siloed data across city departments

made it difficult for the City of Chicago

to intelligently analyze situations deliver

services to its citizens

City needed a system that could not

only handle 7 million pieces of data /

day from 30+ departments, but also

run analytics across it to deliver insight

Used MongoDB’s flexible data model to

build the WindyGrid, a unified view of the

city’s operations that brings together

disparate datasets from 30 departments

Leveraged MongoDB’s rich analytics

features (aggregation framework,

geospatial indexes, etc.) to create maps

that deliver real-time insight

Horizonal scalability with automatic

sharding across commodity servers

ensures the city can continue to cost

effectively deliver real-time results

A single view of the city’s operations

on a map of Chicago is now available to

all managers to help them better analyze

and respond to incidents in real-time

New predictive analytics system is

planned that will help prevent crimes

before they happen

450 data sets have been published to the

public, sparking even further innovation,

e.g., an app that alerts citizens when

street sweepers are coming

What is Chicago’s WindyGrid?

• A real-time situational awareness system that brings over a dozen data sources together into a single application.

• Is built using open source software including MongoDB

What is Chicago’s OpenGrid?

• A new project by the City of Chicago

• An open-source, next generation version of WindyGrid

• That will allow the public to navigate Chicago’s open data

• Has a more intuitive interface.

DEMO

http://opengrid.io/

mongodb evenings dc: mongodb - the new default database for giant ideas

Technology