introduction to memsql · 2020. 2. 5. · scalable ansi sql petabyte scale real-time and historical...

40
Introduction to MemSQL 1

Upload: others

Post on 25-Aug-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

Introduction to MemSQL

1

Page 2: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

(1) Import image to presentation (2) Go to crop tool, click down arrow (3) Select circle shape (4) Adjust image size using handles and place over this text

Today’s Agenda

● Company Overview

● Common Use Cases

● Product Architecture

● Customer Examples

● Q&A

2

Mike Boyarski

Sr Director, Product Marketing

[email protected] questions in the Question Box

Page 3: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

MemSQL at a Glance

3

Mission Deliver speed and scale for critical data infrastructure

Product Top-ranked database and data warehouse

Customers

Page 4: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

4

Latency Holding Back the Enterprise

Lengthy Queries

Slow query responsesBillion to trillion row tables

No real-time response

Limited User Access

Single-threaded operationsData silos for mixed workloads

Single-node performance

Slow Data Loading

Batch processingHours to load

Sampled data views

Page 5: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

5

The Enterprise Requires Performance

Fast Queries

Scalable ANSI SQLPetabyte scaleReal-time and historical insights

Scalable User Access

Scale-out for performanceConverged transactions and analyticsPowerful workload management

Live Loading

Stream and bulk dataOn-the-fly transformations

Multiple sources

Page 6: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

6

What is MemSQL

Page 7: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

7

MemSQL: The No-Limits Database

● Scale-out relational database

● Extreme performance at scale

● Lock-free ingest

● High concurrency

● Deploy anywhere

TM

10x faster and 3x more cost effective than legacy database providers

Page 8: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

8

Common Use Cases

Modernize Legacy Workloads

● Enable real-time analytics for operational systems

● Accelerate performance of single- node RDBMS like Oracle RAC/Exadata, SQL Server, MySQL, Greenplum, Netezza, Vertica

Page 9: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

9

Common Use Cases

Modernize Legacy Workloads Enable New Workloads

● Enable real-time analytics for operational systems

● Accelerate performance of single-node RDBMS like Oracle RAC/Exadata, SQL Server, MySQL, Greenplum, Netezza, Vertica

● Streaming and IoT analytics● Augment and accelerate Hadoop● Real-time machine learning● Replace and consolidate single-use

databases like MongoDB, Redis, and Elastic

Page 10: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

10

High-Speed IngestFast bulk load or stream data

with built-in pipelines

Memory-Optimized TablesUltra-low latency for

transactions and analytics

Disk-Optimized TablesPetabyte scale analytics with

compression and performance

Unified Platform

Page 11: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

Relational Data

11

Ecosystem Overview

Streaming Ingest

MemoryOptimized

Tables

Disk Optimized

Tables

Real-Time Data

Transforms

Data Lakes

ETL/ELT

Spark

DBMS

Hadoop Amazon S3

Bare Metal, Virtual Machines, Containers On-Premises, Cloud, As a Service

Modern Applications

TableauLooker Microstrategy

Kafka

CDC

Informatica, Talend, Alooma

Attunity

Data WarehouseBI Dashboards

Page 12: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

12

MemSQL Architecture

Page 13: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

13

MemSQL is a Distributed Relational Database

Page 14: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

14

MemSQL uses Aggregator Nodes and Leaf Nodes

LeafLeafLeafLeaf

AggregatorAggregator

Page 15: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

15

Leaf Nodes Store Data in Partitions

LeafLeafLeafLeaf

AggAggregator

MasterAggregator

LeafLeafLeafLeafPARTITIONS PARTITIONS PARTITIONS PARTITIONS

AggregatorAggregator

Page 16: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

16

LeafLeafLeafLeaf

AggAggregator

MasterAggregator

LeafLeafLeafLeafPARTITIONS PARTITIONS PARTITIONS PARTITIONS

AggregatorMaster Agg

Every Cluster has at Least One Master Aggregator and One Leaf Node

Page 17: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

17

LeafLeafLeafLeaf

AggAggregator

MasterAggregator

LeafLeafLeafLeafPARTITIONS PARTITIONS PARTITIONS PARTITIONS

AggregatorMaster Agg

Applications Connect to an Aggregator

Application / Database Client

Page 18: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

18

Pipelines Ingest Across all Leaf Nodes

LeafLeafLeafLeaf

AggAggregator

MasterAggregator

LeafLeafLeafLeafPARTITIONS PARTITIONS PARTITIONS PARTITIONS

AggregatorMaster Agg

Page 19: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

19

MemSQL Capabilities

Page 20: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

High-Speed Ingestion

● Stream ingestion

● Fast parallel bulk loading

● Built-in Create Pipeline

● Transactional ACID Consistency

● Exactly-Once Semantics

● Native integrations with Kafka, AWS S3, Azure Blob, HDFS

20

Page 21: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

Durable Distributed Storage

21

Highly Available

Online replication ensures data consistency and protects

against outages

Big Data Capacity

Petabyte scale with up to 10x compression and instant

query retrieval

Distributed and Durable

Store and process on clusters of machines for performance

and persistence

Page 22: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

Ultra-Fast Insights

● Scalable ANSI SQL

● Support for JSON, Geospatial, and Full-Text Search

● Fast Query Vectorization and Compilation

● Extensibility with Stored Procedures, UDFs, UDAs

22

Page 23: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

Built for Enterprise Demands

● Interoperable with existing tools & skills

● Deploy on-premises or cloud

● Run on bare metal, VMs, or containers

● Simplified management

● Comprehensive security

23

Page 24: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

Enterprise-Grade Security

● Role-Based Access Control (RBAC)

● Encryption

● Authentication

● Audit logging

● Strict mode

24

Page 25: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

Easy to Manage and Tune

● Monitor performance and capacity

● API-based deployment and config

● Full administration and scaling

● Query profiler

● Visual query plans

25

Page 26: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

Customer Examples

26

Page 27: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

27

How Fanatics Powered Their Wayto a Better Simplified Future

Page 28: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

28

Company Profile

• The global leader in licensed sports merchandise

• $2 billion revenue

• Run online stores of all major North American sports leagues, more than 200 professional and collegiate teams

Page 29: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

Place Image Here

29

Event-Driven ArchitectureFor responsiveness, reliability, and scalability

Page 30: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

Place Image Here

30

Before MemSQL: Event-Driven Analytics Architecture

Page 31: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

Place Image Here

31

With MemSQL: Consolidation, Simplification, and Performance

Page 32: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

Place Image Here

With MemSQL: Consolidation, Simplification, and Performance

Page 33: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

● Migrated from to

● Worked great in the beginning

● Unfortunately, ran into:

○ Scalability issues as the workload increased

○ Significant increase in overall cost

○ Support challenges

○ Outages on Tier 1 workloads impacted visibility of network health

33

+ Leading Wireless Provider in the US

Page 34: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

34

Business Benefits:▪ Collect and process KPIs about wireless

network ▪ Reports on health of the wireless network▪ Significant cost savings over Oracle - 2.3x

times less

▪ No change to reporting front-end

Technical Benefits:▪ Consolidate ~100 TB, ~50K tables, ~55K reports,

~1K concurrent users from 4 Exadatas into 2 MemSQL Clusters

▪ Simpler and easier to manage compared to Oracle

▪ 80-100x faster than Exadata

▪ Run Oracle queries as-is

High Speed Operational Analytics - Call Quality Analysis

+ Leading Wireless Provider in the US

Page 35: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

A top five financial institution uses MemSQL to power its real-time customer dashboard to show the most up-to-date (millisecond) portfolio summaries

3535

Financial Services

Real-Time Decision MakingRisk Management, Algorithmic Trading, and Fraud Detection

Page 36: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

Using MemSQL, a top energy provider ingestsdrill bit data and reports back key indicators within minutes, allowing operators to respondto live conditions and preventing costly drill failures and outages

3636

Energy & Utilities

IOT AnalyticsPredictive Maintenance, Supply Chain Analytics, and Utility Grid Management

Page 37: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

At Uber, MemSQL provides a live operational view into activity for millions of drivers and customers every day, delivering geospatial views for optimizing service delivery and managing day-to-day operations

3737

High Technology

Operational AnalyticsReal-time dashboards, geospatial analytics, and customer 360

Page 38: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

Media &

38

Ad Analytics, Personalization, and Streaming Media Analytics

Comcast uses MemSQL to deliver analytics on data from millions of DVR boxes in real time,helping Comcast to provide superior customer experience with proactive service alerts and user insights for optimizing advertising programs

38

Communication

Page 39: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

Place Image Here

MemSQLRecap

39

• Distributed, ANSI SQL, database

• Full ACID transactions

• Lock free, shared nothing

• Compiled queries

• Massively parallel

• Geospatial and JSON

• In-memory and on-disk

• MySQL wire protocol

• Comprehensive security

• Rowstore & columnstore tables

Page 40: Introduction to MemSQL · 2020. 2. 5. · Scalable ANSI SQL Petabyte scale Real-time and historical insights ... Redis, and Elastic. 10 High-Speed Ingest Fast bulk load or stream

40

Thank You! Questions?

Learn more at memsql.com/product

Questions? Email us at [email protected]

Try at memsql.com/download● No time limit● Deploy to production● Full featured● Up to 128GB of RAM, unlimited disk● Get support at forum.memsql.com