nosql: an architects perspective
DESCRIPTION
Talk about NoSQL and what it means for software architects from BEDcon 2013TRANSCRIPT
![Page 1: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/1.jpg)
03.04.13
NoSQL: An Architect‘s Perspective Eberhard Wolff Architecture and Technology Manager adesso AG
![Page 2: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/2.jpg)
About me
► Eberhard Wolff
► Architecture & Technology Manager at adesso
► adesso is a leading IT consultancy in Germany
► Speaker
► Author (e.g. first German Spring book)
► Blog: http://ewolff.com
► Twitter: @ewolff
![Page 3: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/3.jpg)
Back in the Days….
03.04.13 NoSQL aus Sicht eines Architekten
![Page 4: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/4.jpg)
NoSQL Is All About the Persistence Question
03.04.13 NoSQL aus Sicht eines Architekten
![Page 5: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/5.jpg)
Key-Value Stores
► Maps keys to values
► Just a large globally available Map
► i.e. not very powerful data model
► No complex queries or indices
► Just access by key
► Redis: Think cache + Persistence
► Riak: Think massive scale
Key Value 42 Some
data
03.04.13 NoSQL aus Sicht eines Architekten
![Page 6: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/6.jpg)
Wide Column
► Add any "column" you like to a row
► Not key-value - "key-(column-value)”
► Column families are like tables
► E.g. in the "Users" column family > "someuser" è ("username"è"someuser"),
("email" è"[email protected]")
► Columns named: indexing possible
► So queries possible
► Apache Cassandra
► Amazon SimpleDB
► Apache HBase
► All tuned for large data sets
XX
XX XX XX XX
XX XX XX
XX XX XX XX
XX XX XX XX
XX XX XX XX
XX XX XX XX
XX XX
XX XX XX
XX XX XX
XX XX XX
XX XX XX XX
XX XX XX XX
XX xX XX XX XX
03.04.13 NoSQL aus Sicht eines Architekten
![Page 7: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/7.jpg)
Document Stores
► Aggregates are typically stored as "documents“ (key-value collection)
► JSON quite common
► No fixed schema
► Indexes possible
► Queries possible > E.g. "find all baskets that contain the product 123"
► Still great horizontal scalability
► Relations might be modeled as links
► MongoDB, CouchDB
03.04.13 NoSQL aus Sicht eines Architekten
![Page 8: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/8.jpg)
Graph
► Nodes with Properties
► Typed relationships with properties
► Ideal e.g. to model relations in a social network
► Easy to find number of followers, degree of relation etc.
► Hard to scale out
► Neo4j
03.04.13 NoSQL aus Sicht eines Architekten
![Page 9: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/9.jpg)
NoSQL Benefits
03.04.13 NoSQL aus Sicht eines Architekten
Costs • Scale out instead of Scale Up • Cheap Hardware • Usually Open Source
Flexibility • Schema in code not in
database • Easier to upgrade schema • Easier to handle
heterogeneous data No Object/relational impedance mismatch • NoSQL database are more OO like
Ops
Dev
![Page 10: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/10.jpg)
Drivers
Exponential Data Growth
More Connected Data
Semi Structured Data
Scale Out
Key Value
Wide Column
Document
Graph
Cost
Flexibility
03.04.13 NoSQL aus Sicht eines Architekten
![Page 11: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/11.jpg)
Document-oriented Databases are the best NoSQL database
03.04.13 NoSQL aus Sicht eines Architekten
For at least one definition of “best”
![Page 12: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/12.jpg)
Document-oriented databases
► Offer scale out > Unless you need huge amounts of data
► Offer a rich and flexible data model > …and queries
► Other databases have other sweet spots
> Huge data sets > Graph structures > Analyzing data
► Niches or mainstream?
04.04.13 NoSQL aus Sicht eines Architekten
Cost
Flexibility
![Page 13: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/13.jpg)
Polyglot Persistence in Ecommerce Application
03.04.13 NoSQL aus Sicht eines Architekten
Financial Data
RDBMS
Product Catalog
Document Store
Shopping Cart
Key / Value
Recommendation
Graph
Needs transactions & reports. Data fit well in tables.
High Performance & Scalability No complex queries
Complex document-like data structures and complex queries
Based on friends, their purchases and reviews
![Page 14: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/14.jpg)
The NoSQL Game
03.04.13 NoSQL aus Sicht eines Architekten
Financial Data
RDBMS
Product Catalog
Document Store
Shopping Cart
Key / Value
Recommendation
Graph
Needs transactions & reports. Data fit well in tables.
High Performance & Scalability No complex queries
Complex document-like data structures and complex queries
Based on friends, their purchases and reviews
0 1000
900 800
2700 High Score!
![Page 15: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/15.jpg)
Just Like the Patterns Game!
04.04.13 NoSQL aus Sicht eines Architekten
Points for each Pattern used Extra points if one class implements multiple Pattern
![Page 16: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/16.jpg)
This is not how Software Architecture works.
03.04.13 NoSQL aus Sicht eines Architekten
![Page 17: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/17.jpg)
Why not?
03.04.13 NoSQL aus Sicht eines Architekten
More is worse!
More hardware More Ops Trouble • Installation • Backup • Disaster Recovery • Monitoring • Optimizations
More Developer Skillz Not necessarily bad
![Page 18: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/18.jpg)
But: Polyglott Persistence Has a Point
03.04.13 NoSQL aus Sicht eines Architekten
Object-oriented Databases did it wrong • Strategy: Replace RDBMS • Enterprises will stick to RDBMS • Pure technology migration basically
never happens • …only vendors think differently
Example: Archive Database • Store current data in RDBMS • Store archive in NoSQL (MongoDB) • Archive contains mainframe data
• Benefit: Use flexibility to allow for many data formats • Benefit: No need to convert mainframe data • Benefit: Store lots of data cheaply
![Page 19: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/19.jpg)
Complex Document Processing System
04.04.13 NoSQL aus Sicht eines Architekten
MongoDB Redis elastic search
Document-oriented
Meta Data for quick access
Search index
Documents
Key/value in memory Search
engine
![Page 20: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/20.jpg)
Alternative: Only elasticsearch
04.04.13 NoSQL aus Sicht eines Architekten
elastic search • Stores original documents as well
• (like a key/value store) • Support for complex queries • Very powerful features also for data
mining / analytics
![Page 21: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/21.jpg)
Alternative: Only MongoDB
04.04.13 NoSQL aus Sicht eines Architekten
MongoDB • Now with (limited beta) fulltext search • Quite fast – memory mapped files • So why Redis? • Map/Reduce support
![Page 22: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/22.jpg)
What about Redis?
04.04.13 NoSQL aus Sicht eines Architekten
Redis • Like a Swiss Knife • Cache • Messaging • Central coordination in a distributed
environment
![Page 23: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/23.jpg)
Your Choice – a trade off!
04.04.13 NoSQL aus Sicht eines Architekten
Typical architecture decision
![Page 24: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/24.jpg)
Who Does What? RDBMS
Developer / Architect
► Schema design (at least partly) ► Access code
DBA
► Performance tuning ► Indices ► Query optimization
► Changes do not influence code
03.04.13 NoSQL aus Sicht eines Architekten
![Page 25: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/25.jpg)
Data Access: RDBMS
03.04.13 NoSQL aus Sicht eines Architekten
RDBMS
• Schema • Stored Procedures
Data Model
Data Access • Queries • Other code
Architect/ Developer
Optimizations
• Indices • Tables
spaces No need to change code • …
DBA
![Page 26: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/26.jpg)
RDBMS separate data from data access
04.04.13 NoSQL aus Sicht eines Architekten
Joins and normalization allow flexible data access patterns
Indices
![Page 27: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/27.jpg)
Data Access MongoDB
04.04.13 NoSQL aus Sicht eines Architekten
MongoDB
• Influences access patterns
Data Model
Data Access • WriteConcerns
how much do love your data?
• Shard key • Consistency
Architect/ Developer
Optimizations
• Only basic indices
Other optimizations must be done in code
DBA
![Page 28: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/28.jpg)
Cluster: RDBMS
► Works somehow
► A special setup of hardware and RDBMS software
04.04.13 NoSQL aus Sicht eines Architekten
DBA
![Page 29: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/29.jpg)
Cluster: MongoDB ► CAP theorem
> Consistency > Availabilty > Partition tolerance > Choose any two
► Deals with replication ► MongoDB has master /
slave replication
04.04.13 NoSQL aus Sicht eines Architekten
Architect/ Developer
MongoDB
► Write Concerns: > Unacknowledge > Acknowledged > Jounrnaled > Some nodes in the
replica set
► Queries might go to master only or also slaves
► Influences consistency
![Page 30: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/30.jpg)
More Power and more Responsibility
03.04.13 NoSQL aus Sicht eines Architekten
DB Admin
Architect
![Page 31: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/31.jpg)
Architects
► Architect has always been a multi-dimensional problem
► Need to choose persistence technology
► Need to think about operations
► Needs to do DBA work 03.04.13 NoSQL aus Sicht eines Architekten
![Page 32: NoSQL: An Architects Perspective](https://reader034.vdocuments.us/reader034/viewer/2022042623/54c68ba04a7959434e8b457d/html5/thumbnails/32.jpg)
NoSQL Is All About the Persistence Question
03.04.13 NoSQL aus Sicht eines Architekten