part 2 of the webinar - which freaking database should i use?
TRANSCRIPT
1
Introduc+on to Document Databases
and Couchbase
Dipti Borkar Director, Product Management
2
2.0�
NoSQL Database NoSQL Document Database
Couchbase Server
3
Easy Scalability
Consistent High Performance
Always On 24x365
Grow cluster without applicaCon changes, without downCme with a single click
Consistent sub-‐millisecond read and write response Cmes with consistent high throughput
No downCme for soIware upgrades, hardware maintenance, etc.
Couchbase Server -‐ Core Capabili+es
JSONJSONJSON
JSONJSON
PERFORMANCE
Flexible Data Model
JSON document model with no fixed schema.
4
Rela+onal vs Document data model
RelaConal data model Document data model Collec+on of complex documents with arbitrary, nested data formats and
varying “record” format.
Highly-‐structured table organiza+on with rigidly-‐defined data formats and
record structure.
JSON JSON
JSON
C1 C2 C3 C4
{ }
5
User ID First Last Zip
1 DipC Borkar 94040
2 Joe Smith 94040
3 Ali Dodson 94040
4 Sarah Gorin NW1
5 Bob Young 30303
6 Nancy Baker 10010
7 Ray Jones 31311
8 Lee Chen V5V3M
• • •
50000 Doug Moore 04252
50001 Mary White SW195
50002 Lisa Clark 12425
Country ID
TEL3
001
Country ID
Country name
001 USA
002 UK
003 ArgenCna
004 Australia
005 Aruba
006 Austria
007 Brazil
008 Canada
009 Chile
• • •
130 Portugal
131 Romania
132 Russia
133 Spain
134 Sweden
User ID Photo ID Comment
2 d043 NYC
2 b054 Bday
5 c036 Miami
7 d072 Sunset
5002 e086 Spain
Photo Table
001
007
001
133
133
User ID Status ID Text
1 a42 At conf
4 b26 excited
5 c32 hockey
12 d83 Go A’s
5000 e34 sailing
Status Table
134
007
008
001
005
Country Table
User ID Affl ID Affl Name
2 a42 Cal
4 b96 USC
7 c14 UW
8 e22 Oxford
AffiliaCons Table Country
ID
001
001
001
002
Country ID
Country ID
001
001
002
001
001
001
008
001
002
001
User Table
. . .
Making a Change Using RDBMS
6
Making the Same Change with a Document Database
{ “ID”: 1, “FIRST”: “Dipti”, “LAST”: “Borkar”, “ZIP”: “94040”, “CITY”: “MV”, “STATE”: “CA”, “STATUS”: { “TEXT”: “At Conf”
} }
“GEO_LOC”: “134” }, “COUNTRY”: ”USA”
Just add information to a document
JSON
, }
7
Couchbase Server 2.0 Architecture
Heartbeat
Process m
onito
r
Glob
al singleton supe
rviso
r
Confi
gura+o
n manager
on each node
Rebalance orchestrator
Nod
e he
alth m
onito
r
one per cluster
vBucket state and
replica+
on m
anager
hfp RE
ST m
anagem
ent A
PI/W
eb UI
HTTP 8091
Erlang port mapper 4369
Distributed Erlang 21100 -‐ 21199
Erlang/OTP
storage interface
Couchbase EP Engine
11210 Memcapable 2.0
Moxi
11211 Memcapable 1.0
Memcached
New Persistence Layer
8092 Query API
Que
ry Engine
Data Manager Cluster Manager
8
Couchbase Server 2.0 Architecture
Heartbeat
Process m
onito
r
Glob
al singleton supe
rviso
r
Confi
gura+o
n manager
on each node
Rebalance orchestrator
Nod
e he
alth m
onito
r
one per cluster
vBucket state and
replica+
on m
anager
hfp RE
ST m
anagem
ent A
PI/W
eb UI
HTTP 8091
Erlang port mapper 4369
Distributed Erlang 21100 -‐ 21199
Erlang/OTP
storage interface
Couchbase EP Engine
11210 Memcapable 2.0
Moxi
11211 Memcapable 1.0
Object-‐level Cache
New Persistence Layer
8092 Query API
Que
ry Engine
9
COUCHBASE “THE BASICS”
10
COUCHBASE SERVER CLUSTER
Basic Opera+on
• Docs distributed evenly across servers
• Each server stores both acCve and replica docs – Only one server ac+ve at a +me
• Client library provides app with simple interface to database
• Cluster map provides map to which server doc is on – App never needs to know
• App reads, writes, updates docs
• MulCple app servers can access same document at same Cme
User Configured Replica Count = 1
READ/WRITE/UPDATE
ACTIVE
Doc 5
Doc 2
Doc
Doc
Doc
SERVER 1 ACTIVE
Doc 4
Doc 7
Doc
Doc
Doc
SERVER 2
Doc 8
ACTIVE
Doc 1
Doc 2
Doc
Doc
Doc
REPLICA
Doc 4
Doc 1
Doc 8
Doc
Doc
Doc
REPLICA
Doc 6
Doc 3
Doc 2
Doc
Doc
Doc
REPLICA
Doc 7
Doc 9
Doc 5
Doc
Doc
Doc
SERVER 3
Doc 6
APP SERVER 1
COUCHBASE Client Library CLUSTER MAP
COUCHBASE Client Library CLUSTER MAP
APP SERVER 2
Doc 9
11
Add Nodes to Cluster
• Two servers added with one-‐click operaCon
• Docs automaCcally rebalance across cluster – Even distribu+on of docs – Minimum doc movement
• Cluster map updated
• App database calls now distributed over larger number of servers
REPLICA
ACTIVE
Doc 5
Doc 2
Doc
Doc
Doc 4
Doc 1
Doc
Doc
SERVER 1
REPLICA
ACTIVE
Doc 4
Doc 7
Doc
Doc
Doc 6
Doc 3
Doc
Doc
SERVER 2
REPLICA
ACTIVE
Doc 1
Doc 2
Doc
Doc
Doc 7
Doc 9
Doc
Doc
SERVER 3
SERVER 4
SERVER 5
REPLICA
ACTIVE
REPLICA
ACTIVE
Doc
Doc 8 Doc
Doc 9 Doc
Doc 2 Doc
Doc 8 Doc
Doc 5 Doc
Doc 6
READ/WRITE/UPDATE READ/WRITE/UPDATE
APP SERVER 1
COUCHBASE Client Library CLUSTER MAP
COUCHBASE Client Library CLUSTER MAP
APP SERVER 2
COUCHBASE SERVER CLUSTER
User Configured Replica Count = 1
12
Fail Over Node
REPLICA
ACTIVE
Doc 5
Doc 2
Doc
Doc
Doc 4
Doc 1
Doc
Doc
SERVER 1
REPLICA
ACTIVE
Doc 4
Doc 7
Doc
Doc
Doc 6
Doc 3
Doc
Doc
SERVER 2
REPLICA
ACTIVE
Doc 1
Doc 3
Doc
Doc
Doc 7
Doc 9
Doc
Doc
SERVER 3
SERVER 4
SERVER 5
REPLICA
ACTIVE
REPLICA
ACTIVE
Doc 9
Doc 8
Doc Doc 6 Doc
Doc
Doc 5 Doc
Doc 2
Doc 8 Doc
Doc
• App servers accessing docs
• Requests to Server 3 fail
• Cluster detects server failed – Promotes replicas of docs to
ac+ve – Updates cluster map
• Requests for docs now go to appropriate server
• Typically rebalance would follow
Doc
Doc 1 Doc 3
APP SERVER 1
COUCHBASE Client Library CLUSTER MAP
COUCHBASE Client Library CLUSTER MAP
APP SERVER 2
User Configured Replica Count = 1
COUCHBASE SERVER CLUSTER
13
New in 2.0
JSON support Indexing and Querying
Cross data center replicaCon Incremental Map Reduce
JSONJSONJSON
JSONJSON
14
COUCHBASE SERVER CLUSTER
Cluster wide -‐ Indexing and Querying
User Configured Replica Count = 1
ACTIVE
Doc 5
Doc 2
Doc
Doc
Doc
SERVER 1
REPLICA
Doc 4
Doc 1
Doc 8
Doc
Doc
Doc
APP SERVER 1
COUCHBASE Client Library CLUSTER MAP
COUCHBASE Client Library CLUSTER MAP
APP SERVER 2
Doc 9
• Indexing work is distributed amongst nodes
• Large data set possible
• Parallelize the effort
• Each node has index for data stored on it
• Queries combine the results from required nodes
ACTIVE
Doc 5
Doc 2
Doc
Doc
Doc
SERVER 2
REPLICA
Doc 4
Doc 1
Doc 8
Doc
Doc
Doc
Doc 9
ACTIVE
Doc 5
Doc 2
Doc
Doc
Doc
SERVER 3
REPLICA
Doc 4
Doc 1
Doc 8
Doc
Doc
Doc
Doc 9
Query
15
Cluster wide -‐ XDCR
COUCHBASE SERVER CLUSTER NY DATA CENTER
ACTIVE
Doc
Doc 2
SERVER 1
Doc 9
SERVER 2
SERVER 3
RAM
Doc Doc Doc
ACTIVE
Doc
Doc
Doc RAM
ACTIVE
Doc
Doc
Doc RAM
DISK
Doc Doc Doc
DISK
Doc Doc Doc
DISK
COUCHBASE SERVER CLUSTER SF DATA CENTER
ACTIVE
Doc
Doc 2
SERVER 1
Doc 9
SERVER 2
SERVER 3
RAM
Doc Doc Doc
ACTIVE
Doc
Doc
Doc RAM
ACTIVE
Doc
Doc
Doc RAM
DISK Doc Doc Doc
DISK Doc Doc Doc
DISK
16
Full Text Search Integra+on
• Elas+c Search is good for ad-‐hoc queries and faceted browsing • Couchbase adapter uses XDCR to push muta+ons to ES • Couchbase ES Adapter is cluster-‐aware
ElasCcSearch
Unidirec+onal Cross Data Center Replica+on
17
Couchbase Server Admin Console
18
19
USE CASES
20
Data driven use cases
• Support for unlimited data growth • Data with non-‐homogenous structure • Need to quickly and oaen change data structure • 3rd party or user defined structure • Variable length documents • Sparse data records • Hierarchical data
21
Performance driven use cases
• Low latency macers • High throughput macers • Large number of users • Unknown demand with sudden growth of users/data
• Predominantly direct document access • Workloads with very high muta+on rate per document
22
Use Case Examples
Web app or Use-‐case Couchbase SoluCon Example Customer
Content and Metadata Management System
Couchbase document store + Elas+c Search McGraw-‐Hill…
Social Game or Mobile App
Couchbase stores game and player data
Zynga…
Ad TargeCng Couchbase stores user informa+on for fast access
AOL…
User Profile Store Couchbase Server as a key-‐value store
TuneWiki…
Session Store Couchbase Server as a key-‐value store
Concur….
High Availability Caching Tier
Couchbase Server as a memcached +er replacement
Orbitz…
Chat/Messaging Plaoorm
Couchbase Server DOCOMO…
23
Q & A
24
Andrew Oliver [email protected] www.osintegrators.com
Dipti Borkar [email protected] www.couchbase.com