from sql to nosql: the fourth time’s the charm – couchbase connect 2016

©2016 Couchbase Inc. 1

The Couchbase Connect16 mobile appTake our in-app survey!


Beyond Rows and Columns:Is the Fourth Time the

Charm?Michael J. Carey

University of California, Irvine and Couchbase, Inc.


Michael J. CareyBren Professor of I&CS, UC IrvineConsulting Architect, CouchbaseDatabase researcher and architect. Background includes BEA, IBM, Propel Software, University of Wisconsin-Madison, and UC Irvine.ACM Fellow, member of the National Academy of Engineering, and recipient of the ACM SIGMOD E.F. Codd Innovations Award. Current interests all center around data-intensive computing and scalable data management (a.k.a. Big Data).Personal goal: Retire early enough to go to music school and have a second (unsuccessful but fun ) career playing music.

©2016 Couchbase Inc. 4©2016 Couchbase Inc.

Databases and Data Models from Pre-SQL to “NoSQL”

• The pre-relational era• The relational database era• Beyond rows and columns?

1. The object-oriented database era2. The object-relational database era3. The XML database era4. The NoSQL database era

• Where does Couchbase technology fit in?• Reflections, challenges, and Q&A


The Birth of Today’s DBMS Field

• In the beginning was the Word, and the Word was with Codd, and the Word was Codd...• 1970 CACM paper: “A relational model of data for large shared data

banks”

• Some view this as the first generation of (real) DB management systems...


The First Decade B.C.

• The need for a data management library, or a database management system, had actually been well recognized before Codd• Hierarchical database systems (e.g., IMS from IBM)• Network database systems (most notably the CODASYL

standard)

• These systems provided navigational APIs• Systems provided files, records, pointers, and indexes• Programmers had to (carefully!) scan or search for records,

follow parent/child structures or pointers, and maintain their code for correctness and performance when anything physical changed


The First Decade B.C. (cont.)

Order (id, custName, custCity, total)

Item (ino, qty, price)

Product (sku, name, listPrice, size, power)

Item-ProductItem-Order

Item-Order

123 Fred LA 25.97401 Garfield T-Shirt 9.99 XL -

544 USB Charger 5.99 - 115V1 2 9.99 2 1 3.99

Order

Item Item

Product

Product

Item-Product

Item-Product

Set Member

Set OwnerSet


Enter the Relational DB Era

• Be sure to notice that• Everything’s now (logical) rows and columns• The world is flat; columns are atomic (1NF)• Data is now connected via values (foreign/primary keys)

Order (id, custName, custCity, total)

Item (order-id, ino, product-sku, qty, price)

Product (sku, name, listPrice, size, power)123 Fred LA 25.97

401 Garfield T-Shirt 9.99 XL null

544 USB Charger 5.99 null 115V

123 1 401 2 9.99

123 2 544 1 3.99

!!!


As the Relational Era Unfolded

• The Spartan simplicity of the relational data model made it possible to start tackling the many opportunities (and challenges) of a logical data model• Declarative queries (Relational algebra, Relational calculus, Quel, QBE,

SQL, ...)• Transparent indexing (physical data independence)• Query optimization and (efficient) execution• Views, constraints, referential integrity, security, ...• Scalable (shared-nothing) parallel processing

• Today’s multi-$B database industry was slowly but surely born• Serious commercial adoption took ~10-15 years• Commercial parallel DB systems took ~5 more years


Enter the Object-Oriented DB Era

• Notice that:• Data model contains objects and pointers (OIDs)• The world is no longer flat – the Order and Item schemas now “contain”

set(Item) and Product , respectively• The world is no longer straight-jacketed – Product details can vary

(subject to OO model)

123 Fred LA 25.97 { }401 Garfield T-Shirt 9.99 XL

544 USB Charger 5.99 115V1 2 9.99 2 1 3.99

Order

ItemItem

ClothingProduct

ElectricProduct


What OODBs Sought to Offer

• Motivated largely by late 1980’s CAx applications (e.g., mechanical CAD, VLSI CAD, software CAD, ...)• Rich schemas with inheritance, complex objects, object identity,

references, ...• Methods (“behavior”) as well as data in the DBMS• Tight bindings with (OO) programming languages• Fast navigation, some declarative querying

• Ex: Gemstone, Ontos, Objectivity, Versant, Object Design, O2, ...


Why OODBs “Fell Flat”

• Too soon for another (radical) DB technology• Also technically immature relative to RDBMSs

• Tight PL bindings were a double-edged sword• Data most often shared, and outlives programming languages• Bindings led to significant system heterogeneity• Also made schema evolution a major challenge

• Systems “overfitted” in some dimensions• Inheritance, version management, ...• Focused on thick clients (e.g., CAD workstations)


Enter the Object-Relational DB Era

• Be sure to notice:• “One size fits all!” ()• UDTs/UDFs, table hierarchies, references, ...• But the world got flatter again... (Timing lagged OODBs by just a few years)

Product (sku, name, listPrice) ClothingProduct (size) under Product ElectricProduct (power) under Product

Order (id, customer, total)

Item (order-id, ino, product-sku, qty, price) 401 Garfield T-Shirt 9.99 XL

544 USB Charger 5.99 115V(123) 1 (401) 2 9.99

(123) 2 (544) 1 3.99

123 Fred LA 25.97


What O-R DBs Sought to Offer

• Motivated in large part by newly emerging application opportunities (multimedia, spatial, text, ...)• User-defined functions (UDTs/UDFs) & aggregates• Data blades (UDTs/UDFs + indexing support)• OO goodies for tables: row types, references, ...• Nested tables (well, at least Oracle added these)

• Back to a programming model where applications were loosely bound to the DBMS (e.g., ODBC/JDBC)

• Ex: ADT-Ingres, Postgres, Starburst, UniSQL, Illustra, DB2, Oracle


Why O-R DBs “Fell Flat”

• Significant differences across DB vendors• SQL standardization lagged somewhat• Didn’t include details of UDT/UDF extensions• Tough to extend the system innards (for indexing)

• Application issues (and multiple platform support)• Least common denominator vs. coolest features• Tools (e.g., DB design tools, ORM layers, ...)

• Also still probably a bit too much too soon• IT departments still rolling in RDBMSs and creating relational data

warehouses


Then Came the XML DB Era

<Order id=”123”> <Customer> <custName>Fred</custName> <custCity>LA</custCity> </Customer> <total>25.97</total> <Items> <Item ino=”1”> <product-sku>401</product-sku> <qty>2</qty> <price>9.99</price> </Item> <Item ino=“2”> < product-sku>544</product-sku> <qty>1</qty> <price>3.99</price> </Item ino=”2”> </Items></Order>

<Product sku=”401”> <name>Garfield T-Shirt</name> <listPrice>9.99</listPrice> <size>XL</size></Product>

<Product sku=”544”> <name>USB Charger</name> <listPrice>5.99</listPrice> <power>115V</power></Product>

Note that - The world’s less flat again - We’re now in the 2000’s


What XML DBs Sought to Offer

• One <flexible/> data model fits all (XML)• Origins in document markup (SGML)• Nested data• Schema variety/optionality

• New declarative query language (XQuery)• Designed both for querying and transformation• Early standardization effort (W3C)

• Two different DB-related use cases, in reality• Data storage: Lore (pre-XML), Natix, Timber, Ipedo, MarkLogic,

BaseX; also DB2, Oracle, SQL Server• Data integration: Nimble Technology, BEA Liquid Data (from

Enosys), BEA AquaLogic Data Services Platform


Why XML DBs “Fell Flat” Too

• Document-centric origins (vs. data use cases) of XML Schema and XQuery made a mess of things• W3C XPATH legacy ()• Document identity, document order, ...• Attributes vs. elements, nulls, ...• Mixed content (overkill for non-document data)

• Two other external trends also played a role• SOA and Web services came, but then went• JSON (and RESTful services) appeared on the scene

• Note: Likely still an important niche market...


Now the NoSQL DB Era?

• Not born in the DB world• Distributed systems folks built them• Also various startups

• From caches K/V use cases• Needed huge (SN) scale-out• OLTP (vs. parallel DB) apps• Simple, low-latency API• Need a key, but want no schema for V• Record-level ACIDity, replica consistency varies

• In the context of this talk, NoSQL does not mean• Hadoop (nor SQL on Hadoop! )• Graph databases or graph analytics platforms


NoSQL Data (JSON-based)

Collection(Order){“id: “123”, “Customer”: { “custName”: “Fred”, “custCity”: “LA” } “total”: 25.97, “Items”: [ {“product-sku”: 401, “qty”: 2, “price”: 9.99 }, {“product-sku”: 544, “qty”: 1, “price”: 3.99 } ]}

Collection(Product){“ sku”: 401, “name”: “Garfield T-Shirt”, “listPrice”: 9.99, “size”: “XL” }

{“sku”: 544, “name”: “USB Charger”, “listPrice”: 5.99, “power”: “115V” }

Note that - The world’s not flat, but now it’s less <messy/> - We’re now in the 2010’s, timing-wise


Current NoSQL Trends

• Users coveting the benefits of many DB goodies• Secondary indexing and non-key access• Declarative queries• Aggregates and now (initially small) joins

• We seem to be heading towards a world of ...• BDMS (think scalable, OLTP-aimed, parallel DBMS)• Declarative queries and query optimization applied to schema-less data• Return of (some, optional!) schema information

©2016 Couchbase Inc.©2016 Couchbase Inc.

One Early Example: Apache AsterixDB

22

http://asterixdb.apache.org/

Teaching

Research

Applications





Queries in AsterixDB: AQL (Asterix Query Language)

• Ex: List the user name and messages sent by those users who joined the Mugshot social network in a certain time window:

from $user in dataset MugshotUserswhere $user.user-since >= datetime('2010-07-22T00:00:00') and $user.user-since <= datetime('2012-07-29T23:59:59')select { "uname" : $user.name, "messages" : from $message in dataset MugshotMessages where $message.author-id = $user.id select $message.message};


Another “Relevant” Example: Couchbase Server (!)

CouchDB

Memcached

Membase

+

Caching

Distributed Key-Value

Document

Operational Big Data Database


Couchbase Data Management and Data Access

• Everything is built on top of Key-Value storage

• A Document store is a special case of Key-Value

• Views provide aggregation and real-time analytics through incremental map-reduce

• Global Secondary Indexes provide low latency/high throughput indexes

• N1QL is a language that provides a powerful and expressive (and SQL-like!) way of accessing and manipulating JSON documents – including joins (on keys), aggregates, ..., and access path selection


Coming Next: Couchbase Analytics (DP)

One stop shopping for both operations and analytics

Couchbase QueryOptimized for operational (narrow) queries

Many queries Each touches a little data

Couchbase Analytics

Fewer queries Each touches a lot of data

Optimized for analytical (big) queries


Couchbase: A Database Redefined

Data Query Index Search Analytics

Transport

Unified Administration

Unified Declarative Programming Interface


I Believe that the Fourth Time’s the Charm (!)

• Applications have long needed a shared “place for their stuff”• Relational databases and SQL (along with 1NF) drastically

changed the game in the 70’s and 80’s• Rows, columns, and declarative queries• Tremendous advances in algorithms, indexing, and parallelism

• Since then, applications have unsuccessfully sought more “stuff-friendly” places• Object-oriented DB systems• Object-relational DB systems• XML DB systems

• But this time it’s different – really!• NoSQL != no queries, but rather, no 1NF and no (or optional)

schema• AsterixDB, Couchbase Server, and now Couchbase Analytics are all

examples


Related Connect16 Sessions

• Overview of Couchbase Server Architecture• David Haikney• Tuesday, 10:00 am - 10:50 am• Great America 2

• How to Understand and Use the Query Optimizer• Keshav Murthy• Tuesday, 11:00 am - 11:50 am• Great America 2

• Keynote: Beyond NoSQL – Building a Digital Future• Ravi Mayuram• Tuesday, 1:45 pm - 2:45 pm• Hall D

• Utilizing Arrays: Modeling, Indexing, and Querying• Keshav Murthy• Wednesday, 9:00 am -9:50

am• Great America K

• Sneak Peek: Couchbase Analytics• Till Westmann & Yingyi Bu• Wednesday, 2:00 pm - 2:50

pm• Great America 2

• SQL++: SQL for NoSQL• Yannis Papakonstantinou• Wednesday, 3:10 pm - 4:00

pm• Great America 2


Q&A


Thank You!


Share your opinion on Couchbase

1. Go here: http://gtnr.it/2eRxYWn

2. Create a profile

3. Provide feedback (~15 minutes)

http://gtnr.it/2eRxYWn

