build web-scale applications - static.rainfocus.com · . copyright © 2017, oracle and/or its...

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Build web-scale applications with NoSQL and Node.JS using Docker

Mayuresh A Nirhali Principal Product Manager Oracle NoSQL Database, BerkeleyDB August, 2017

Confidential – Oracle Internal/Restricted/Highly Restricted


Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Confidential – Oracle Internal/Restricted/Highly Restricted 3


• Mayuresh A Nirhali

– Principal Product Manager at Oracle,

– Over 16 years of experience in Platform and Infrastructure domain; a decade at Sun Microsystems contributing to all layers of technology stack, currently building cloud services at Oracle

– Twitter/Github: @nirmay


About Me


Agenda

What are we building today?

Architecture and Technologies (Docker, NoSQL, Node)

Show me the Code, Deployment

Single command deploy to Public Cloud

1

2

3

4



What are we building today??

•Quotes web-app for Oracle Code attendees

• Identify and build services e.g. db, server, UX

• Build docker images for all services

• Recipe that can deploy all services at once

• Move the services to cloud – CI/CD


Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 7

http://140.86.13.67:3000/ (Live Now - Add your quotes)

http://140.86.13.67:3000/


Architecture and Technologies



Why NoSQL Database?

9

Flexible Data Model

Globally Distributed,

“Always On”

Low Latency Access, High volume ingest

Lower TCO, commodity

HW scale-out

ERP

EAM

Inventory Control

Accting & Payroll

Process Mgmt

Business Analytics

CRM

…

Driver

Application Real Time Event

Processing

Customer Portals

Online Banking

…

Mobile Data Management

Time Series & Sensor Data Mgmt

Last 30+ years and so on Recent trends


• Simple to set up and use

• JSON, Table, Key-value data model

• Java, C, Node.JS, Python, .Net, SQL

• Easy to evolve schema

• Configurable transaction semantics

– Shard Local ACID

• Secondary Indexing

• Parallel operations

• Trusted foundation (Berkeley DB)

• IoT Cloud Service uses Oracle NoSQL Database

10

Why Oracle NoSQL Database ? Addresses Developer Needs

Application

Storage Nodes Datacenter B

Storage Nodes Datacenter A

Application

NoSQL DB Driver

Application

NoSQL DB Driver

Application


3 DenseIO nodes to run the Oracle NoSQL Database.

DenseIO node: 36 OCPU, 512 GB RAM, 28.8 TB NVMe SSD, 10 Gbps network.

YCSB Benchmark

9 Shards, Rep factor = 3


Oracle NoSQL Database on Oracle Bare Metal Cloud

https://blogs.oracle.com/nosql/oracle-nosql-database-performance-on-oracle-bare-metal-iaas

https://cloud.oracle.com/en_US/bare-metal-compute/pricing



















Building the Server application?


Why NodeJS?

Asynchronous and Event driven

Lightweight and Powerful


Architecture

Application

Oracle NoSQL Database

Proxy

HTTP Request

JSON Response

Data


Docker Architecture

Oracle Confidential – Restricted 15

• Docker client – Command Line Interface (CLI) for interfacing with the Docker

• Dockerfile – Text file of Docker instructions used to assemble a Docker Image

• Image – Hierarchies of files built from a Dockerfile, the file used as input to the docker build command

• Container – Running instance of an Image using the docker run command

• Registry – Image repository


Why Docker? = Resource Efficiency

Image credits

http://linux-unix-windows.blogspot.in/2015/10/containers-vs-vms-whats-difference.html


Why Docker? = Portability

Build Once, Run Anywhere On-Prem workstation | Data Center Servers | Public/Private Cloud


Docker 3

Docker 2

Docker 1


Architecture

Application

Oracle NoSQL Database

Proxy

HTTP Request

JSON Response

Data


Just 1 Docker container for web-scale db? Machine as Docker Container

19

R1

R2

R3

R2

R2 R1

R1 R3

R3

Shard1 Shard2 Shard3

Agents

A

A

A

Machine1

Machine2

Machine3

Application

NoSQL Driver

D

D D

D

D D

Elastic Auto Sharding

Elected Master

Read any node

Data rebalance on expand


Let’s build it!



Pre-requisites

• Install

– Docker

– Docker-compose



Download docker image for Oracle NoSQL Database


http://hub.docker.com/r/oracle/nosql


• Enterprise Edition & Community Edition

• Sample scripts to create nosql cluster and other components

• > docker run -d --name=kvstore oracle/nosql


Do more with Oracle NoSQL Database Docker


• Dockerfile

FROM oracle/nosql:latest

COPY scripts/* /scripts/

COPY load/* /kv-4.3.11/

https://github.com/oracle/cloud-native-devops-workshop/blob/master/containers/docker001/Participant-Guide.md

• Create Tables (Use JSON/Table model)

– create table IF NOT EXISTS oraclecode_table (

id string, event string,

name string, quote string,

primary key (shard(event), id) );

• Load data (if any)

– import -table oraclecode_table -file data.json


Build custom Oracle NoSQL Database Image












NodeJS – package.json {"name": "docker-node-ondb", "version": "1.0.0", "description": "Node + Oracle NoSQL development on Docker",

“main": "index.js", "scripts": { "start": "nodemon app.js“ },

"dependencies": {

"body-parser": "^1.17.2",

"ejs": "^2.5.7",

"express": "^4.15.3",

"nodemon": "^1.11.0",

"nosqldb-oraclejs": "^4.3.10",

"shortid": "^2.2.8"

}



NodeJS – app.js var nosqldb = require('nosqldb-oraclejs');

// Create a configuration object

var configuration = new nosqldb.Configuration();

// recommended to run Proxy on the same node as NodeApp,

//but for this demo, we run separate docker

configuration.proxy.startProxy = false;

// Create a store with the specified configuration

var store = nosqldb.createStore(configuration);

http://docs.oracle.com/cd/NOSQL/html/driver_table_nodejs/GSGTablesNJS/index.html


http://docs.oracle.com/cd/NOSQL/html/driver_table_nodejs/GSGTablesNJS/index.html


Docker-compose

• Define and run multi-container docker application

• Specify docker images and configuration in a YAML file

– Override ports, entrypoint, command etc. as needed

• Run docker-compose up -d



docker-compose.yml kvstore:

container_name: "kvstore"

image: nirmay/oraclenosql-appdata:0.1

ports:

- 5000:5000

environment:

- RNHOST=kvstore

- ZONENAME=NoSQL-OCCS-Zone

command: /scripts/deploy-admin.sh

kvproxy:

container_name: "kvproxy"

image: nirmay/oraclenosql-proxy:4.3

ports:

- 7010:7010

depends_on:

- kvstore

command: /bin/bash -c “java -cp lib/kvproxy.jar:lib/kvclient.jar oracle.kv.proxy.KVProxy -helper-hosts kvstore:5000 -port 7010 -store kvstore"

nodeapp:

container_name: "nodeapp"

image: nirmay/node-oraclenosql:0.1

ports:

- "127.0.0.1:3000:3000“

depends_on:

- kvproxy

command: /bin/bash -c “npm start -- kvproxy:7010 kvstore:5000 kvstore"



Ability to manage containers across multiple distributed physical nodes or virtual machines

High Availability Clusters

Load Balancing

Confidential – Oracle Internal/Restricted/Highly Restricted

Container Orchestration


Oracle Cloud and Docker Containers

Container CS Compute CS Application Container CS

DIY Container Management

Oracle Managed Container Service

Docker-based Cloud Polyglot Platform

IaaS CaaS PaaS

31


Oracle Container Cloud Service


• Create a new service for each of the staged docker images

• Create a new Stack and wire them together

• Deploy


Build a new OCCS Stack

nirmay/oraclenosql-appdata:0.1

nirmay/oraclenosql-proxy:4.3

nirmay/node-oraclenosql:0.1


http://140.86.13.67:3000/ (Live Now - Add your feedback)

http://140.86.13.67:3000/


Continuous Integration with Docker • CI Server builds Docker image and pushes image to a Docker Registry

• Image deployed in next SDLC region

35 Oracle Confidential – Restricted


Safe Harbor Statement

The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Note: The speaker notes for this slide include instructions for when to use Safe Harbor Statement slides.

Tip! Remember to remove this text box.


build web-scale applications - static.rainfocus.com · . copyright © 2017, oracle and/or its...

Documents