scaling oracle with pnfs: improving database …...2012 storage developer conference. © insert your...

2012 Storage Developer Conference. © Insert Your Company Name. All Rights Reserved.

Scaling Oracle with pNFS: Improving

Database Efficiency for Scale-out

Architecture

Bikash Roy Choudhury

Solutions Architect, NetApp


Agenda

Comparison betweenNFSv3, NFSv4 and

NFSv4.1

How is pNFS different

pNFS Implementation

Oracle on NetApp Cluster-Mode(Scale-out)

Oracle, pNFS and NetApp Cluster-Mode

2


Personality

Semantics

Authentication

Identification

Permissions

Transport

Caching

Data Locality

Stateful

Supports UNIX & WINDOWS

Strong (Kerberos)

Stringbased ([email protected])

WINDOWS like access

TCP Only

File Delegations

Referrals

NFSv4

Stateless

UNIX Only

Weak(AUTH_SYS)

32 bit UID/GID

UNIX based

UDP & TCP

Ad-hoc

No Remote Access

NFSv3

Stateful

Supports UNIX & WINDOWS

Strong (Kerberos)

Stringbased ([email protected])

More WINDOWS friendly

TCP Only

Files & Directory Delegations*

Referrals

NFSv4.1

Comparison of NFSv3, NFSv4 and NFSv4.1

3

NFSv4.1

• Is a minor release of NFSv4

• Does not modify any NFSv4 features

• Is only in DOT8.1 c-mode

• Bug fixes

• Supports pNFS

• Supports sessions and trunking *


Challenges with regular NFS

Meta-data, data and control share the same

IO path

Single server bottleneck - Limited CPU

File system resides on a single NFS server

Restricts to distribute the workload

Single server design with a single IP

assigned to a single physical interface

With large file size data transmission

becomes the limiting factor

4


5

pNFS

pNFS Protocol

− Minor version of NFSv4.1

− Transparent to applications

Storage Access Protocol

− files (NFSv4.1)

− blocks (FC, iSCSI, FCoE)

− objects (OSD2)

Control Protocol

− Outside of the

pNFS standard

User Applications

NFSv4.1

Generic pNFS layout

File layout

OSD layout

Block layout

SUN RPC iSCSI

SCSI

TCP FCP RDMA

pNFS Protocol

Control Protocol

Metadata Server Data Servers

Client

Storage Access Protocol


Regular NFS Vs. pNFS

pNFS with Data Ontap 8.1

6

Meta-data, Control,

Data use the same path

NFS

Single Stack NFS, Meta-Data, Data , Locks

NFS Client

Meta-Data Path

Data Path

Network

NFS pNFS

Control Path

Network

NFS pNFS

Data Meta-Data, Data

Locks

Data Meta-Data, Data

Locks

pNFS Client

pNFS Client

pNFS Client


pNFS Architecture and Core

Protocols

Isolates Meta-data from data into

different IO paths

Meta-data Server (MDS)

Handles all non-data; pure meta-data

File layout information

Data servers (DS)

Stores meta-data and data; no protocol meta-data operations

Clients read and write directly to data server

Need to have network/LIFs configured on each data server

7


pNFS Architecture and Core Protocols continued

pNFS client

Supported pNFS client (RHEL6.2)

File Layout driver

Protocols pNFS / 4.1 protocol for clients to access MDS

for retrieving layouts

Storage access protocol (NFSv4.1) for clients to access the DS

Control protocol to provide synchronization between MDS and DS

NetApp Confidential – Internal Use

Only

8


CLIENT

SIDE

SWITCH

pNFS implementation in Data Ontap

8.1

NetApp Confidential - Internal Use

Only

9

Meta-Data Path

Data Path

Control Path

N-Blade

pNFS VIFMgr

D-Blade Meta-Data, Data

Locks

N-Blade

pNFS VIFMgr


Locks

pNFS Client

pNFS Client

pNFS Client

N-Blade

pNFS VIFMgr


Locks

N-Blade

pNFS VIFMgr


Locks

VLDB

VLDB

VLDB

VLDB

vol1

vol3

vol2

root

Cluster Interconnect Switch



LIF2

LIF1

LIF3

LIF4

LIF=<ip_address>

Mounting /vol1 using LIF1



Vol Move vol1

Client accessing /vol1

pNFS client


root

vol1 vol2 vol3

Provides Data Locality

Client un -mount is not required

Transparent to client


pNFS workflow

The LIF that mounts on the client becomes the Meta-data server

(MDS)

pNFS operation flow

Opens the file with a OPEN() ,

Client gets the layout of the file, layout has information about DS

When a READ or WRITE request goes to DS

pNFS layout has information about

Constituents of a volume

For flexvol, there is only one constituent

Network location of constituent volumes

Basically, network address that is hosted on the same controller

as the volume

Nodes need to have a LIF configured for the client to get to the remote volumes

10


Challenges With Traditional Storage

for Oracle

11

Data

Data

Logs

Capacity is limited within storage controller

IP is always attached to physical network ports

Re-distribution of load leads to disruption

While adding Oracle scales out (RAC), storage scales up

Moving data for more capacity, maintenance, load balancing needs downtime or outage

Managing disparate islands of storage are not efficient

home

NFS

Real Application Cluster


Cluster-Mode or Scale-Out …

What is it buying for Oracle?

Increased flexibility − Storage virtualization – v-servers

− Cluster namespace

− Unified storage architecture

− Re-distribution of Oracle data across nodes

− IP address is not tied to physical network port

− Data Mobility and Non-disruptive operations

Tiered Storage

− Archival or less frequently used data

− Data for compliance purposes

− Isolate sequential workloads like logs & parallel queries

12


What’s more…

Agility

Quick deployment for new customers

Dynamically adjust to changing SLAs without user/

database disruption

Oracle databases with different performance

needs

With respect to cost and resiliency

characteristics

Scalability – Scale-up/scale-out

Capacity scaling

Spread out the flexvols across the nodes

Network load balancing 13


Still provides the traditional efficiency

Storage efficiency

RAID-DP

Snapshot

Thin Provisioning

Space efficient Cloning

De-duplication

Data Compression

Data Protection

SnapMirror between v-servers

SnapMirror between clusters

14


Icing on the cake…

Manageability

A single point of management

Tracking and monitoring

User interfacing tools like

SnapCreator 3.5 for

backup/restore/clone/replicate

15


NetApp

Cluster-mode

• Balance applications and

data among storage systems

Without Disruption!

• Add controllers and storage

Without Disruption!

• Move temporary resources

to critical applications –

Without disruption!

• Move data between types

of storage to optimize

storage utilization

Without Disruption!

Checklist for Oracle DB With Cluster-Mode

16

Storage– Seamless

Unified – NAS, SAN

Immort– Data is alive and active

Multi-T– Storage Virtualization

Mobilit– Data Motion of Volumes

Efficie – Parity with 7g/7-mode

Manag– Simple & Easy


Oracle on Cluster-Mode with pNFS

Oracle can definitely benefit from pNFS

Less OPEN/CLOSE

Large IOs

Oracle over pNFS is not tested yet

Patches for o_direct just got out

Fedora17 and Linux3.5 kernel has it

Validating Oracle over pNFS is in the roadmap

17


Thank You

[email protected]

18


19

scaling oracle with pnfs: improving database …...2012 storage developer conference. © insert your...

Documents