2 components by storage type application os file system database system lvm raw partition scsi...

31
SCALEWAY

Upload: elvin-little

Post on 24-Dec-2015

227 views

Category:

Documents


3 download

TRANSCRIPT

Page 1: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

SCALEWAY

Page 2: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

2

Components by Storage type

Application

OS File System

Database System

LVMRAW

Partition

SCSI Device Driver

SCSI BUS Adapter

Computer System

Application

OS File System

Database System

LVMRAW

Partition

SCSI Device Driver

FC Host Bus Adapter

Computer System

Application

OS File System

Database System

LVMRAW

Partition

iSCSI Layer

TCP/IP Stack

Computer System

NIC

SCSI Interface Adapter

FC Interface Adapter

IP Network

iSCSI Appliance

NIC

TCP/IP Stack

iSCSI Layer

I/O Bus Adapter

Application

Computer System

NIC

OS FileSystem

I/O Redirector

NFS/CIFS

TCP/IP Stack

File I/O

IP Network

NAS Appliance

NIC

TCP/IP Stack

File System/LVM

Device Driver

Application

Computer System

NIC

OS FileSystem

I/O Redirector

NFS/CIFS

TCP/IP Stack

NAS Cluster Node

NIC

TCP/IP Stack

Cluster File System/LVM

Device Driver

NAS Cluster Node

NIC

TCP/IP Stack

Cluster File System/LVM

Device Driver

NAS Cluster Node

NIC

TCP/IP Stack

Cluster File System/LVM

Device Driver

SAN

SAN

DAS SAN iSCSI NAS Scale-Out NAS

Source: Google

Block I/O

Block I/O

Page 3: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

3

Scale-up NAS vs Scale-Out NAS

Scale-Out NASTraditional NAS

• Limitation of performance & expansion- Single File System size

- # of Active Header

• Over-provisioning of storage• Multi mount point• Need to manage location of file

stored• Duplicated file storing(Read

performance down)

• TCO saving• Just-in-time expansion• Each expansion available of

performance and capacity• Higher utilization rates• Non-disruptive technology• Unification of management• Automated management by policy

Client

Network

SCALEWAY New SCALEWAY

Page 4: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

4

Introduction effects of Scale-Out NAS▶Scalability / performance / availability / management efficiency

Scale-Out NAS

Scalability

Performance

Availability

Managementefficiency

Page 5: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

5

SCALEWAY Network Architecture

Volume

LAN FCremark :

User LAN Fabric (1GbE, 10GbE, iSCSI, IB)

8Gb FC

Storage Controller

RAID Controller

RAID Controller

Disk1 Disk2 Disk3 Disk4

Disk5 Disk6 Disk7 Disk8

Disk9 Disk10 Disk11 Disk12

Disk13 Disk14 Disk15 Disk16

RAID Technology

FC Fabric

Cluster LAN Fabric (1GbE, 10GbE)

8Gb FC

Each node Active-Active

Cluster Management Network

User Network

SCALEWAY NODE1

1GbE/10GbE

8Gb FC

SCALEWAY NODE2

8Gb FC

SCALEWAY NODE3

8Gb FC

SCALEWAY NODE n

8Gb FC

1GbE/10GbE 1GbE/10GbE 1GbE/10GbE

Page 6: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

6

▶Clients can access storage volume through every node▶Scale up to 1000 nodes and each node is active mode▶Automated node fail-over within 10 seconds, node and storage added

uninterruptedly ▶Performance increased by adding node but no bottleneck

SCALEWAY I/O Handling

TCP/IP LAN

SCALEWAY

Disk Array

ClientGroup

Client Data

. . . .

Storage Volume

Page 7: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

7

▶I/O distribution to normal nodes when Node or N/W failures▶Automatically data stored on shared storage via other node

Auto Fail-Over

/Data

SCALEWAY Shared StorageClients

Page 8: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

8

scalable up to over 1000 nodes

Clients

TCP/IP LAN

SCALEWAY . . . .

• Performance increased linearly by adding node

• Can solve performance issue by adding node when increase of users and I/Os(Scale up to over 1,000 nodes)

• Can add node on-line uninterruptedly

Flexible structure for performance improve-ment

Group 4Group 1 Group 2 Group 3

10G NIC 1G NIC

Band-width20,000

1(Gb

)

1(Node)

2 3 Max1,000

Node

.

.

.

. . .

n Node

1Gb NIC n x 4GB

10Gb NIC n x 20GB

Page 9: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

9

▶Configures disks online only required

(unnecessary to buy all predicted capacity on initial step)▶Reconfiguration of Filesystem based on striping

(Only expansion of volume space)▶No data corruption, can configure automatically or manually

Thin Provisioning – Thin Quota

When adding new storage

Page 10: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

10

▶ Utilize snapshots as backup and test without damaging source data

Instant replication based on POINT-IN-POINT

Recoverable with replicated information by schedule in case of logical data

corruption

Replicated volumes are available in multi-purpose

Maximum 255 snapshots available per one file system

Provided for free of charge

Snapshot

Snapshot

Storage

WED15:00

TUE10:00

MON 09:00

Page 11: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

11

▶ Asynchronous Replication with file level▶ Low cost required through LAN/WAN infrastructure▶ Heterogeneous storages allowed between primary and secondary

center

Replication

B SiteA Site

TCP/IP LAN

SCALEWAY Cluster

Console

TCP/IP LAN

SCALEWAY Cluster

Console

Original Volume

A

WAN/LANBackup

Volume B

Original Volume B

Backup Volume

A

Page 12: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

12

▶Integrated management of storage by management S/W based on GUI▶Access through CS or web browser▶Protocol management and control, Thin & General folder management▶Management of Monitoring, Data and performance

OntopView (Management Tool)

Page 13: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

13

Integrated Task Manager

Node ViewNode configuration and control

Filesystem ViewFilesystem information

Shared Folder ManagerProtocol & User ACL

Control of user and protocol

▶ Provide functions : Node Reboot, Poweroff (included cluster Start/stop)▶ Show usage of CPU and Memory ▶ Configure and Control filesystem, shared folder and various protocols on a single view▶ Contro the Quota value of shared folder▶ Powerful control of access users to the shared folder

Page 14: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

14

Monitoring Status Manager

Node Status ViewPower, Temperature, Fan status

Node Performance Status View

Status Alarm ViewError alert

Page 15: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

15

Performance Manager (Chart & Grid Report)

Node Performance Chart ViewHourly, daily, monthly and yearly inquiry

Page 16: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

16

Notify Configuration

Status Alarm Notify Configuration

▶ Send SMS and e-mail in case of node failure▶ Notify to each administrator assigned for node

Page 17: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

17

User Manager

User ManagerOntopView user registration, change, delete

Page 18: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

18

SLB (SCALEWAY Load balancer)

B Site

Usage 60% Usage 58% Usage 61% Usage 59%

User N/W

▶ Assure optimal storage performance by load-balancing Distribute load of clusters to underutilized node by monitoring node utiliza-

tion

Enhance user convenience

User can see reachable network drive list by entering his ID & PASWD

Provide ease of management

As providing exe. file, available immediate execution and no need of individ-

ual management

according to the user environment

Applicable immediately by substituting old file in version upgrade(No Instal-

lation)

High Availability

For cluster structure, support quick switch to another node in node failure

Page 19: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

19

TCM (Transfer Control Manager)

B Site

Support backup, replication, transmission and recovery of data by policy(automatic)

Support manual backup, replication, transmission and recovery of data by user’s

opinion(manual)

Maximum utilization of existing N/W and system infrastructure

Spare time of data transmission by minimum use of user N/W and computing resource

Strengthen ease of storage management as user-base

Strengthen business productivity of user

Source(SAN/NAS)

Source(NAS)

Destination(NAS)

Temp(NAS)

R

TCM Server

TCM ClientR

SAN

Ethernet(LAN/WAN)

▶ TCM is a total data management solution by policy decision(automatic) or user discretion(manual) as generation, storage and transmission of

various data. And can save cost by maximum utilization of existing network

Page 20: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

20

Component

• SCALEWAY• Storage• SAN Switch• Network

Switch

Physical Configuration

1/10G Ethernet switch for Interconnect

SAN Switch

SCALEWAY Node

Storage

Page 21: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

21

L Manufacturing company

▶ Customer requirements• Efficient management for inspection images created from production line• Efficient capacity management according to expansion or reduction of production lines• Performance guaranty for high capacity data management

▶ Benefits after Introduction• Prompt check relevant items in case of defective goods

Integrating and storing data stored in each facility PC increases utilization of stored data linked with image management system

• Establish standard of future data management• Collecting the trends of data increase and

performance based on current system establishes standard of system infrastructure linking with future business

• Improve data utilization rate By collecting log data, extract various data for

production items and utilize for future production process

CACHE 12GB

NAS Gateway Spec

HDD 500GB 7.2K * 2 EA

FC PORTS FC 8 Gbs x 2EAN/W PORTS 10GbE x 2EA

# of N/W ports connected with BackBone : 1 Giga Ethernet: 1 Port UTP type(DB server) 10 Giga Ethernet: 8 Port(SR-LC type) 4 ports for ECM Server 4 ports for SCALEWAY

SCALEWAYNASGateway

Facility PC collecting system * 300ea

image mngmt. system

ECM Server * 2EA

DB Server * 1EA

1 1 1 1

2 2

10 GbE area

1

BackB

on

e N

etw

ork

(1

Gb

E)

2TB SATA 7.2K RPM DISK

Usable 1,200 TB or 1,400TB

R5(15D+1P)

1 1 1 1 1 1 1 1 11 GbErange

...

BackB

on

e N

etw

ork

(1

0G

bE)

10G Ethernet remarks :

1G Ethernet 8Gbps FC

Page 22: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

22

Finance customer ▶ Customer requirements

• Data Integration

• Accessible to storage for virtualized

system(VM Ware)

• Support SAN/NAS simultaneously

Availability

• Performance guaranty when service

increases

▶ Benefits after introduction• Secure management convenience

with implementing integrated

infrastructure

• Separated operation for critical data

and backup data

• Real-time data sharing with SCALEWAY

• Provide best service and business

continuity with only 3 nodes

• Performance increased by adding nodes4TB(FC HDD) 10TB(SATA HDD)

SCALEWAY NodeGroup

Gigabit Switch

HP Blade ServerVM Ware Implemented

Windows 20031G * 3 LAN

1G * 4 LAN

SAN Switch

SAN

Page 23: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

23

Public customerDigital Forensic Investigation

squad(6F)

Computer room(5F)

Digital forensic teamWorkstation for analytics * 20

Target HDD

50M distance between floors

8Gb HBA 4 * 31Gb NIC 2 * 310Gb NIC 4 * 34,000MB/sec Bandwidth * 3

Storage systemDual Controller * 51,200MByte/sec read per-formance * 58.7TB for each storage * 5Total 43.5TB

Data is loaded and stored to storage from Seized HDDs for evidence and analyze to prove criminal facts.SCALEWAY provides work efficiency and time value

Page 24: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

24

C Broadcasting system ▶Customer requirements• Meet required performance for each 3

jobs

• Low resolution storage is for storage replication

• Guarantee performance with adding jobs in the future

▶Benefits after introduction• High-resolution :( performance on actual

environment : 1,650GB/s) SCALEWAY : 3nodes (10G)

AMS2500*2ea, SAS 240TB

• Temporary :(performance on actual environment : 768GB/s) SCALEWAY : 2nodes (10G)

AMS2500*1ea, SAS 36TB

• Low-resolution :(performance on actual environment : 212GB/s) SCALEWAY : 2nodes (10G)

AMS2500*2ea, SAS 36TB * 2, TrueCopy

• User satisfaction for BMT performance File servers have and manage data

previously

Using data sharing and Agent of SCALEWAY gives significant performance improvement and satisfaction to the customer

10G Ethernet remarks : 1G Ethernet 8Gbps FC

TRANSFER MANAGER

01~04

IN-GEST01~10

TRANSCODER1~10

INGEST MANAGER

DIVA ACTOR05~10

.... .... ....

10/1Gigabit Ethernet

....

high-resol storage(162TB)

low-resol storage(38TB)

Temporary storage(38TB)

STREAM

SERVER 1,2

WEB/WASSERVER 1,2

WEB/WASDev.

high1 high2 high3

low1 low2 temp1 temp2

Page 25: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

25

M Broadcasting system

▶Customer requirementsM emphasized stability, security and data sharing in building Digital news system and presented as belows.

• High performance FTP service for external connection

• Sharing editing data• Support high capacity file system(over

20TB)• Uninterrupted service in node failure• Guarantee performance improvement in

accordance with service increase

▶Benefits after introductionMeet the required performance : About over 80 clients including reporters send-ing many articles from outside and each department in the company upload, edit and transmit articles and media data via SCALEWAY are satisfied with perfor-mance.

• High performance service through 2-Scaleway nodes

• Shorten editing time by sharing data real time between inside and outside of the company

• Provide work continuity in case of failure with 2 nodes operation.

• Provide scalability in case of service increase in the future

remarks : 1G Ethernet 4Gbps FC

SCALEWAYNASGateway

BackB

on

e N

etw

ork

(1

Gb

E)

DS4700 (SATA) 90TBActiveRAID (SATA) 128TB R5(4D+1P)

FTP ser-vice

CMSdata

transmit

Streaming service Catalog

HD NLE SD NLE

Page 26: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

26

Public

Reference

Broadcast & Media

Page 27: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

27

27

ManufacturingFinance Others

Reference

Page 28: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

28

Highlights of SCALEWAY

Extreme performance 1G/10G/iSCSI/IB etc. supports various N/W

Unlimited Scalability Minimum 2 up to Maximum 1,000 nodes

Large single/multi volume

Support up to 8PB as a single volume(available multi volume)

Flexibility of configu-ration

Complex configuration of various homo/heterogeneous storages

Perfect stability RAID, Replication, Snapshot

Cost saving Planned expansion from initial to post

Thin Provisioning Free change of capacity and prepare unpredicted storage expansion

Convenience Ease of Maintenance and user based integrated storage management

Performance im-provement

Promotion for best performance through combining sys-tem utilization properly

Bundle solutions Snapshot, Replication, Thin provisioning

Page 29: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

29

SCALEWAY SG-3000 Appliance

  Item Feature

Node

Standard Configuration

Active / Active, Default NAS Gateway 2node

Cluster Expansion Over 1,000 nodes Support

Default Form Factor 2U / 2node

Spec.

Processor Intel Xeon multi-Core Processor (Max. 2 CPU)

Memory 12GB(1G) / 24GB(10G) – Max 192GB support

Filesystem size 8PB(Single filesystem), 2EB(Max capacity)

LAN Connectivity

Dual Gigabit Ethernet / node, Optical or Copper Quad Port Support GbE, 10GbE, iSCSI, Myrinet, IB, HPS Sup-ports

Fibre Connectivity

Dual Fibre Channel Ports, 4Gbps or 8Gbps Optical – LC Type / Single, Dual, Quad Port HBA

Page 30: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

30

In storage SG-1000

  Item Feature

Node

Standard Configuration

SCALEWAY Engine embedded NAS Storage

Default Form Factor

2U(Engine) + 2U/4U(Storage)

Spec.

Processor Intel Xeon Multi-Core Processor (Max. 2 CPU)

Memory 12GB(1G) / 24GB(10G) – Max 96GB support

Filesystem size 8PB(Single filesystem), 2EB(Max capacity)

LAN Connectivity

Dual Gigabit Ethernet / node, Optical or Copper Quad Port Support GbE, 10GbE, iSCSI, Myrinet, IB, HPS Sup-ports

Fibre Connectivity

Dual Fibre Channel Ports, 4Gbps or 8Gbps Optical – LC Type / Single, Dual, Quad Port HBA

Page 31: 2 Components by Storage type Application OS File System Database System LVM RAW Partition SCSI Device Driver SCSI BUS Adapter Computer System Application

31

Thank you