Download - ATMOS ARCHITECTURE OVERVIEW
1 © Copyright 2011 EMC Corporation. All rights reserved.
ATMOS ARCHITECTURE OVERVIEW Mark O’Connell
2 © Copyright 2011 EMC Corporation. All rights reserved.
Agenda
• Why Object Storage • Architecture • Data Management at Scale • Multi-tenancy for management at scale
3 © Copyright 2011 EMC Corporation. All rights reserved.
Trends in IT
• Companies, information, and services are global • Growth of unstructured, digital information • 24x7, worldwide access to data • Growth in data access points – laptops, mobiles,
etc. • Scale out architecture, grow with your data needs • Huge worldwide networking infrastructural building
blocks
4 © Copyright 2011 EMC Corporation. All rights reserved.
EMC Atmos – Managing Big Data in the Cloud
• Purpose-built cloud storage platform
• Globally distributed
• Intelligent data management
• Web-scale
5 © Copyright 2011 EMC Corporation. All rights reserved.
Typical Customer Usage Models Build and Manage Enterprise Cloud Storage • Manage global unstructured data • Protect data • Reduce IT storage costs
Make Existing Storage Cloud-Ready • Utilize existing storage resources and
hardware • Atmos Cloud storage in virtual environment
Provide Cloud Storage-as-a-service • Service multiple consumers • Provide secure multitenancy • Meter and bill for usage
Archive to the Cloud • Isolate distributed archives • Manage chargeback • Ensure security and protect data
6 © Copyright 2011 EMC Corporation. All rights reserved.
EMC Atmos Design Principles • Massively scalable infrastructure
– Multiple petabytes; billions of objects – Hundreds of sites, fully distributed – Unified namespace
• Policy-based management – Policy tells content where to go, what
actions to take – Integrated object based metadata – Event driven policy evaluation
7 © Copyright 2011 EMC Corporation. All rights reserved.
EMC Atmos Design Principles • Data services and access
methods – Replication, GeoProtection,
compression, de-duplication, spin-down
– Web Services (REST/SOAP) and file system access
– Full metering and billing infrastructure
• Management at scale – Auto-config and auto-healing
architecture – Multi-tenancy – Manage more with fewer people – Partitioned management services
allows sharing a common infrastructure amongst disparate users
8 © Copyright 2011 EMC Corporation. All rights reserved.
LAN access protocols Atmos Native Client Access API
Firewall
LAN access
WAN access
REST
REST Server
Blob storage
Runs on each node
Runs across nodes
Atmos High-Level Architecture
9 © Copyright 2011 EMC Corporation. All rights reserved.
CIFS
Atmos Native Client Access API
Firewall
LAN access
WAN access
Installable Filesystem
NFS Atmos CAS
REST
GeoDrive Browser
NFS CIFS
Custom Apps
REST Server
CDP log collection and processing
Blob storage
Atmos - Access Architecture
10 © Copyright 2011 EMC Corporation. All rights reserved.
Atmos – Storage Architecture
One per node Locally stores data No communication with other SSs
Atmos Native Client Access API Write K46DJ…3X2
Replicated
Replicated, identifies an MDS
Queues asynchronous
replication requests
Track online storage resources
Policy Management
Policy Management
Policy Management
Storage Service
Storage Service
Job Service
Resource Management
Service A-M N-Z
Metadata Service
Metadata Service
Node 2 Node 1
MDLS MDLS MetaData Location Service
11 © Copyright 2011 EMC Corporation. All rights reserved.
Atmos Storage Architecture • MDS (MetaData Service) design is the key to scalable
global storage • Each MDS manages a slice of the object namespace
– 3x replicated for HA and availability – MDS unavailability only affects some objects
• MDS can use any available storage server – Storage outages do not affect ability to store new data – No impact for reads unless no stored copies are available
• Normal operation favors MDSs local to the client – Reduces latency for most operations while maintaining global
availability • REST protocol for globally scalable access
12 © Copyright 2011 EMC Corporation. All rights reserved.
Creating an Atmos System
13 © Copyright 2011 EMC Corporation. All rights reserved.
Architecture Overview – Building Blocks • Industry standard components
– Processor, memory, etc details
• Standard data center environments
• Factory configured, racked, and cabled
• Flexible, cost-effective service model
• User serviceable
• Up to 30 drives per 3U drawer
• Up to 720 TB per rack
• 120, 240, or 360 drive configurations
WS 360 WS 120/240 Compute Configuration
• 1:15 Server-to-drive ratio • 120/240/480 TB capacity • GbE or 10GbE
connectivity
Capacity Configuration
• 1:60 Server-to-drive ratio • 360/720 TB capacity • GbE or 10GbE
connectivity
14 © Copyright 2011 EMC Corporation. All rights reserved.
REST clients
IFS clients CAS clients
Architecture Overview – Virtual Building Block � Identical software components
and capabilities in a virtual (VMware ESX) environment - Interoperates with HW based
Atmos deployments
� Extends any storage with a web services interface and object storage − Heterogeneous SAN, NAS, and
Object access on a single platform
− Enables scale out unified object storage across data centers or geographies
IS1 IS4 IS3 IS2 RMG
ESX Servers
Atmos RMG, 4 IS, 8 Nodes FC/NFS
REST
LAN Protocols Atmos CAS
15 © Copyright 2011 EMC Corporation. All rights reserved.
Forming a System from the Building Blocks
RMG 2 (London)
RMG 1 (Boston)
RMG 3 (Raleigh)
RMG 4 (Rome)
IS1 IS4 IS3 IS2 RMG
ESX Server
s
• Each physical or virtual building block is typically one RMG
– RMG = Resource Management Group, unit of geographic scaling
– LAN connected nodes, typically collocated
– One RMG can house multiple racks
• RMGs organize the nodes into distinct locations
– Used in policy placement of data – Resource availability tracked locally and
then communicated globally
• Client requests localized via RMG • Responses optimized to use
resources local to the RMG
16 © Copyright 2011 EMC Corporation. All rights reserved.
Atmos Data Storage
17 © Copyright 2011 EMC Corporation. All rights reserved.
Atmos Object Creation and RMG Interactions 1) REST request recv’d 2) MDLS finds a Metadata Service (normally in local RMG), creates
record 3) Local data copy(s) stored, normally synchronous 4) Asynchronous remote data copy(s) queued via Job Service 5) Multiple remote asynchronous copies leverage RMG local resources 6) A record is made of bandwidth and storage consumed
RMG 2 (London)
RMG 1 (Boston)
3
6
2 4
5
1
18 © Copyright 2011 EMC Corporation. All rights reserved.
Atmos Data Path – Data Placement How is it decided where the metadata and data are placed? • User policies guide the
placement decision • Can be based on
– Initiating user – Location of request – User specified metadata – Or more
• Policy specifies – Metadata location – Data protection level – Data protection location
RMG 2 (London)
RMG 1 (Boston)
RMG 3 (Raleigh)
RMG 4 (Rome)
IS1 IS4 IS3 IS2 RMG
ESX Server
s
19 © Copyright 2011 EMC Corporation. All rights reserved.
Atmos Data Path – Policy Specification
Policy specifications detail how an object is to be stored within the system
Metadata is normally placed in the RMG which receives the create request
Specify data storage options – N way mirroring, erasure encoding, location, sync vs async, etc.
20 © Copyright 2011 EMC Corporation. All rights reserved.
Atmos Data Path – Policy Selector Policy selectors use criteria from the object to choose how the object will be stored.
Specify the policy specification, which controls where the data is stored
Specify how the decision is made – user metadata value(s), system metadata, etc
Specify if the policy should be applied when objects are created or when they are updated
21 © Copyright 2011 EMC Corporation. All rights reserved.
Accessing Data in Atmos
22 © Copyright 2011 EMC Corporation. All rights reserved.
Atmos Read interactions with RMG and Policy 1) REST read request recv’d 2) MDLS uses Object ID to identify Metadata Service 3) Metadata Service returns locations of all replicas 4) Closest replica used to satisfy the read 5) A record is made of the bandwidth consumed
RMG 2 (London)
RMG 1 (Boston)
3
5
2 4
5 1
4 3
3
1
23 © Copyright 2011 EMC Corporation. All rights reserved.
Decomposition of an Atmos object • Blob
– The actual data bits of the object, protected according to policy
• System Metadata – Standard information added automatically to the object – E.g. time last modified, time created, size, name, storage policy, etc.
• ACL – Security information about which UIDs may access this object – May be freely changed by the object creator and others with access
• User Metadata – Arbitrary information which can be associated with an object – Can be used to drive policy decisions for the object – Applications independent of the creator can add metadata
• Subject to object ACL permission – Can be declared listable for easy object access
24 © Copyright 2011 EMC Corporation. All rights reserved.
Accessing Your Data • How can I identify the piece of data that I want?
– Use the object id • May require an external database to manage the object ids
– Named objects (filesystem names)
/home/finance/records/2011
25 © Copyright 2011 EMC Corporation. All rights reserved.
Accessing Your Data Using Metadata • Associate descriptive metadata with objects of interest
• Use listable metadata tags to quickly index similar items – Walk the listable tag, possibly examining the object(s) to find the one of
interest, e.g. • Records from 2011 • Records from blackjack table with dealer = “Mark O’Connell”
26 © Copyright 2011 EMC Corporation. All rights reserved.
Accessing Your Data • Atmos provides multiple object access schemes
– Object id • Automatically assigned by the Atmos system • Fastest mechanism to retrieve your data
– Named objects • Optionally associate a name with a particular object • Names are like pathnames (e.g. /mark/files/presentation.ppt) • Supports directory list operations • Additional overhead on R/W to translate name
– Listable tags • Based on metadata in the object • Quickly find a set of objects which share a certain property • Useful for application specific index and search operations
27 © Copyright 2011 EMC Corporation. All rights reserved.
Atmos Multi-tenancy
28 © Copyright 2011 EMC Corporation. All rights reserved.
Architecture Overview Atmos Tenancy Model • Foundation for cloud storage architecture
– Conceptual grouping of resources – Hierarchical model enables flexible application integration – Platform to support millions of users – Provides robust reporting and resource management
• Improves operational efficiency – Manage access and resources per tenant/subtenant – Allows optimal policy definition and management
29 © Copyright 2011 EMC Corporation. All rights reserved.
Blob storage
Atmos Tenancy Model
Tenants • Dedicated network connections
− Control of ingest bandwidth • Controls storage usage (policies)
− N+3, N+6, 2x mirror, etc • Creates subtenants
− Storage system + storage admin
Subtenants • Preconfigured storage system • A distinct object namespace • Can be metered and billed • Manages/creates application
connection credentials
UIDs • Represent an entity which
can connect to the blob store and create/access objects
• Can be metered/billed N+3 2x mirror
Tenant 1 Tenant 2 Tenant 3
30 © Copyright 2011 EMC Corporation. All rights reserved.
Atmos Tenancy Model - Data • Tenants
– Tenancy is a logical compartmentalization of data and resources – Tenants have their own access nodes, security control, storage
policies, and access to the data • Not aware of any other resources except for its own • No access to other tenant’s resources
– All underlying resources shared among Tenants
• Subtenants – Each Subtenant owns a distinct storage environment
• Users, objects and filesystem • UserID needs to be unique only within a subtenant
– Each subtenant manages a disjoint set of data • Stronger than an ACL, like a virtual storage machine
– Subtenant inherits policy specifications and selectors from the Tenant
– Subtenant may reorder policy specifications
31 © Copyright 2011 EMC Corporation. All rights reserved.
Atmos Tenancy Model - Data • Users
– Authenticates to store/retrieve/manipulate data in the cloud – Per-user listable tags (metadata organization) – May share files with other users in the same subtenant (via ACLs)
32 © Copyright 2011 EMC Corporation. All rights reserved.
Atmos Tenancy Model - Management • SysAdmin
– SysAdmin creates tenants – SysAdmin assigns nodes to tenants – SysAdmin manages the physical state of the system
• Upgrades, adding nodes to the system, etc – SysAdmin has no data access, SysAdmin cannot create/alter
policies
• Tenant admin – Tenant admin creates policy specifications – how data will be
stored in the system – Tenant admin creates subtenants (virtual storage environment)
and subtenant admins – Tenant admin assigns policies to a subtenant
33 © Copyright 2011 EMC Corporation. All rights reserved.
Atmos Tenancy Model - Management • Subtenant admin
– Subtenant admin allows access to the storage environment (creates/deletes users)
– Subtenant may have a level of control over storage policies which are in effect
• If the tenant admin allows multiple policies to be used by a subtenant storage environment, then the subtenant admin has a level of control over how the policies are applied
• Users – No management rights
34 © Copyright 2011 EMC Corporation. All rights reserved.
Atmos Tenancy Model – Policy Application
Storage Policy
Subtenants and Assigned Policy Selectors
2 Sync copies 3 Sync EC copies
1 Sync copy 2 Sync copies 1 async copy India
2 Sync EC copies
1 async EC India
Engineering
Default policy Normal
Finance
Criticality Low, use normal Default policy Payroll
IT Beta
Default policy Test
Research
Size > 1MB, use Silver Criticality High, use Gold
Normal Payroll Test Silver Gold
35 © Copyright 2011 EMC Corporation. All rights reserved.
Atmos Management Strategy • Atmos storage was designed to be customized to the needs
of multiple individual environments – Tenant and subtenant model – Multiple individuals can have multiple custom environments on one
storage system
• Atmos management was designed to be similarly customized
• Full REST/POX API available for management operations – Allows customizable management flows
• Management interface could be different for each tenant admin or subtenant admin
– Authenticate as SysAdmin, TenantAdmin, or SubtenantAdmin – Management API can also be used to fully automate Atmos system
management
36 © Copyright 2011 EMC Corporation. All rights reserved.
Default Atmos GUI • Common operations available in the default GUI
– Logins available for SysAdmin, TenantAdmin, SubtenantAdmin – Different screens and information shown based on login
37 © Copyright 2011 EMC Corporation. All rights reserved.
Customizing the Atmos GUI – Atmos Online • Designed for a fully automated compute and storage service • Customized workflow based on the Atmos system management
APIs – Allows automated onboarding of new customers, querying of billing
information, etc • Available at www.atmosonline.com
38 © Copyright 2011 EMC Corporation. All rights reserved.
Atmos Architecture Summary • A multi-site architecture for storing massive amounts of
content • Policy management allows control over data placement
and protection • Multi-tenancy for both data and management
– Allows customization of the storage for multiple, independent environments
– Allows a fine degree of logical separation of data – Allows scoped management control for administrators
39 © Copyright 2011 EMC Corporation. All rights reserved.
See Atmos in Action – Visit Booth 201
Theatre presentations every half hour Find Atmos-powered Storage-as-a-Service options available worldwide See how easy it is to integrate and customize Atmos via the open API and SDK Learn how Centera and Atmos work together for compliance via the cloud Sneak peek at new features – like GeoDrive! Meet with experts, peers and customers
Enter to Win an iPad 2.0
40 © Copyright 2011 EMC Corporation. All rights reserved.
Additional Resources Atmos Web landing page http://www.emc.com/products/family/atmos.htm Video: Why Beth Israel Deaconess Chooses Atmos for Cloud Computing http://www.emc.com/collateral/demos/microsites/mediaplayer-video/beth-israel-deaconess-emc.htm IDC Analyst Paper, EMC Atmos: Making Rain in the Clouds http://info.emc.com/mk/get/SDL?reg_src=web&P.ctp_program_execution.Source_ID=AMA00018247 Analyst Report: ESG: EMC Atmos Cloud Storage Helps Vistaprint Cut Storage Infrastructure Costs and Accelerate Production http://www.emc.com/collateral/analyst-reports/esg-solution-impact-analysis-emc-vistaprint.pdf Sandbox: Atmos Online Developer Network https://community.emc.com/community/edn/atmos
41 © Copyright 2011 EMC Corporation. All rights reserved.
42 © Copyright 2011 EMC Corporation. All rights reserved.
THANK YOU