cumulonimbus cloud tier deep dive & best practices · block sizes (before compression) •...

Cumulonimbus Cloud Tier Deep Dive & Best Practices

Marco Luske

Systems Engineering Team Lead ANZ

David Hills

Cloud Systems Engineer ANZ

Agenda

Quick overview

Basic architecture

Deeper dive

Cloud Tier

Quick overview and basic architecture

Veeam Cloud Tier

glossary

Cloud TierMarketing name given to this feature in Veeam® Backup & Replication™ 9.5 Update 4

Object Storage RepositoryName given to the repository that is backed by Amazon S3, S3, Azure Blob or IBM Cloud

Capacity TierName given to the extent on a Scale-out Backup Repository™ using an object storage repository

Scale-out Backup RepositoryScale-out Backup Repository consisting of Performance Tier and Capacity Tier extents

Veeam Cloud Tier

The automatic tiering feature of Veeam’s Scale-out Backup Repository

Unlimited capacityfor scalable data storage requirements

No double chargesfor storing data in the cloud, no additional per-GB licensing or subscription charges

Native object storage integration with Amazon S3, Azure Blob Storage, IBM Cloud object storage, S3-compatible service providersor on-premises storage offerings

No lock-insagnostic approach to object storageand freedom of choice

Veeam Cloud Tier architecture

The built-in automatic tiering feature of the Scale-out Backup Repository offloads older backup files to cheaper storage, such as cloud or on-premises object storage.

DAS

NAS

Dedupeappliance

PerformanceTier

Capacity Tier

Older backup files

Microsoft Azure Blob

Scale-out Backup Repository

Amazon S3

IBM Cloud

S3-compatible

Required components

Veeam Backup& Replication

server

LicenseScale-out Backup Repository

Object storage repo

Veeam Backup & Replication

repository

Supported Capacity Tier extents

S3-compatible object storageOn-premises storage device (hardware or software-based)S3-compatible cloud object storage from service providers

• AWS S3 Signature v4 https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.htm

Amazon S3S3 and S3 infrequent access (IA)

Microsoft AzureBlob Storage

https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.htm

Compatible platforms

Forums listing:https://forums.veeam.com/object-storage-f52/unoffizial-compatibility-list-for-veeam-cloud-tier-t56956.html

Veeam-ready database:https://www.veeam.com/ready.html

https://forums.veeam.com/object-storage-f52/unoffizial-compatibility-list-for-veeam-cloud-tier-t56956.html

https://www.veeam.com/ready.html

Public cloud object storage

TieringCosts Lifecyle rules

ME

TA

DA

TA

VBK FileVBK File Shell

DATA Chunk

DATA Chunk

DATA Chunk

ME

TA

DA

TA

DATA Chunk

DATA Chunk

DATA Chunk

Veeam difference - Infinite storage

ME

TA

DA

TA

ME

TA

DA

TA

ME

TA

DA

TA

ME

TA

DA

TA

Scale-out Backup Repository(Performance Tier)

Object storage repository(Capacity Tier)

Deeper dive

How does data get offloaded?

Two conditions need to be met:

Operational restore windowPoints are located outside of operational restore windowDictated by policy defined in Scale-out Backup Repository configuration

Sealed backup chainPoints belong to sealed backup chain (ones that won’t be transformed)

Offload process

Backup data is collected from the extents and jobs

Works out what data is a candidate to move based on conditions

Uses veeamagent.exe to offload data to Capacity Tier

The offload process is done via Scale-out Backup Repository offload job

• Runs every four hours

• Can be manually run via console or PowerShell

Operational restore window

Mon Tue Wed Thu Fri Sat Sun Mon


FORWARD

REVERSE

.vbk+.vibs

.vrbs (except latest two)

Sealed backup jobs

Mon Tue Wed Thu Fri Sat Sun MonMonMonMonMon

WeeklyMonthlyQuarterlyYearly

GFS ONLY

Sealed backup copy jobs

Performance Tier Capacity Tier


Operational restore window

Mon Tue Wed Thu Fri

ab

c d e f g h

ij

ab

c d e f

Dehydration


a b

c

d

a b

METADATA

File1 File2

BLOCKS

a b c d

VM BACKUP FILES

DEHYDRATION INDEX (PERSISTENT)

Dehydration


a b

c

METADATA

File1

BLOCKS

a b c

VM BACKUP FILES

REHYDRATION INDEX (RUNTIME)

a b

d

FullBackup2.vbk

Rehydration

FullBackup1.vbk

d

Performance Tier

Capacity Tier

File1

Blocks

a b c

Restore target

a b

a

b

c

FullBackup1.vbk

FullBackup2.vbk

Restore

Block cloning and deduplication

Once transferred, only metadata remains in each backup file

SOBR Offload Job:

• Offloads data blocks• Replicates backup files with metadata

.vbk .vib .vib .vib .vbk

.vib .vib .vib.vbk

.vib

Replicated versions of backup files with metadata

Offloaded blocks of data in object storage

Scale-out backup repository extents

Inactive chain Active chain

A B C

Mon Tue Wed Thu Fri Sat

Metadata

Data blocks

Figure A Figure B

Object Storage Repository in the same scale-out backup repository

Block sizes (before compression)

• Local target (16 TB + backup files): 4,096 KB data blocks• Local target: 1,024 KB data blocks• LAN target: 512 KB data blocks• WAN target: 256 KB data blocks

The smaller the block size, the more calls we make to object storage to upload data which will impact cost to upload to Amazon S3 or Azure Blob

• Intelligent block recovery will significantly reduce egress charges

Calculating Capacity Tier sizing

• Modeled on ReFS fast clone technology (http://rps.dewin.me)

Estimating sizing and cost

http://rps.dewin.me/alpha

S3: What’s in the bucket?


SOBR Offload Job:



.vib .vib .vib.vbk

.vib





A B C


Metadata

Data blocks

Figure A Figure B


Resiliency


SOBR Offload Job:



.vib .vib .vib.vbk

.vib





A B C


Metadata

Data blocks

Figure A Figure B


The Veeam difference

Offloaded backup files remain on the Performance Tier extents

All Veeam restore functionalities “just work” transparently

Intelligent block recovery

No double charges

No Vendor Lock-in

Thank you

cumulonimbus cloud tier deep dive & best practices · block sizes (before compression) •...

Documents