solving the azure storage maze - amazon web services · azure: •54 announced azure regions around...
TRANSCRIPT
2
Aidan Finn
Microsoft MVP (Azure)
Principal Consultant – Innofactor Norway• https://www.innofactor.com• Cloud Platform Infrastructure
Managing Director – Cloud Mechanix • http://cloudmechanix.com• Custom-written, hands-on Azure Training
@joe_elway
http://aidanfinn.com
3
About Altaro Software
Altaro is a fast-growing developer of easy-to-use backup solutions for Managed Service Providers (MSPs), IT resellers and IT departments
Virtual Backup trusted by 40,000+ Customers, 9,000+ Partners and 1,200+ MSPs worldwide
Flagship product: Altaro VM Backup for Hyper-V and VMware –altaro.com/vm-backup
MSP program – altaro.com/msp
youtube.com/c/AltaroSoftware
linkedin.com/company/altaro
@AltaroSoftware
facebook.com/AltaroSoftware
5
The Azure Storage MazeThe old simplified version!
Storage Account
StartHere
Managed Disk
ChooseTier
Standard SSD Premium SSDStandard HDD
ChooseService
Table FilesBlob Queue Page Blob & Disk
GPv2
Hot Blob
GPv1
Cool Blob
LRS
ZRS
GRS
RA-GRS
Standard
Premium
Ultra SSD
You AreHere
6
Storage Account
StartHere
Managed Disk
ChooseTier
Standard SSD Premium SSDStandard HDD
ChooseService
Table FilesBlob Queue Page Blob & Disk
GPv2
Hot Blob
GPv1
Cool Blob
LRS
ZRS
GRS
RA-GRS
Standard
Premium
Ultra SSD
Premium File Storage
The Azure Storage MazeThe current simplified version! You Are
Here
8
Azure:• 54 announced Azure regions around the world• Multiple data centres per region• Millions of physical Hyper-V hosts• High levels of resilience
How many SANs are in Azure?a. 1,000b. 10,000c. 100,000d. 1,000,000
QuestionAzure Storage
9
There are no SANs in Azure
Software-defined storage• Commodity hardware PLUS intelligent software• Reduces costs• Increases scale & performance
See Windows Server 2016 Storage Spaces Direct (S2D)
AnswerHow many of you opted for ZERO?
10
A cluster of servers with replicated internal disks
Provides storage services on the network
Provides at least 3x redundancy
Exists in a single room in a single data centre in a single region
Optional replication outside of the room
Storage ClusterSoftware-defined storage for Azure services & customers
Storage Stamp
LB
Partition Layer
Front-Ends
DFS Layer
Intra-stamp replication
11
Resilience LevelsWhat happens if components fail
LRS ZRS GRS RA-GRS
Full Name Locally-Redundant Storage Zone-Redundant Storage
Geo-Redundant Storage
Read-Access Geo-Redundant Storage
Storage Profile 3 synchronous copies 3 synchronous copies LRS PLUS GRS PLUS
Locations Single storage clusterSingle computer room
3 storage clusters3 availability zones
3 asynchronous copies in additional region
Asynchronous copies are read-accessible
Object durability
11 9s 12 9s 16 9s 16 9s
13
Think of it like a LUN
Traits:• Storage Account Name
oMust be globally uniqueoPrefix for Azure-managed DNS name
• Resiliency level• Performance level• Tier (more later)
Up to:• 2 PiB (US/Europe) / 500 TiB in size (Elsewhere, including UK)• 20,000 IOPS
The Storage AccountA storage address
14
8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096?• 1 TB = 1024 GB• 1 GB = 1024 MB
Not anymore – thank you, Apple:• 1 TB = 1000 GB• 1 GB = 1000 MB
Computer science:• 1 Tebibyte (TiB) = 1024 Gibibytes (GiB)• 1 Gibibyte = 1024 Mebibytes (MiB)
What is a TiB?Remember computer science?
15
Standard tier:• Based on HDD• Offers all storage account services
Premium tier:• Based on SSD• Higher performance• Only support virtual hard disks (VHD) / Page Blob & Disk
Storage Account PerformanceStorage Tiers
16
Different support for:• Storage tiers• Storage account resilience
Different billing rates
Storage account services• Block Blob• Page Blob & Disk• Table• Queue• Azure Files
Storage Account ServicesAutomatically chosen based on your usage
18
Ultra cheap form of storage
Often used for huge capacity storage by applications• Azure, e.g. backup of Azure VMs• On-premises, e.g. Altaro Backup
Features blob tiering
Also in Azure for diagnostics/trace logs
Block BlobBasic form of file storage
19
Tiers:• Hot: Most expensive per GB, cheapest transaction costs• Cool: Medium cost per GB, medium transaction costs• Archive: Cheapest per GB, most expensive transaction costs
Minimum storage duration:• You are billed for a minimum amount of time in cool & archive tiers• Cool = 30 days minimum• Archive = 180 days minimum
Archive tier• Blobs are not accessible• Must be rehydrated to cool or hot tier• Can take up to 15 hours
Blob TieringCost optimisation without path/name change
20
Storage Account Types4 types – note Blob Tiering
Hot Blob Cool Blob General Purpose v1
AKA GPv1
General Purpose v2
AKA GPv2
Purpose Legacy support Legacy support Many kinds of
storage
Many kinds of
storage
Supports Blob Blob Blob, Disk, Files,
Queue, Table
Blob, Disk, Files,
Queue, Table
Blob Tiering All 3 tiers
Hot by default
All 3 tiers
Cool by default
No All 3 tiers
Relevance today None None Yes None in
infrastructure
Why Use It? Don’t Don’t IaaS roles
Diagnostics &
performance
If your PaaS app
understands & does
blob tiering
Notes Expensive
transactions charge
Expensive
transactions charge
Micro transactions
charge
Expensive
transactions – tier!
21
JSON policy in a GPv2 storage account
Automatic:• Moving of blobs to cool/archive tiers• Deletion of blobs
Optional filter matching
Based on the age of a blob/file
Blob Tiering Lifecycle ManagementAutomated Tiering
22
GPv2 only
Policy based service• Time-period OR• While a “case” is active
Force retention of files without modification
Features:• Ability to upload files• Cannot delete/modify files while a policy matches
Immutable Storage for BlobsWORM storage
23
Policy based service
Snapshots of blobs are retained when:• A blob is modified• A blob is deleted
You can recovery older versions of a blob
Soft DeleteA recycle bin
24
Azure Security Center feature• Machine learning reads the matrix• IDS just at the firewall is so 2010!
Detect unusual & potentially harmful events
Alerts:• Anomalous access patterns• Anomalous extracts/uploads• Suspicious activities
Advanced Threat ProtectionGenerally available yesterday
26
Based on flash storage• No tiering today
Huge capacities – with better performance:• Lower latency• Higher IOPS• Higher MB/s
Intended for customers who need:• Large capacities• High speed• Example: video editing
Premium Block BlobGenerally available recently
28
Standard tier storage accounts:• Up to 500 IOPS per VHD• Up to 60 MB/s throughput per disk• Pay based on amount of data
Premium tier storage accounts:• LRS only• Performance increases with size of disk (flash storage)• Up to 7,500 IOPS per VHD*• Up to 250 MB/s throughput per disk• Pay based on size of VHD
* Remember max of 20,000 IOPS per storage account!
AKA Un-Managed DisksLegacy form of virtual machine virtual hard disk (VHD) storage - no longer
improved
30
Structured table data• Rows & columns
Typically used to store Azure resource performance metrics• Azure Monitor
Table StorageNo SQL
32
Used by developers – event-based computing• Webjobs• Functions• Logic Apps
Place something onto a queue
Some processor is triggered and picks up the first item in the queue
Queue StorageSequential processing
34
Create a share in an Azure storage account• No file server
Limitations:• Does not apply permissions• 5 TiB limit per share – use multiple shares for more capacity• 100 TiB limit per share in limited preview
Intended for use by applications• Not suitable for direct connections by end users (immature permission system)
Used by Azure File Sync
Azure File SharesCloud based shares
35
Azure File SyncDigitally transforming file servers
Branch Office
!
Branch Office
SMB
NFS
Users
Applications
PaaS
File Share
HQ Server
Azure Backup Vault
Work Folders
36
Requires a new kind of storage account:• Premium FileStorage
Higher performance for file storage:• Single digit latency• 5 GiB/second throughput per share• Up to 100,000 IOPS per share with bursting
Not for user shares!
Premium Azure FilesLimited Preview Today
38
Actively developed by Microsoft
Up to 32 TiB in size each• Aggregate disks (like in RAID) to accumulate capacity and IOPS
Offers lots of easy, GUI-based, management features• Some not available with un-managed disks• Others only available with un-managed disks via complex PowerShell
Managed DisksThe current method of deploying VM virtual hard disks (VHD)
39
Managed Disks TiersPerformance versus cost
Standard HDD Standard SSD Premium SSD Ultra SSD (Preview)
Type HDD Flash Flash Flash
Max IOPS Up to 2,000 IOPS Up to 6,000 IOPS Up to 20,000 IOPS Up to 160,000
Latency Moderate Very Low Very Low Sub millisecond
Throughput Up to 500 MB/s Up to 750 MB/s Up to 900 MB/s 2,000 MB/s
Resiliency LRS / ZRS LRS / ZRS LRS / ZRS LRS / ZRS
Note Entry level Smoother & lower
latency than HDD
Offers a VM SLA option Provision based
billing
40
Convert from older un-managed disks (1 way)
Higher availability (availability sets)
Permissions on disks
Create per-disk snapshots• Create new disks from snapshots
Create many VMs from 1 generalized managed disk image
Easy operations:• Convert disks between tiers• Create a new VM from an existing OS disk – add data disks after
Managed Disk FeaturesAll easy to do
41
Diagnostics & performance:• I recommend 1 GPv1 storage account per resource group• All resources place diagnostics (Block Blob) & performance (Table) data there• Especially virtual machines!
Still used for other features:• On-premises StorSimple cloud-tiered block-based storage (Block Blob)• Azure File Sync cloud-tiered file storage (Azure Files)• Big data (Block Blob)• Development (Table, Block Blob, and Queue)• And more!
Are Storage Accounts Useless?No!
43
Altaro VM Backup
Virtual backup trusted by 40,000+ SMBs, 9,000+ Partners and 1,200+ MSPs worldwide
Efficient backup setup – Easy & quick to setup and use
Full control of your backups – Powerful, flexible and easy to scale
Praise-winning Support
Latest developments• Backup directly to Azure Storage Accounts!
• Continuous Data Protection now supported with 5 Minute RTOs andbest storage savings in the industry!
• WAN Optimized Replication and DR
• Windows Server 2019 support
For more info & 30-day trial: altaro.com/vm-backup
44
Thank you for attending
Glad you could be part of the conversation!
www.altaro.com/hyper-vwww.altaro.com/vmware www.altaro.com/msp-dojo
Get more educational content from Altaro
youtube.com/c/AltaroSoftware
linkedin.com/company/altaro
@AltaroSoftware
facebook.com/AltaroSoftware @joe_elway
http://aidanfinn.com
