hybrid architectures for database
TRANSCRIPT
![Page 1: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/1.jpg)
![Page 2: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/2.jpg)
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid architectures for database backups & file migrations
S T G 3 1 3
Jeff Bartley
Solutions Architect, Hybrid Cloud Storage and Transfer
Amazon Web Services
Peter Levett
Storage Specialist Solutions Architect
Amazon Web Services
![Page 3: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/3.jpg)
Agenda
AWS Storage Gateway overview
Scenario 1: SQL Server backup to AWS
Scenario 2: NFS migration with AWS DataSync and AWS Storage Gateway
Start working
![Page 4: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/4.jpg)
Related sessions
STG201 - Leadership session: Storage state of the union
STG225 - Getting started with hybrid file storage using File Gateway
STG226 - Hands-on with hybrid block storage using a volume gateway
STG217 - Shift your tape backups to AWS to save time and money
STG305 - Build hybrid storage architectures with AWS Storage Gateway
STG336 - Using hybrid cloud storage to close a data center and migrate
STG208 - Backup-and-restore and disaster-recovery solutions with AWS
STG204 - Get data to AWS: How to choose and use data migration services
STG224 - Getting started with AWS DataSync
STG220 - How to move 700 TB over the wire to AWS with AWS DataSync
STG316 - Get hands-on & learn best practices for AWS data migrations
![Page 5: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/5.jpg)
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
![Page 6: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/6.jpg)
AWS Storage Gateway
Provides on-premises access to virtually unlimited cloud storage . . .
. . . regardless of cloud adoption stage
Move on-premises backups
to the cloud
Provide low-latency access
for on-premises applications
to cloud data
Shift on-premises storage to
cloud-backed file shares
![Page 7: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/7.jpg)
AWS CloudCustomer premise
Files
(NFS/SMB)
Volumes
(iSCSI)
Tapes
(iSCSI VTL)
AWS Storage Gateway
Integrated with AWS Identity and Access Management
(IAM), AWS Key Management Service (AWS KMS),
AWS CloudTrail, Amazon CloudWatch services
Amazon S3
Glacier
Amazon Simple
Storage Service
(Amazon S3)
Amazon Elastic
Block Store
(Amazon EBS)
AWS Storage Gateway
Configuration: VMware ESXi, Microsoft Hyper-V,
Amazon Elastic Compute Cloud (Amazon EC2),
hardware appliance
AWS Backup
Amazon S3
Glacier Deep
Archive
Storage Gateway serviceStorage Gateway
HTTPS
![Page 8: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/8.jpg)
Move on-premises backups to the cloud
iSCSI VTL
AWS Cloud
File
gateway
Volume
gateway
Tape
gateway
AWS
Storage
Gateway
managed
service
Database/
application
server
Application
server
Backup
server
iSCSI
NFS/SMB
Tape library
(Amazon S3)Tape archive
(S3 Glacier/GDA)
Amazon S3 Amazon EBSAWS Backup
HTTPS
HTTPS
HTTPS
On premises
Any S3 storage class
Lifecycle
Amazon S3
Eject
Maintain your backup workflows while reducing your backup infrastructure on-premises
![Page 9: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/9.jpg)
File Gateway for on-premises backupMove database and file backups into the cloud and free up on-premises storage capacity
Features
NFS/SMB protocol support; mount shares directly
on database and application servers
Files stored durably in Amazon S3, lifecycle to
any S3 storage class
Local cache for accessing recent backups
Windows ACL support to control access to
backup files
Support for S3 object lock
Bandwidth optimized; only changes are
transferred
Reduce on-premises storage for backups
Easily integrates with SAP, SQL Server,
Oracle, HDFS, and other applications
Restore backups on-premises or in the
cloud on Amazon EC2 or Amazon RDS
Benefits
AWS Cloud
HTTPSFile
gateway
NFS/SMBDatabase/
application
server
Any S3 storage class
On premises
Amazon S3
Lifecycle
![Page 10: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/10.jpg)
Shift on-premises storage to cloud-backed file sharesAccess virtually unlimited, highly durable cloud storage using common file protocols
Features Benefits
Supports NFS and SMB protocols; no application
changes required
Files stored durably in Amazon S3, lifecycle to
any S3 storage class
SMB shares integrate with Microsoft Active
Directory
Amazon CloudWatch events for automated
workflows
Reduce costs by moving storage to Amazon
S3 while still accessing from on premises
Virtually unlimited cloud storage; no more
running out of capacity
Eliminate expensive hardware refresh cycles
Files stored as native S3 objects for further
processing in AWS
AWS Cloud
HTTPSFile
gateway
NFS/SMBApplication
Any S3 storage class
On premises
Amazon S3
Lifecycle
NAS storage
![Page 11: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/11.jpg)
Low-latency access for on-premises applications to cloud data Access files quickly from distributed locations and scale capacity as needed
Features Benefits
Generate data in the cloud or ingest from on
premises using AWS DataSync or AWS Snowball
Up to 16 TB local cache per gateway
Fully managed gateway cache provides low-
latency access to data
Refresh cache at the bucket or prefix level
Access cloud storage from any on-premises
location
Process data in the cloud and refresh
gateway cache for up-to-date results
Data stored cost-effectively and centrally in
the cloud
AWS Cloud
Application
NFS/SMB
Cache refresh
HTTPS
Cache refresh
HTTPS
Application
NFS/SMB
On-premises
File gateway
On-premises
File gatewayIn-cloud processing
AWS
DataSync
AWS
Snowball
![Page 12: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/12.jpg)
2019 feature releases
YTD 2019 Just launched
Protocols &
services
• Tape gateway supports archiving existing and
new tapes to Amazon S3 Glacier deep archive
• AWS PrivateLink support for all gateway types
• FGW adds SMB ACLs and additional encryption
options for enhanced security
• S3 object lock support to enable WORM
workloads
Management
& automation
features
• Hardware appliance available in the EU
• Selective cache refresh to enable content
distribution
• TGW support for Amazon S3 Glacier deep archive
• Tag-on-create, tag-based access control
• High availability for VMware deployments across
all gateway types
• Improved Amazon CloudWatch logging and
metrics for all gateway types
• Gateway software update scheduling for all
gateway types
Limits• 5× read perf. increase for file gateway (to 4.8
Gbps)
• 3× perf. increase for tape gateway (to 2.7 Gbps)
Regions • ARN, BAH, HKG, GovCloud (PDT), GRU (TGW)
![Page 13: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/13.jpg)
Learn more
aws.amazon.com/storagegateway/
• Whitepapers
• Webinars
• Demos & videos
• Developer resources
• Recorded AWS re:Invent
Sessions
• Product documentation
• Pricing
• FAQs
• Customer case studies
![Page 14: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/14.jpg)
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
![Page 15: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/15.jpg)
Workshop scenario 1: MS SQL backup to Amazon S3
AWS Cloud
Region (eu-west-1)
Availability Zone
SQL instance
Domain controller
instance
AWS Storage Gateway
instance
CacheShare
(smb)
AWS Cloud
Region (eu-central-1)
Amazon CloudWatchAWS Storage
Gateway
Amazon S3 Glacier
Amazon S3: Infrequent access
Backup
bucket
Expire/delete backup after 7 years
Lifecycle after 30 days
Amazon SQS
Bastion Amazon RDS
instance
![Page 16: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/16.jpg)
Workshop scenario 1: MS SQL backup to Amazon S3
AWS Cloud
Region (eu-west-1)
Availability Zone
SQL instance
Domain controller
instance
AWS Storage Gateway
instance
AWS Cloud
Region (eu-central-1)
1. Deploy VPC, domain controller,
SQL Server, and Storage
Gateway (simulate on-premises
in eu-west-1)
2. Active gateway in eu-central-1
AWS Storage
Gateway
![Page 17: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/17.jpg)
Workshop scenario 1: MS SQL backup to Amazon S3
AWS Cloud
Region (eu-west-1)
Availability Zone
SQL instance
Domain controller
instance
AWS Storage Gateway
instance
Share
(smb)
AWS Cloud
Region (eu-central-1)
Backup
bucket
1. Create S3 bucket
2. Attach gateway to AD
domain
3. Create SMB share attached
to new bucket
4. Test access to bucket from
SQL Server
![Page 18: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/18.jpg)
Workshop scenario 1: MS SQL backup to Amazon S3
AWS Cloud
Region (eu-west-1)
Availability Zone
SQL instance
Domain controller
instance
AWS Storage Gateway
instance
CacheShare
(smb)
AWS Cloud
Region (eu-central-1)
Amazon CloudWatchAWS Storage
Gateway
Backup
bucket
Amazon SQS
1. Create SQS queue
2. Create Amazon CloudWatch rule that triggers Amazon SQS
![Page 19: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/19.jpg)
Workshop scenario 1: MS SQL backup to Amazon S3
AWS Cloud
Region (eu-west-1)
Availability Zone
SQL instance
Domain controller
instance
AWS Storage Gateway
instance
CacheShare
(smb)
AWS Cloud
Region (eu-central-1)
Amazon CloudWatchAWS Storage
Gateway
Backup
bucket
Amazon SQS
1. Run backup process
![Page 20: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/20.jpg)
Workshop scenario 1: MS SQL backup to Amazon S3
AWS Cloud
Region (eu-west-1)
Availability Zone
SQL instance
Domain controller
instance
AWS Storage Gateway
instance
CacheShare
(smb)
AWS Cloud
Region (eu-central-1)
Backup
bucket
Domain controller
instance
Amazon RDS
instance
1. Provision Bastion host & RDS instance
2. On-premises backup to RDS
![Page 21: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/21.jpg)
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
![Page 22: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/22.jpg)
Migrate on-premises NFS server data to Amazon S3, validate
access to Amazon S3 from File Gateway, and shut down the on-
premises NFS server
Workshop goal
Workshop scenario 2
Application
server
NFS
server
NFS
In-cloud Region (eu-central-1)On-premises Region (eu-west-1)
DataSync
agent
File gateway
NFS
S3 bucket
DataSync
managed service
HTTPS
TLSNFS
![Page 23: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/23.jpg)
• Resources deployed using CloudFormation templates
• Connect to the application server, mount NFS, and verify files
Deploy resources in 2 regions and verify data on the NFS server
Workshop scenario 2: Module 1
Application
server
NFS
server
NFS
In-cloud Region (eu-central-1)On-premises Region (eu-west-1)
DataSync
agent
File gateway
S3 bucket
![Page 24: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/24.jpg)
Activate the DataSync agent, create source and destination locations, trigger the task to copy files to S3
Copy file data to Amazon S3 using AWS DataSync
Workshop scenario 2: Module 2
Application
server
NFS
server
NFS
In-cloud Region (eu-central-1)On-premises Region (eu-west-1)
DataSync
agent
File gateway
S3 bucket
DataSync
managed service
TLSNFS
Source
location
Destination
location
![Page 25: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/25.jpg)
Activate the file gateway, create an NFS share, mount the share
on the application server
Access the S3 bucket on premises using File Gateway
Workshop scenario 2: Module 3
Application
server
NFS
server
NFS
In-cloud Region (eu-central-1)On-premises Region (eu-west-1)
DataSync
agent
File gateway
NFS
S3 bucket
DataSync
managed service
HTTPS
TLSNFS
![Page 26: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/26.jpg)
Rerun the DataSync task to copy remaining files from the NFS
server to the S3 bucket
Perform an incremental transfer to get remaining data
Workshop scenario 2: Module 4
Application
server
NFS
server
NFS
In-cloud Region (eu-central-1)On-premises Region (eu-west-1)
DataSync
agent
File gateway
NFS
S3 bucket
DataSync
managed service
HTTPS
TLSNFS
![Page 27: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/27.jpg)
Unmount the NFS server on the application server, all traffic
now goes to File Gateway
Cutover to File Gateway and shut down the NFS server
Workshop scenario 2: Module 5
Application
server
NFS
server
In-cloud Region (eu-central-1)On-premises Region (eu-west-1)
DataSync
agent
File gateway
NFS
S3 bucketHTTPS
![Page 28: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/28.jpg)
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
![Page 29: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/29.jpg)
Go to https://dashboard.eventengine.run
![Page 30: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/30.jpg)
Click on the AWS Consolebutton
![Page 31: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/31.jpg)
Launch the AWS Console
(opens a new tab)
![Page 32: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/32.jpg)
Click on the Readme for next steps
![Page 33: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/33.jpg)
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Visit aws.amazon.com/training/path-storage/
Classroom offerings, like Architecting on AWS, feature AWS expert instructors and hands-on activities
45+ free digital courses cover topics related to cloud storage, including:
Learn storage with AWS Training and Certification
• Amazon S3
• AWS Storage Gateway
• Amazon S3 Glacier
• Amazon Elastic File System
(Amazon EFS)
• Amazon Elastic Block Store
(Amazon EBS)
Resources created by the experts at AWS to help you build cloud storage skills
![Page 34: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/34.jpg)
Thank you!
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
![Page 35: Hybrid architectures for database](https://reader030.vdocuments.us/reader030/viewer/2022032418/62338746b60146280c58b965/html5/thumbnails/35.jpg)
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.