Download - Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg
![Page 1: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/1.jpg)
Windows AzureData Services
Scott Klueppel
Chief Cloud Evangelist
SOAlutions, Inc.#gwab
@kloopdogg
![Page 2: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/2.jpg)
Agenda - Overview - Management Portal / Tools - Windows Azure Storage - Tables, Blobs, Queues - Windows Azure SQL Database - Putting them together with Cloud Design Pattern Examples
![Page 3: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/3.jpg)
Windows Azure StorageOverviewManagement PortalStorage Emulator / Tools
![Page 4: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/4.jpg)
What is Windows Azure Storage? - Storage as a service – Blobs, Tables, and Queues - Lots of it (up to 200 TB) – pay as you go
- Highly Available - Redundancy
- Highly Scalable - Partitions
- Blazing Speed - Performance “Targets”
![Page 5: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/5.jpg)
Getting Started with Storage - Create a storage account in the portal - Connection string Account Name Account Key
![Page 6: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/6.jpg)
Getting Started with Storage - Use storage emulator for development - Connection string UseDevelopmentStorage=true
![Page 7: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/7.jpg)
Windows Azure SDK
Windows AzureSDK
Visual Studio2013
Visual Studio2012
Visual Studio2010
2.2 Oct2013 Supported Supported Not Supported
2.1 Jul2013 Not Supported Supported Supported
2.0 Apr2013 Not Supported Supported Supported
- SDK contains client libraries - Committed to backwards compatibility
![Page 8: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/8.jpg)
Storage Client Library - SCL 2.1 included with SDK 2.2 - SCL 3.0.x upgrade available with NuGet
![Page 9: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/9.jpg)
Tools to interact with storage Visual Studio Server Explorer CloudXplorer/TableXplorer
AzureXplorer (VS Extension)
![Page 10: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/10.jpg)
DemoManagement Portal
Storage
Tools
![Page 11: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/11.jpg)
Windows Azure StorageTablesBlobsQueues
![Page 12: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/12.jpg)
Tables Concept OverviewAccount Table Entity
gwabjax
racers
results
Name=RickyStatus=ActiveName=DaleStatus=ActiveRace=Jax 500Date=3/29/14
![Page 13: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/13.jpg)
- Structured storage in the form of tables-1MB max size-252 properties
- Quick queries by clustered index (only index)- 2,000 entities per second per partition- Data-defined partition scheme
-Table name + PK
Table Details
![Page 14: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/14.jpg)
- Tables store data more like key-value pairs - No schema
- De-normalized structure optimized for performance
Table Details
First Name Last Name Date of Birth Last Win
Ricky Bobby 1/1/1971
Dale Earnhardt 2/2/1972 Daytona 500
Cal Naughton March 3, 1973
![Page 15: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/15.jpg)
Table Entities - Property Types byte[]
boolDateTimedoubleGuidInt32Int64String
![Page 16: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/16.jpg)
Table Entities - Required properties PartitionKey, RowKey, Timestamp
- Operations on entities Delete Insert Upsert (Insert + Merge/Replace) Update (Merge or Replace) Retrieve Query
![Page 17: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/17.jpg)
Using Tables1. Get table reference
2. Execute operations on entities
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("DefaultConnection"));
CloudTableClient tableClient = storageAccount.CreateCloudTableClient();CloudTable table = tableClient.GetTableReference("racers");
TableOperation upsertOperation = TableOperation.InsertOrReplace(racer);table.Execute(upsertOperation);
![Page 18: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/18.jpg)
Using Tables with DTOs/POCOs1. Get table reference (like before)2. Execute operations on entities via
EntityAdapterWith TableEntity – RacerEntity
var operation = TableOperation.InsertOrReplace(racerEntity);table.Execute(operation);
With DTO – Racer
var adapter = new RacerAdapter(racer);var operation = TableOperation.InsertOrReplace(adapter);table.Execute(operation);
![Page 19: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/19.jpg)
Table Best Practices1. Avoid querying across partitions2. Batch inserts (within partition)
- Order and details in same table/partition
3. Storing aggregate copies - Ensures 1 query 1 partition - Format data up front
4. Use intelligent PKs
![Page 20: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/20.jpg)
DemoTables
Blobs
Queues
![Page 21: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/21.jpg)
Blobs Concept Overview
Account Container Blob
gwabjax
photos
videos
DSC000256.jpg
teamlogo.png
1stPlace.avi
http://<account>.blob.core.windows.net/<container>/<blobname>
![Page 22: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/22.jpg)
Blobs - Types of Blobs - Page - Block
- Operations on blobs Exists List Upload from Stream, ByteArray, Text, File Download to Stream, ByteArray, Text, File Delete
![Page 23: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/23.jpg)
Using Blobs1. Get blob container reference
2. Perform operations with blobs
CloudStorageAccount account = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("DefaultConnection"));
CloudBlobClient client = account.CreateCloudBlobClient();CloudBlobContainer container = client.GetContainerReference("photos");
CloudBlockBlob blockBlob = container.GetBlockBlobReference(refId);blockBlob.UploadFromStream(stream);blockBlob.Metadata["FileName"] = fileName;blockBlob.SetMetadata();
![Page 24: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/24.jpg)
DemoTables
Blobs
Queues
![Page 25: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/25.jpg)
Queues Concept OverviewAccount Queue Messages
gwabjax
orders
emails-to-send
Bulkhttp://...
Smallhttp://...
http://...
![Page 26: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/26.jpg)
Queues - Operations on queues Add Peek Get Update Delete
- Queue naming - All lowercase - Alpha-numeric and hyphen “-” - 3-63 characters
![Page 27: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/27.jpg)
Client Web Role
Storage
Queues in the wildWorker Role
Browser
Web App
Get Survey
CompleteSurvey
Post Results
SurveysQueue
Process
Task
Thank you
Add
Get
Delete Process*
![Page 28: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/28.jpg)
Client Web Role
Storage
Queues in the wildWorker Role
BlobStore
Browser
Web App
Get Survey
CompleteSurvey
Post Results
SmallSurveys
LargeSurveys
SmallTask
BigTask
Small Survey
Get
Delete Process*
AddThank you
![Page 29: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/29.jpg)
Client Web Role
Storage
Queues in the wildWorker Role
Blob/TableStore
Browser
Web App
Get Survey
CompleteSurvey
Post Results
SmallSurveysQueue
LargeSurveysQueue
SmallTask
LargeTask
LargeSurvey
Get
Delete Process*
AddThank you
Upload
Download
![Page 30: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/30.jpg)
Using Queues1. Get queue reference
2. Perform operations with messages
CloudStorageAccount account = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("DefaultConnection"));
CloudQueueClient client = account.CreateCloudQueueClient();CloudQueue queue = client.GetQueueReference("orders");
CloudQueueMessage message = new CloudQueueMessage("ticket-order-17"); queue.AddMessage(message);
![Page 31: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/31.jpg)
DemoTables
Blobs
Queues
![Page 32: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/32.jpg)
Windows Azure SQL Database(formerly SQL Azure)Management PortalData Access
![Page 33: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/33.jpg)
Windows Azure SQL DatabaseTools - SSMS- Visual Studio/SSDT- Modeling toolsCode- ADO.NET- Enterprise Library- ORMs (e.g. Entity Framework, nHibernate)
![Page 34: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/34.jpg)
Key Features- High Availability/Durability
- Three Replicas (1 Primary – 2 Secondary)- 99.9% SLA
- Manageability- Portal, PowerShell, SSMS, REST API
- Predictable Performance- Scale-out*
![Page 35: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/35.jpg)
What’s Different – Maybe Bad?Missing- Extended stored procedures- SQL-CLR- Service Broker- Table partitioning
Important: All tables must have a clustered index
![Page 36: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/36.jpg)
Data Types?Data Type Category Windows Azure SQL Database Support
Exact numerics bigint, bit, decimal, int, money, numeric, smallint, smallmoney, tinyint
Approximate numerics
float, real
Date and time date, datetime2, datetime, datetimeoffset, smalldatetime, time
Character strings char, varchar, text
Unicode character strings
nchar, nvarchar, ntext
Binary strings binary, varbinary, image
Spatial data types geography, geometry
Other data types cursor, hierarchyid, sql_variant, table, timestamp, uniqueidentifier, xml
![Page 37: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/37.jpg)
Development Story- Local SQL 2012/Express database - SSDT Database projects
- Deploy directly to Azure from Visual Studio- Create deployment package- Use Release Management
- Azure Connection String<add name="DefaultConnection"
connectionString="Server=tcp:[server].database.windows.net,1433;Database=[database]; User ID=[database]-host@[server]; Password=[password];Encrypt=true;"
providerName="System.Data.SqlClient"/>
![Page 38: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/38.jpg)
DemoManagement Portal
Create Server
Create Database
Firewall Rules
![Page 39: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/39.jpg)
Patterns at Work• Cache-aside Pattern• Materialized View Pattern• Competing Consumers Pattern• Compensating Transaction Pattern
![Page 40: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/40.jpg)
Cache-Aside Pattern- Build cache on-demand
- Read cache first- If not there
- Get from data store- Store in cache
![Page 41: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/41.jpg)
Materialized View Pattern- Improve performance in systems with
difficult queries- Pre-populated views - Not updated by app, can be entirely rebuilt- Inherent delays- Doesn’t query well
![Page 42: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/42.jpg)
Competing Consumers Pattern- Protect against a large influx (burst) of
requests- Balanced workload- Scalable
![Page 43: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/43.jpg)
Compensating Transaction Pattern- Eventual consistency
- Series of autonomous steps- Intermediate steps appear inconsistent
- Compensating Transaction- Intelligent process to undo succeeded steps
![Page 44: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/44.jpg)
Compensating Transaction Pattern- Example: Itinerary Creation
- Book 3 flights and 2 hotel rooms- If one or more are unavailable, we start
over
![Page 45: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/45.jpg)
Compensating Transaction Pattern- Example:
Itinerary Creation
![Page 46: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/46.jpg)
What did we learn?- Azure data storage options/uses- Each has pros/cons/best fit- Management Portal and Tools- Design patterns
![Page 47: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/47.jpg)
Questions?
Scott Klueppel
Chief Cloud Evangelist
SOAlutions, Inc.#gwab
@kloopdogg
![Page 48: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/48.jpg)
Thank you!
Scott Klueppel
Chief Cloud Evangelist
SOAlutions, Inc.#gwab
@kloopdogg
![Page 49: Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1b5503460f949f140b/html5/thumbnails/49.jpg)
References• Cloud Design Patterns (P&P)
http://msdn.microsoft.com/en-us/library/dn568099.aspx
• Data Services – Storagehttp://msdn.microsoft.com/en-us/library/windowsazure/gg433040.aspx
• Storage Differences – Emulator vs Cloudhttp://msdn.microsoft.com/en-us/gg433135
• Data Services – SQL Databasehttp://msdn.microsoft.com/en-us/library/windowsazure/ee336279.aspx