extending application data in the cloud

http://ronaldbradford.com(c) Copyright 2008

Extending application data with the cloud

Version 1.0 15.Jul.2008

Extending your

applications data

with the cloud

Ronald Bradford

A panel on Cloud Computing

July 2008



Session Focus

❖Existing developed application ❖Data in relational databases (e.g. MySQL)❖Traditional problems❖ Data growth❖ Performance ❖Data Size ❖Application Response

❖ Backup & Recovery

http://ronaldbradford.com




Agenda

❖Existing examples❖Cloud problems❖How to approach❖Data store options





Example 1 - Processing

❖Extend existing processing power❖Near instant access for demand❖Using AWS - EC2❖Animoto - images+music to audio slide shows❖Batch processing, not real time (*)

www.animoto.com



http://www.animoto.com

http://www.animoto.com



Example 2 - Storage

❖Extend existing storage capacity❖Near instant access for demand❖Using AWS - S3❖SmugMug - Online Photo publisher❖Short term or long term

www.smugmug.com





Problems?

❖Control❖Availability❖ Latency❖Data Consistency❖Data Migration❖Data Synchronization❖Data Security





Problems - Control

❖ Loss of physical location❖ Loss of physical backup❖ Loss of tuning/optimization capabilities❖ Tools DBAʼs, SAʼs use for monitor?

❖ Loss of ability to fix unavailability❖What is the SLA?❖Who does the CxO rant to?





Problems - Availability

❖What do you do when services unavailable❖ e.g. S3 down 7 hrs❖ GAE unavailable for long time

Donʼt put all eggs in one basket





Problems - Latency

❖Keep your critical data close❖Present a layered approach of data retrieval❖Parallelism for cloud sources❖Move polling of data to push on demand





Problems - Data Consistency

❖No way to ensure data consistent across multiple sources





Problems - Data Synchronization

Using BigTable/SimpleDB

❖Different storage technologies❖Different interface languages❖ Lack of bulk load tools





Problems - Data Security

Using BigTable/SimpleDB

❖No access control for data❖ Administrator❖ Application Writer❖ Application Reader





How to approach

❖Model/View/Controller (MVC)❖API to all application data❖Data change timeline❖Think differently❖Eating an elephant





Approach - MVC

❖May be an old concept❖Separates presentation, logic and data❖Enables data changes without logic or

application changes





Approach - API

❖Standard Interface to your data❖Enables transparent storage/retrieval❖ Cache Layer❖ Read Scalability Layer❖ Alternative Data Stores





Approach - Data Timeline

❖Choose the right data❖ Write once data❖ Colder data - accessed less frequently❖ Partitioned data





Approach - Thinking

❖ 3rd normal form doesnʼt apply❖Key access❖No joins❖ Little/no integrity





Approach

How do you eat an elephant?





Approach

How do you eat an elephant?

One bite at a time





Data Store Options

❖Existing RDBMS❖BigTable (via GAE)❖SimpleDB❖CouchDB❖Others





Data Store Options - Existing

❖EC2/S3 powerful offering❖Can deploy same stack❖Can use same tools❖Hard to beat for ease of use❖Greater track record





Data Store Options - Big Table

❖Available via Google App Engine (GAE)❖Python Interface only❖Unreliable availability❖ Lack of support❖Reasonable data semantics❖ Data types (including lists)❖ Relationships





Data Store Options - SimpleDB

❖Available via AWS❖Greater interfaces ( ! python)❖Very simple, poor data semantics❖ No data types❖ No relationships

❖Support level unknown?





Data Store Options - Others

❖CouchDB - distributed, fault tolerant❖Drizzle - A lightweight Cloud/Web DB❖NimbusDB - Jim Starkey





Professional Advice and Assistance

❖ 2 decades expertise & experience ❖Enterprise RDBMS Data Architecture❖Performance & Scalability❖Data Analysis & Migration❖Alternatives to traditional data modeling

❖ 10 years in Oracle (includes at Oracle Corporation)❖ 9 years in MySQL (includes at MySQL Inc)

ronaldbradford.com



extending application data in the cloud

Technology

cloud computing

relational databases

cloudronald bradford

yourapplications datawith

c version