how to use the cloud for your gis - nebraska gis/lis · • amazon web services account •...

Mike SchonlauDouglas County

How to use the Cloudfor your GIS

Nebraska GIS Symposium 2013

Sunday, May 5, 13

• Moved 2-1/2 years ago• On-premise geodatabases• Cloud-hosted applications and geodatabases• 13 production web apps running on EC2 servers• Replicated Enterprise Geodatabase (SDE & SQL)

on EC2 server• Various web services and GP tools • 100+ REST services (including cached)• Always testing new versions of software• Hoping to make better use of scripts

Douglas County GIS

Sunday, May 5, 13

• Understanding terms

• on-premise (my hardware)

• cloud (Amazon’s hardware)

• hosted (ArcGIS Online)

• What is my IT infrastructure situation?

• Do I have viable internal options?

Considerations

Sunday, May 5, 13

• Do you have the authority and support to move to the cloud?

• Goals for your server environment?

• Make smart, strategic investments

Considerations

Sunday, May 5, 13

• The “cloud” is not a mystery

• It’s hard to plan long-term for GIS - stay flexible

• It’s better to be nimble

• Think about audience, interfaces, costs, performance, software, security, scalability, and adaptability

Observations

Sunday, May 5, 13

Getting Started

Sunday, May 5, 13

• Amazon Web Services account• aws.amazon.com• credit card required (PO’s possible)

• Remote admin tools• Windows - Remote Desktop• Mac - CoRD (open source)

• ESRI license(s)• EDN• ArcGIS Server Standard or Advanced• SDE (ArcGIS Server Basic)

• Access to ESRI AMI’s (contact customer service)

Prerequisites

Sunday, May 5, 13

• Cloud• Hosted infrastructure, infrastructure-as-a-service,

platform-as-a-service, etc• Simple - servers somewhere else

• AWS• Amazon Web Services

• EC2• Amazon elastic compute cloud (virtual servers)

• S3• Amazon storage platform

Definitions

Sunday, May 5, 13

• AMI• Amazon Machine Image (pre-configured server or

backup of your configured server)

• Elastic IP• IP addresses you can associate/disassociate to EC2

instances• We point our domains/subdomains to these IP’s

• EBS volumes• Scalable storage that can be added to EC2

instances like attached hard disks• Can be detached and then attached to other

instances• Snapshots provide data backup

Definitions

Sunday, May 5, 13

• Monthly billing to credit card with detailed usage statement

• Mostly a la carte pricing

• Pricing varies by region

AWS Costs

Sunday, May 5, 13

• http://aws.amazon.com/ec2/pricing/• Instance types

• On Demand - flat hourly rate• Reserved - upfront fee, lower hourly rate• Spot - bid on unused EC2 space, variable

• Data transfer - monthly i/o• EBS volumes - per Gb, per month• Elastic IP’s• Monitoring (CloudWatch)• Elastic load balancers

EC2 Costs

Sunday, May 5, 13

• http://aws.amazon.com/s3/pricing/• Based on quantity• Reduced redundancy available at lower cost

• 99.99% durability• Standard storage is 99.999999999%

• Separate data transfer pricing

S3 Costs

Sunday, May 5, 13

• Part of your long-term strategy• ArcGIS Server (how many cores?)• ArcGIS Desktop• EDN

Software Costs

Sunday, May 5, 13

Data

Sunday, May 5, 13

• AWS Import/Export• Send them a hard drive

• Direct upload to S3 “bucket”• 3rd party S3 tools

• Cloudberry Explorer• Bucket Explorer

• Copy/paste• Most of our data transferred this way

• Dropbox

Getting It Up There

Sunday, May 5, 13

• S3• Fast• Very scalable• Inexpensive• Could be a great place to store static data (i.e.

imagery, cache tiles, graphics, etc)• EBS

• Familiar (like an attached drive)• Flexible (move between servers)• Scalable• More work to attach

S3 or EBS

Sunday, May 5, 13

• ESRI AMI• SDE + PostrgeSQL/SQL Server

• BYOGDB• We use SDE + SQL• Created our own AMI

• ODBC• can work from your network to the cloud, but it is

slow...

Geodatabase

Sunday, May 5, 13

Geodatabase

• Connectivity between EC2 servers is very good

Sunday, May 5, 13

• Geodata services

• Setup data stores in ArcGIS Server to sync on-premis and cloud geodatabases

• Publish maps from Desktop and data will sync if Cloud GDB matches

• Test this!!!!

• ESRI how-to video

Replication

Sunday, May 5, 13

Apps

Sunday, May 5, 13

• Using IIS with SSL• Mix of Javascript, Flex, and Silverlight apps

• IDE’s on EC2 instance• Aptana (Javascript)• Flash Builder (Flex)• Visual Studio (Silverlight)• Code copied to local network regularly

Web Server

Sunday, May 5, 13

• 3rd party apps• Cityworks - production• Pictometry Online - production

• Test instances• For new releases and service packs• One of the most valuable aspects

• Performance• Better than we were getting with local servers

Web Server

Sunday, May 5, 13

• Reliability• 2 hours of downtime in 2.5 years• Apps are near 100% uptime• SLA’s - http://aws.amazon.com/ec2-sla/

• Redundancy• 15-minute failover• All critical data backed up by AWS• AMI’s to preserve configurations

• Security• World class• http://aws.amazon.com/security/

Web Server

Sunday, May 5, 13

• DOGIS

• Omaha Lead Database

• Cityworks

Examples

Sunday, May 5, 13

• Use the same zone for servers talking to each other

• Change the system time

• Set the machine name to static

• Change the Windows admin password

• Plan ahead for data replication

• Use scripts to stop/start instances and save $$

• Carefully use C: drive - it is small

• Read and understand SLA’s, security, and backup policies

• Remember to delete EBS volumes and elastic IP’s after terminating server instances

Tips & Tricks

Sunday, May 5, 13

• Dropbox (www.dropbox.com) - easy file sharing

• CoRD (http://cord.sourceforge.net) - open source remote desktop for Mac

• WhatIsMyIP (www.whatismyip.com) - add security rules for remote connection

• Cloudberry Explorer (www.cloudberrylab.com) - for moving data back and forth to S3

Tools

Sunday, May 5, 13

Our Costs - January 2012

Sunday, May 5, 13

• Less control over physical environment

• IT resistance

• Costs - if you don’t manage them

• Sensitivity to housing secure data in cloud

• Fear of the unknown

Disadvantages

Sunday, May 5, 13

• Spending more time on GIS vs server admin

• Flexibility• Easy test instances

• Administer from anywhere

• Elastic IP’s and scalable storage

• No VPN - just remote desktop tools

• Reliability

• Security

• Redundancy

• Performance

• Cost - when you consider all costs

Advantages

Sunday, May 5, 13

Mike Schonlaumschonlau@douglascounty-ne.gov

Twitter: @MikeSchonlau

Thank You

Sunday, May 5, 13