how to use the cloud for your gis - nebraska gis/lis · • amazon web services account •...
TRANSCRIPT
Mike SchonlauDouglas County
How to use the Cloudfor your GIS
Nebraska GIS Symposium 2013
Sunday, May 5, 13
• Moved 2-1/2 years ago• On-premise geodatabases• Cloud-hosted applications and geodatabases• 13 production web apps running on EC2 servers• Replicated Enterprise Geodatabase (SDE & SQL)
on EC2 server• Various web services and GP tools • 100+ REST services (including cached)• Always testing new versions of software• Hoping to make better use of scripts
Douglas County GIS
Sunday, May 5, 13
• Understanding terms
• on-premise (my hardware)
• cloud (Amazon’s hardware)
• hosted (ArcGIS Online)
• What is my IT infrastructure situation?
• Do I have viable internal options?
Considerations
Sunday, May 5, 13
• Do you have the authority and support to move to the cloud?
• Goals for your server environment?
• Make smart, strategic investments
Considerations
Sunday, May 5, 13
• The “cloud” is not a mystery
• It’s hard to plan long-term for GIS - stay flexible
• It’s better to be nimble
• Think about audience, interfaces, costs, performance, software, security, scalability, and adaptability
Observations
Sunday, May 5, 13
Getting Started
Sunday, May 5, 13
• Amazon Web Services account• aws.amazon.com• credit card required (PO’s possible)
• Remote admin tools• Windows - Remote Desktop• Mac - CoRD (open source)
• ESRI license(s)• EDN• ArcGIS Server Standard or Advanced• SDE (ArcGIS Server Basic)
• Access to ESRI AMI’s (contact customer service)
Prerequisites
Sunday, May 5, 13
• Cloud• Hosted infrastructure, infrastructure-as-a-service,
platform-as-a-service, etc• Simple - servers somewhere else
• AWS• Amazon Web Services
• EC2• Amazon elastic compute cloud (virtual servers)
• S3• Amazon storage platform
Definitions
Sunday, May 5, 13
• AMI• Amazon Machine Image (pre-configured server or
backup of your configured server)
• Elastic IP• IP addresses you can associate/disassociate to EC2
instances• We point our domains/subdomains to these IP’s
• EBS volumes• Scalable storage that can be added to EC2
instances like attached hard disks• Can be detached and then attached to other
instances• Snapshots provide data backup
Definitions
Sunday, May 5, 13
• Monthly billing to credit card with detailed usage statement
• Mostly a la carte pricing
• Pricing varies by region
AWS Costs
Sunday, May 5, 13
• http://aws.amazon.com/ec2/pricing/• Instance types
• On Demand - flat hourly rate• Reserved - upfront fee, lower hourly rate• Spot - bid on unused EC2 space, variable
• Data transfer - monthly i/o• EBS volumes - per Gb, per month• Elastic IP’s• Monitoring (CloudWatch)• Elastic load balancers
EC2 Costs
Sunday, May 5, 13
• http://aws.amazon.com/s3/pricing/• Based on quantity• Reduced redundancy available at lower cost
• 99.99% durability• Standard storage is 99.999999999%
• Separate data transfer pricing
S3 Costs
Sunday, May 5, 13
• Part of your long-term strategy• ArcGIS Server (how many cores?)• ArcGIS Desktop• EDN
Software Costs
Sunday, May 5, 13
Data
Sunday, May 5, 13
• AWS Import/Export• Send them a hard drive
• Direct upload to S3 “bucket”• 3rd party S3 tools
• Cloudberry Explorer• Bucket Explorer
• Copy/paste• Most of our data transferred this way
• Dropbox
Getting It Up There
Sunday, May 5, 13
• S3• Fast• Very scalable• Inexpensive• Could be a great place to store static data (i.e.
imagery, cache tiles, graphics, etc)• EBS
• Familiar (like an attached drive)• Flexible (move between servers)• Scalable• More work to attach
S3 or EBS
Sunday, May 5, 13
• ESRI AMI• SDE + PostrgeSQL/SQL Server
• BYOGDB• We use SDE + SQL• Created our own AMI
• ODBC• can work from your network to the cloud, but it is
slow...
Geodatabase
Sunday, May 5, 13
Geodatabase
• Connectivity between EC2 servers is very good
Sunday, May 5, 13
• Geodata services
• Setup data stores in ArcGIS Server to sync on-premis and cloud geodatabases
• Publish maps from Desktop and data will sync if Cloud GDB matches
• Test this!!!!
• ESRI how-to video
Replication
Sunday, May 5, 13
Apps
Sunday, May 5, 13
• Using IIS with SSL• Mix of Javascript, Flex, and Silverlight apps
• IDE’s on EC2 instance• Aptana (Javascript)• Flash Builder (Flex)• Visual Studio (Silverlight)• Code copied to local network regularly
Web Server
Sunday, May 5, 13
• 3rd party apps• Cityworks - production• Pictometry Online - production
• Test instances• For new releases and service packs• One of the most valuable aspects
• Performance• Better than we were getting with local servers
Web Server
Sunday, May 5, 13
• Reliability• 2 hours of downtime in 2.5 years• Apps are near 100% uptime• SLA’s - http://aws.amazon.com/ec2-sla/
• Redundancy• 15-minute failover• All critical data backed up by AWS• AMI’s to preserve configurations
• Security• World class• http://aws.amazon.com/security/
Web Server
Sunday, May 5, 13
• DOGIS
• Omaha Lead Database
• Cityworks
Examples
Sunday, May 5, 13
• Use the same zone for servers talking to each other
• Change the system time
• Set the machine name to static
• Change the Windows admin password
• Plan ahead for data replication
• Use scripts to stop/start instances and save $$
• Carefully use C: drive - it is small
• Read and understand SLA’s, security, and backup policies
• Remember to delete EBS volumes and elastic IP’s after terminating server instances
Tips & Tricks
Sunday, May 5, 13
• Dropbox (www.dropbox.com) - easy file sharing
• CoRD (http://cord.sourceforge.net) - open source remote desktop for Mac
• WhatIsMyIP (www.whatismyip.com) - add security rules for remote connection
• Cloudberry Explorer (www.cloudberrylab.com) - for moving data back and forth to S3
Tools
Sunday, May 5, 13
Our Costs - January 2012
Sunday, May 5, 13
• Less control over physical environment
• IT resistance
• Costs - if you don’t manage them
• Sensitivity to housing secure data in cloud
• Fear of the unknown
Disadvantages
Sunday, May 5, 13
• Spending more time on GIS vs server admin
• Flexibility• Easy test instances
• Administer from anywhere
• Elastic IP’s and scalable storage
• No VPN - just remote desktop tools
• Reliability
• Security
• Redundancy
• Performance
• Cost - when you consider all costs
Advantages
Sunday, May 5, 13