Introduction to AWShttp://bit.ly/scico-lab

Julian Lau – Solutions ArchitectureAdrian White – Research & Technical Computing

2017

Overview of the day

• 9:00 AWS - Your first steps (1.5 hours)• 10:30 Morning break (30 minutes)• 11:00 Moving data into AWS (1 hour)• 12:00 Data Analytics & Machine Learning (1 hour)• 13:00 Lunch (1 hour)• 14:00 Lab: Jupyter Notebooks on AWS (2 hours)• 16:00 Wrap up and next steps

Agenda

• Overview of AWS services• Essential services• Accessing your resources• Commercial models• Monitoring• Securing your resources• Research on AWS• Educate

AWSGlobalInfrastructure

Availability Zone (AZ)

• Each region has multiple, isolated locations known as Availability Zones

• Low-latency links between AZs in a region

• When launching an EC2 instance, a customer chooses an AZ

AVAILABILITYZONE3

EC2

AVAILABILITYZONE2

AVAILABILITYZONE1

EC2EC2

EC2

REGION

AWSprovidesbroadanddeepservicestosupportanycloudworkload

AWS Global Infrastructure

Application Services

Networking

Deployment & Administration

DatabaseStorageCompute

ENTERPRISEAPPS

DEVELOPMENT &OPERATIONSMOBILE SERVICESAPPSERVICESANALYTICS

DataWarehousing

Hadoop/Spark

StreamingDataCollection

MachineLearning

ElasticSearch

VirtualDesktops

Sharing&Collaboration

CorporateEmail

Backup

Queuing&Notifications

Workflow

Search

Email

Transcoding

One-clickAppDeployment

Identity

Sync

SingleIntegratedConsole

PushNotifications

DevOpsResourceManagement

ApplicationLifecycleManagement

Containers

Triggers

ResourceTemplates

TECHNICAL&BUSINESSSUPPORT

AccountManagement

Support

ProfessionalServices

Training&Certification

Security&PricingReports

PartnerEcosystem

SolutionsArchitects

MARKETPLACE

BusinessApps

BusinessIntelligence DatabasesDevOps

Tools NetworkingSecurity Storage

Regions AvailabilityZones

PointsofPresence

INFRASTRUCTURE

CORE SERVICES

ComputeVMs,Auto-scaling,&LoadBalancing

StorageObject,Blocks,Archival,Import/Export

DatabasesRelational,NoSQL,Caching,Migration

NetworkingVPC,DX,DNSCDN

AccessControl

IdentityManagement

KeyManagement&Storage

Monitoring&Logs

Assessmentandreporting

Resource&UsageAuditing

SECURITY&COMPLIANCE

ConfigurationCompliance

Webapplicationfirewall

HYBRIDARCHITECTURE

DataBackups

IntegratedAppDeployments

DirectConnect

IdentityFederation

IntegratedResourceManagement

IntegratedNetworking

APIGateway

IoT

RulesEngine

DeviceShadows

DeviceSDKs

Registry

DeviceGateway

StreamingDataAnalysis

BusinessIntelligence

MobileAnalytics

Essential Services

Compute Services

Amazon EC2 Auto Scaling Elastic Load Balancing

Actual

EC2

Elastic Virtual servers in the cloud

Dynamic traffic distribution

Automated scaling of EC2 capacity

Networking Services

Amazon VPC: AWS DirectConnect Amazon Route 53

Availability Zone B

Availability Zone A

Private, isolatedsection of the AWS

Cloud

Private connectivity between AWS and your

datacenter

Domain Name System (DNS) web service.

Storage Services

Amazon EBS

EBS

Block storage for use with Amazon EC2

Amazon S3

ImagesVideosFilesBinariesSnapshots

Internet scale storage via API

AWS Storage Gateway

S3, Glacier

Integrates on-premises IT and AWS storage

Amazon Glacier

ImagesVideosFilesBinariesSnapshots

Storage for archivingand backup

Database Services

Amazon RDS Amazon DynamoDB

Managed relational database service

Managed NoSQL database service

DBA

Amazon ElastiCache

In-Memory Caching Service

Big Data Services

Amazon EMR (Elastic Map Reduce)

AWS Data Pipeline

Hosted Hadoop framework

Move data among AWS services and on-

premises data sources

Amazon Redshift

Petabyte-scale data warehouse service

Deployment & Administration

Amazon CloudWatch

AWS IAM (Identity & Access Mgmt)

AWS OpsWorks

AWS CloudFormation

AWS Elastic Beanstalk

Web App

Enterprise App

Database

Monitor resources Manage users, groups &

permissions

Dev-Ops framework for application

lifecycle management

Templates to deploy & manage

Automate resource management

A little more on CloudFormation

CloudFormation … Rocks !

Templated resourceprovisioning

Infrastructure as code

Declarative and flexible

Easy to use

Common Use Cases

Stack replication Infrastructure scale out

Blue-green deployments

Infrastructure as code

TemplateStructureLevelTemplateStructure

{"Description" : "A text description for the template usage","Parameters": {

// A set of inputs used to customize the template per deployment},"Resources" : {

// The set of AWS resources and relationships between them},"Outputs" : {

// A set of values to be made visible to the stack creator},"AWSTemplateFormatVersion" : "2010-09-09”

}

aws cloudformation create-stack --stack-name ec2InstanceCmdLineDemo--template-url https://s3-eu-west-1.amazonaws.com/cf-templates-

1fhelryvrdrbr-eu-west-1/2014174d0r-ec2Instance.template--parameters ParameterKey=KeyPair,ParameterValue=ManagementKeyPair

arn:aws:cloudformation:eu-west-1:554625704737:stack/ec2InstanceCmdLineDemo/42cc6150-fad7-11e3-8f4d-5017e1aef4e7

UsingatemplatetocreateandmanageastackviatheAWSCLI

Returnsthedetailsofthecreatedstack,intheoutputformatofyourchoice

Samples, samples, samples• Sample templates

– Spin x up, do y on service z• Application frameworks

– LAMP, Rails, Flash etc• Sample Solutions

– SharePoint, WordPress, Chef• Community labs

– https://github.com/awslabs• Reference Implementations

– Exchange, Lynx Server, AD, SAP, Big Data– CfnCluster

• Integrate with your development and management tools

Accessing resources

Console, CLI’s and SDK’s

• Remember- every resource in AWS is accessible through an API

• Everything you can do through the Console, you can do through the CLI or an SDK

• SDK’s for most programming languages– Android, IOS, Java, .Net, Node.js, PHP, Python, Ruby, Go

• IDE’s– Eclipse, Visual Studio

Commercial models

Many pricing models to support different workloads

Reserved

Make a low, one-time payment and receive a significant discount on the hourly charge

For committed utilization

Free Tier

Get Started on AWS with free usage & no commitment

For POCs and getting started

On-Demand

Pay for compute capacity by the hour with no long-term commitments

For spiky workloads, or to define needs

Spot

Bid for unused capacity, charged at a Spot Price which fluctuates based on supply and demand

For time-insensitive or transient workloads

Dedicated

Launch instances within Amazon VPC that run on hardware dedicated to a single customer

For highly sensitive or compliance related workloads

Keeping track of your spend(Billing alerts)

We Want to Avoid This!

Cost Explorer

• Forecasts• Budgets

CloudWatch Billing Alerts

Securing your resources

Shared security responsibility

AWS

• Facilities• Physical Security• Physical Infrastructure• Network Infrastructure• Virtualization

Infrastructure

• Operating System• Application• Security Groups• OS Firewalls• Network Configuration• Account Management

Customer

Secure your account

Identity and Access Management

• Users & Groups

Identity and Access Management

• Users & Groups• Unique Security Credentials

Identity and Access Management

• Users & Groups• Unique Security Credentials• Temporary Security

Credentials

Identity and Access Management

• Users & Groups• Unique Security Credentials• Temporary Security

Credentials• Policies & Permissions

Identity and Access Management

• Users & Groups• Unique Security Credentials• Temporary Security

Credentials• Policies & Permissions• Roles

Identity and Access Management• Users & Groups• Unique Security Credentials• Temporary Security

Credentials• Policies & Permissions• Roles• Multi-factor Authentication

Built in security features

• Secure access• VPC’s• Security groups• Private subnets• Security logs (CloudTrail)

Research workloads in the Cloud

Popular HPC workloads on AWS

Genome processing

Modeling and Simulation

Government and Educational Research

Monte Carlo Simulations

Transcoding and Encoding

ComputationalChemistry

AWS Marketplace – HPC category

aws.amazon.com/marketplace

AWSHPCMarketplacecategory

Education

Certification

aws.amazon.com/certification

Self-Paced Labs

aws.amazon.com/training/self-paced-labs

Try products, gain new skills, and get hands-on practice

working with AWS technologies

aws.amazon.com/training

Training

Validate your proven skills and expertise with the AWS platform

Build technical expertise to design and operate scalable, efficient applications on AWS

AWSTrainingandCertification

AWS EducateAmazon’s Global Initiative for Educational Institutions, Educators & Students

Robust Offering: Four Pillars of AWS Educate

What is AWS Academy?

• AWS Academy provides participating educational institutions worldwide with an AWS-authorized curriculum, so institutions can more easily offer cloud computing courses and students can become proficient and certified on AWS technology.

• https://aws.amazon.com/education/awsacademy/

AWS-authorizedcurriculum

GlobalreachPost-secondaryaccreditedprograms

AWSCertification

Break