from zero to hero on awseventos.uanl.mx/techday/archivos/amazonws.pdf · from zero to hero on aws...
Post on 22-May-2020
6 Views
Preview:
TRANSCRIPT
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
From Zero to Hero on AWSEduardo Patiño - balague@amazon.comMauricio Trejo – trejmaur@amazon.com
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
?
… AWS: plataforma bajodemanda de cómputo en la
nube
Amazon.com…
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
AWS for Research & Technical Computing
Time to ScienceAccess research
infrastructure in minutes
Low CostPay-as-you-go pricing
ElasticEasily add or remove
capacity
Globally AccessibleEasily Collaborate with
researchers around the world
SecureA collection of tools to
protect data and privacy
ScalableAccess to effectively
limitless capacity
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Collaboration is easier in the cloud
More time spent computing the data than moving the data.
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
AWS Global Infrastructure
18 Reg ions
55AZs
121 EdgeLocations
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Sample US Region
Datacenter Datacenter
Datacenter
Sample Availability Zone
Availability Zone B
Availability Zone A
Availability Zone C
Anatomy of a region
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
A tool for almost every job
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
A tool for almost every jobbecause you’ll want tools you don’t know about yet
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Account Support
Support
Managed Services
Professional Services
Partner Ecosystem
Training & Certification
Solution Architects
Account Management
Security & Pricing Reports
Technical Acct. Management
Marketplace
Business Applications
DevOps Tools
Business Intelligence
Security
Networking
Database & Storage
SaaS Subscriptions
Operating Systems
Mobile
Build, Test, Monitor Apps
Push Notifications
Build, Deploy, Manage APIs
Device Testing
Identity
Enterprise Applications
Document Sharing
Email & Calendaring
Hosted Desktops
Application Streaming
Backup
Game Development
3D Game Engine
Multi-player Backends
Mgmt. Tools
Monitoring
Auditing
Service Catalog
Server Management
Configuration Tracking
Optimization
Resource Templates
Automation
Analytics
Query Large Data Sets
Elasticsearch
Business Analytics
Hadoop/Spark
Real-time Data Streaming
Orchestration Workflows
Managed Search
Managed ETL
Artificial Intelligence
Voice & Text Chatbots
Machine Learning
Text-to-Speech
Image Analysis
IoT
Rules Engine
Local Compute and Sync
Device Shadows
Device Gateway
Registry
Hybrid Devices & Edge SystemsData Integration Integrated
NetworkingResource
ManagementVMware on
AWSIdentity
Federation
Migration Application Discovery
Application Migration
Database Migration
Server MigrationData Migration
Infrastructure Regions Availability Zones
Points of Presence
Compute Containers Event-driven Computing
Virtual Machines Simple Servers Auto Scaling BatchWeb
Applications
Storage Object Storage Archive Block Storage Managed File Storage
Exabyte-scale Data Transport
Database MariaDB Data Warehousing NoSQLAurora MySQL Oracle SQL ServerPostgreSQL
ApplicationServices Transcoding Step Functions Messaging
Security Certificate Management
Web App. Firewall
Identity & Access
Key Storage & Management
DDoS Protection
Application AnalysisActive Directory
Dev Tools
Private Git Repositories
Continuous Delivery
Build, Test, and Debug
Deployment
Networking Isolated Resources
Dedicated Connections Load Balancing Scalable DNSGlobal CDN
The AWS Platform
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
With AWS, Security Is a Shared Responsibility
DatabaseStorageCompute Networking
Edge Locations
Regions
Avail. Zones
AWS Global Infrastructure
Customers are responsible for
security ‘in’ the Cloud
AWS is responsible for security ‘of’ the Cloud
Customer Data
Platform, Applications, Identity & Access Management
Operating System, Network & Firewall Configuration
Client-side Data Encryption & Data
Integrity Authentication
Server-side Encryption (File System and/or
Data)
Network Traffic Protection (Encryption /
Integrity / Identity)
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
EC2 Compute Instance Types
Well suited for AI/ML simulation and inference workloads
Well suited for AI/ML training workloads
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Clusters in the cloud are fit for purpose
Amazon S3
R4P2 P2 P2
P2 P2 P2 R4C5
C5
C5 C5
C5 C5
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
But it’s not (just) about servers…
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
§Cuanto Alm
acenamiento
necesito?
§ ¿No tengo un cátalo de
Software
preaprobado?§ ¿Necesito HSM?
§¿có
mo Audito to
do lo
que pasa en mi ce
ntro
de datos?§ ¿cómo Automatizo si
se viola alguna politica?
§ ¿Cambio de configuración?
§ ¿Y si tengo un DDoS?
§ ¿Y si la versión no compila ?
§ ¿cuáles son las
herramientas
modernas que
solucionan los
problemas del pasado?
§ ¿Sera necesario tener
un DRP?
§ ¿LOS CONTRATOS
ACTUALES SON
COMPLEJOS?
§ ¿Probar Software que cumpla tarda meses?
§ ¿No tengo Especialista
para Instalar cada App?
§ Cuantos Servidores§ Prevención de perdida
de Datos !! Suena Bonito pero Costoso§ ¿Auditar Logs !! ?
Escasamente los
puede guardar 1
semana !!!
§¿y la seguirdad y la
auditoria ?§ ¿Encriptar o No
Encriptar?
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Physical Virtualisation Containerization Serverless
Evolving Compute Abstractions
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Big DataTechnologies and techniques for
working productively with massive amounts of data at any scale in
either batch or real-time.
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Velocity Volume
Variety
Three Vs of Big Data
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
GB TBPB
ZB
EB
Big Data: Unconstrained Growth
Unstructured data growth is explosive
95% of the 1.2 zettabytes of data in the digital universe is unstructured
Machine data and IoT will only steepen the curve
70% of this data is user-generated content
Source: IDC, The Internet of Things: Getting Ready to Embrace Its Impact on the Digital Economy, March 2016.
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Relational
NoSQL
Web servers
Mobile
phones/Tablets
3rd party feeds
IoT
Clickstream
Sources
Big Data Sources
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Structured
Unstructured
Text
Binary
Real-time/Near Real-time
Batched
Formats
Big Data Formats and Velocity
Velocity
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Managed Services for Analytics
Retrospectiveanalysis and
reporting
Here-and-nowreal-time processing
and dashboards
Predictionsto enable smart
applications
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Why Big Data?
Security threat detection
User Behavior Analysis
Smart Application (Machine Learning)
Business Intelligence
Fraud detection
Financial Modeling and Forecasting
Spending optimization
Real-time alerting
Get answers faster and be able to ask questions not possible to today.
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Elastic and highly scalable
No upfront capital expense
Only pay for what you use+
+
Available on-demand+
= the Cloud removes constraints
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
No one tool rules them all
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
AWS Big Data Platform
EMR EC2
Glacier
S3
Import Export
Kinesis
Direct Connect
Machine LearningRedshift
DynamoDB
AWS Database Migration Service
Collect Orchestrate Store Analyze
AWS Lambda
AWS IoT
AWS Data Pipeline
Amazon Kinesis Analytics
AmazonSNS
AWS Snowball
AmazonSWF
Amazon Athena
Amazon QuickSight
Amazon AuroraAWS Glue
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Big data ≠ big cost
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Sample Reference Architecture: Data Lake
AthenaGlue
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Some AWS Big Data Services
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
• Scalable object storage for the Internet• 1 byte to 5 TB in size per object +
unlimited number of objects• 99.999999999% durability, 99.99%
availability• Regional service, no single points of
failure• Server side encryption
Storage
AWS Global Infrastructure
App Services
Deployment & Administration
Networking
Compute Database
Analytics
Amazon S3
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Why Is Amazon S3 Good for Big Data?
• Natively supported by big data frameworks (Spark, Hive, Presto, etc.) • No need to run compute clusters for storage (unlike HDFS)• Must be used for transient Hadoop clusters & Amazon EC2 Spot instances• Multiple distinct (Spark, Hive, Presto) clusters can use the same data• Unlimited number of objects • Very high bandwidth – no aggregate throughput limit• Tiered-storage (Standard, IA, Amazon Glacier) via life-cycle policy• Secure – SSL, client/server-side encryption at rest• Low cost
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Object Storage Use Cases
S3
S3-IA
Glacier
Cloud Applications
Big DataAnalytics
Content Distribution Primary Data
File Sync & Share
ActiveArchive
EnterpriseBackup
MediaTranscoding
Disaster Recovery /Geo Redundancy
Deep / Offline
Archives
Tape Vaulting Replacement
WORM Compliant
Data
Temporary & Small
Objects
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
L
i
f
e
c
y
c
l
e
AvailableS3: 99.99%
S3-IA: 99.9%
PerformantLow Latency
High Throughput≥ 30 Days≥ 128K
≥ 90 Days
Durable99.999999999%
ScalableElastic capacity No preset limits
> 0K$0.004 / GB per month
$0.0125 / GB per month
“Hot” DataActive and/or
Temporary Data
“Warm” DataInfrequently
Accessed Data
“Cold” DataArchive and
Compliance Data
≥ 0 Days> 0KStarts at $0.023 / GB per month
1-5 mins
$0.01/GB retrieval
Storage Tiered To Your Requirements
S3-IA
Glacier
S3
3 new retrieval options
3–5 hrs 5–12 hrs
Expedited Standard Bulk$0.03 / GB $0.01 / GB $0.0025 / GB
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon Glacier•Low-Cost Archival Storage•Secure
• SSL & AES-256
•Durable• Designed for 99.999999999% durability
•Optimized for data archiving and backup• Suitable for RTO measured in hours• Includes storage costs and retrieval costs
•Three retrieval options: Expedited, Standard, Bulk •As little as $0.004 per GB/Month (US East pricing)•Integrated with S3
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Query and analyze Amazon S3 data with
standard (ANSI) SQL
No ETL required
Serverless and simple
Pay only for queries you run
Amazon Athena
Storage
AWS Global Infrastructure
App Services
Deployment & Administration
Networking
Analytics
Compute Database
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
BI service performs ad-hoc analysis
Build intuitive visualizations
Share and collaborate via storyboards
Native access on major mobile platforms
Fast insights with SPICE
Amazon QuickSight
Storage
AWS Global Infrastructure
App Services
Deployment & Administration
Networking
Analytics
Compute Database
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Open Data at AWS
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
IT’S ABOUT SCIENCE, NOT SERVERS.
aws.amazon.com/rcp
We recognize that whilst research is often a compute-intensive activity, most researchers are not IT experts.
We want to simplify research in the cloud with easy-to-use tools for researchers and their students, and share the catalogue of “researcher-obsessed” products and services created by many of our partners.
ElevationModels
AerialImagery
ClimateModels
SatelliteImagery
High-resolutionRadar
aws.amazon.com/earth
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Landsat 8 Lives on AWS
Within the first year of Landsat on AWS, the data has been requested over 1
billion times, globally. Over 400,000 scenes
are now available
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Machine Learning on AWS
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
A long heritage of machine learning at Amazon
Personalized recommendations
Inventing entirely new customer experiences
Fulfillmentautomation and
inventory management
Drones Voice driven interactions
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
The Amazon Machine Learning Stack
FRAMEWORKS & INTERFACES
Caffe2 CNTKApache MXNet
PyTorch TensorFlow Chainer Keras Gluon
AWS Deep Learning AMIs
Amazon SageMaker
Rekognition Transcribe Translate Polly Comprehend Lex
AWS DeepLens
EDUCATION
PLATFORM SERVICES
APPLICATION SERVICES
APPLICATION SERVICES
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
API-driven Services
Services
Platforms
Frameworks
Infrastructure
Apache MXNet TorchCognitiv
e ToolkitKerasTheanoCaffe2 & CaffeTensorFlow
AWS Deep Learning AMI
GPU MobileCPU IoT
Amazon ML ECSSpark & EMR Kinesis Batch
Vision Speech Language
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon RekognitionDeep learning-based image recognition serviceSearch, verify, and organize millions of images
Object and SceneDetection
FacialAnalysis
FaceComparison
FacialRecognition
Integrated with S3, Lambda, Polly, Lex
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon Rekognition
Potential Use CasesSearchable Image Library Detect Inappropriate Content in
ImagesFace-based User Verification Sentiment AnalysisFacial Recognition Celebrity Identification
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon PollyPotential Use CasesContent Creation Education & E-learningMobile & Desktop Applications Customer Contact CenterInternet of Things (IoT) Accessibility
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon Lex
Potential Use CasesAppointment Booking Customer SupportInformational Services Access Enterprise DataInternet of Things (IoT)
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Building better machine learning models for recommendation systems
Airbnb is using machine learning to optimize search recommendations and improve dynamic pricing guidance for hosts, both of which translate to increased booking conversions. With Amazon EC2 P3 instances, Airbnb has the ability to run training workloads faster, go through more iterations, build better machine learning models and reduce cost.
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
http://aws.amazon.com/free
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Como empezamos con AWS ?
1. Inscríbase en el programa AWS Educate de manera gratuita www.awseducate.com
2. Abra una cuenta gratuita de AWS y use servicios de manera gratuita https://aws.amazon.com/es/free/
3. Tutoriales de 10 minutos - https://aws.amazon.com/es/getting-started/4. Entrenamiento - https://aws.amazon.com/es/training/5. Contacte a su gerente de cuenta - Mauricio Trejo trejmaur@amazon.com6. Involucre a un Arquitecto de Soluciones para explorar
• Infraestructura y Servicios principales• Disaster Recovery y Migraciones• Seguridad y Cumplimiento, DevOps• Big data, Inteligencia de negocios, Inteligencia Artificial• Aplicaciones Enterprise
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Thank you!
Eduardo PatinoSolutions Architect, AWS
top related