Download - Machine Learning for Developers
![Page 1: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/1.jpg)
©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved
Machine Learning for Developers Danilo Poccia – Technical Evangelist
@danilop danilop
![Page 2: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/2.jpg)
Amazon.com
1994
![Page 3: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/3.jpg)
![Page 4: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/4.jpg)
![Page 5: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/5.jpg)
Three types of data-driven development
Retrospective analysis and
reporting
Amazon Redshift Amazon RDS Amazon S3
Amazon EMR
![Page 6: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/6.jpg)
Three types of data-driven development
Retrospective analysis and
reporting
Here-and-now real-time processing
and dashboards
Amazon Kinesis Amazon EC2 AWS Lambda
Amazon Redshift Amazon RDS Amazon S3
Amazon EMR
![Page 7: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/7.jpg)
Three types of data-driven development
Retrospective analysis and
reporting
Here-and-now real-time processing
and dashboards
Predictions to enable smart
applications
Amazon Kinesis Amazon EC2 AWS Lambda
Amazon Redshift Amazon RDS Amazon S3
Amazon EMR
![Page 8: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/8.jpg)
Machine learning and smart applications
Machine learning is the technology that automatically finds patterns in your data and uses them to make predictions for new data points as they become available
![Page 9: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/9.jpg)
Machine learning and smart applications
Machine learning is the technology that automatically finds patterns in your data and uses them to make predictions for new data points as they become available Your data + machine learning = smart applications
![Page 10: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/10.jpg)
Smart applications by example
Based on what you know about the users: Will they use your product?
![Page 11: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/11.jpg)
Smart applications by example
Based on what you know about the user: Will they use your product?
Based on what you know about an order: Is this order fraudulent?
![Page 12: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/12.jpg)
Smart applications by example
Based on what you know about the user: Will they use your product?
Based on what you know about an order: Is this order fraudulent?
Based on what you know about a news article: What other articles are interesting?
![Page 13: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/13.jpg)
And a few more examples…
Fraud detection Detecting fraudulent transactions, filtering spam emails, flagging suspicious reviews, …
Personalization Recommending content, predictive content loading, improving user experience, …
Targeted marketing Matching customers and offers, choosing marketing campaigns, cross-selling and up-selling, …
Content classification Categorizing documents, matching hiring managers and resumes, …
Churn prediction Finding customers who are likely to stop using the service, free-tier upgrade targeting, …
Customer support Predictive routing of customer emails, social media listening, …
![Page 14: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/14.jpg)
Why aren’t there more smart applications?
1. Machine learning expertise is rare
2. Building and scaling machine learning technology is hard
3. Closing the gap between models and applications is time-consuming and expensive
![Page 15: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/15.jpg)
2001
![Page 16: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/16.jpg)
Decentralized, two-pizza teams Agility, autonomy, accountability, and ownership “DevOps” “Microservices”
![Page 17: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/17.jpg)
Introducing Amazon ML
Easy to use, managed machine learning service built for developers
Robust, powerful machine learning technology based on Amazon’s internal systems
Create models using your data already stored in the AWS cloud
Deploy models to production in seconds
![Page 18: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/18.jpg)
Easy to use and developer-friendly
Use the intuitive, powerful service console to build and explore your initial models
– Data retrieval – Model training, quality evaluation, fine-tuning – Deployment and management
Automate model lifecycle with fully featured APIs and SDKs
– Java, Python, .NET, JavaScript, Ruby, PHP Easily create smart iOS and Android applications with AWS Mobile SDK
![Page 19: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/19.jpg)
Powerful machine learning technology
Based on Amazon’s battle-hardened internal systems
Not just the algorithms: – Smart data transformations – Input data and model quality alerts – Built-in industry best practices
Grows with your needs
– Train on up to 100 GB of data – Generate billions of predictions – Obtain predictions in batches or real-time
![Page 20: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/20.jpg)
Integrated with AWS data ecosystem
Access data that is stored in S3, Amazon Redshift, or MySQL databases in RDS Output predictions to S3 for easy integration with your data flows Use AWS Identity and Access Management (IAM) for fine-grained data-access permission policies
![Page 21: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/21.jpg)
Fully managed model and prediction services
End-to-end service, with no servers to provision and manage
One-click production model deployment
Programmatically query model metadata to enable automatic retraining workflows
Monitor prediction usage patterns with Amazon CloudWatch metrics
![Page 22: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/22.jpg)
Build model
Evaluate and optimize
Retrieve predictions
1 2 3
Building smart applications with Amazon ML
![Page 23: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/23.jpg)
Train model
Evaluate and optimize
Retrieve predictions
1 2 3
Building smart applications with Amazon ML
- Create a Datasource object pointing to your data - Explore and understand your data - Transform data and train your model
![Page 24: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/24.jpg)
Create a Datasource object
>>> import boto
>>> ml = boto.connect_machinelearning()
>>> ds = ml.create_data_source_from_s3(
data_source_id = ’my_datasource',
data_spec= {
'DataLocationS3':'s3://bucket/input/',
'DataSchemaLocationS3':'s3://bucket/input/.schema'},
compute_statistics = True)
![Page 25: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/25.jpg)
Explore and understand your data
![Page 26: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/26.jpg)
Train your model
>>> import boto
>>> ml = boto.connect_machinelearning()
>>> model = ml.create_ml_model(
ml_model_id=’my_model',
ml_model_type='REGRESSION',
training_data_source_id='my_datasource')
![Page 27: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/27.jpg)
Train model
Evaluate and optimize
Retrieve predictions
1 2 3
Building smart applications with Amazon ML
- Understand model quality - Adjust model interpretation
![Page 28: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/28.jpg)
Explore model quality
![Page 29: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/29.jpg)
Fine-tune model interpretation
![Page 30: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/30.jpg)
Fine-tune model interpretation
![Page 31: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/31.jpg)
Train model
Evaluate and optimize
Retrieve predictions
1 2 3
Building smart applications with Amazon ML
- Batch predictions - Real-time predictions
![Page 32: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/32.jpg)
Batch predictions
Asynchronous, large-volume prediction generation Request through service console or API Best for applications that deal with batches of data records
>>> import boto
>>> ml = boto.connect_machinelearning()
>>> model = ml.create_batch_prediction(
batch_prediction_id = 'my_batch_prediction’
batch_prediction_data_source_id = ’my_datasource’
ml_model_id = ’my_model',
output_uri = 's3://examplebucket/output/’)
![Page 33: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/33.jpg)
Real-time predictions
Synchronous, low-latency, high-throughput prediction generation Request through service API or server or mobile SDKs Best for interaction applications that deal with individual data records
>>> import boto
>>> ml = boto.connect_machinelearning()
>>> ml.predict(
ml_model_id=’my_model',
predict_endpoint=’example_endpoint’,
record={’key1':’value1’, ’key2':’value2’})
{ 'Prediction': {
'predictedValue': 13.284348,
'details': {
'Algorithm': 'SGD',
'PredictiveModelType': 'REGRESSION’
}
}
}
![Page 34: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/34.jpg)
<demo> …
</demo>
![Page 35: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/35.jpg)
Architecture patterns for smart applications
![Page 36: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/36.jpg)
Batch predictions with EMR
Query for predictions with Amazon ML batch API
Process data with EMR
Raw data in S3 Aggregated data
in S3 Predictions
in S3 Your application
![Page 37: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/37.jpg)
Batch predictions with Amazon Redshift
Structured data In Amazon Redshift
Load predictions into Amazon Redshift
-or- Read prediction results
directly from S3 Predictions
in S3
Query for predictions with Amazon ML batch API
Your application
![Page 38: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/38.jpg)
Real-time predictions for interactive applications
Your application
Query for predictions with Amazon ML real-time API
![Page 39: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/39.jpg)
Adding predictions to an existing data flow
Your application Amazon
DynamoDB
Lambda
+
Trigger event with Lambda +
Query for predictions with Amazon ML real-time API
![Page 40: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/40.jpg)
Where is the Craftsmanship?
![Page 41: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/41.jpg)
Urban Traffic Model
Date
Day of the Week
It is an Holiday?
Weather
Rain (mm)
Temperature
![Page 42: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/42.jpg)
75% of users select movies based on recommendations
![Page 43: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/43.jpg)
![Page 44: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/44.jpg)
Abraham Wald
![Page 45: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/45.jpg)
Pay-as-you-go and inexpensive
Data analysis, model training, and evaluation: $0.42/instance hour
Batch predictions: $0.10/1000 Real-time predictions: $0.10/1000 + hourly capacity reservation charge
![Page 46: Machine Learning for Developers](https://reader031.vdocuments.us/reader031/viewer/2022030313/58ed05cf1a28ab7b118b4595/html5/thumbnails/46.jpg)
©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved
Machine Learning for Developers Danilo Poccia – Technical Evangelist
@danilop danilop