machine learning with spark - meetupfiles.meetup.com/19103884/3_ml_intro.pdf · mercedes-benz ml...
TRANSCRIPT
![Page 1: Machine Learning with Spark - Meetupfiles.meetup.com/19103884/3_ML_intro.pdf · Mercedes-Benz ML 270 turbodiesel cat CD' € 04/2000 251,000 km 120 kW hp 9 private, 1-00038 vatmontone](https://reader034.vdocuments.us/reader034/viewer/2022042218/5ec471b57de7b60a1b6d79a0/html5/thumbnails/1.jpg)
Machine Learning with Spark
What is Machine Learning?
![Page 2: Machine Learning with Spark - Meetupfiles.meetup.com/19103884/3_ML_intro.pdf · Mercedes-Benz ML 270 turbodiesel cat CD' € 04/2000 251,000 km 120 kW hp 9 private, 1-00038 vatmontone](https://reader034.vdocuments.us/reader034/viewer/2022042218/5ec471b57de7b60a1b6d79a0/html5/thumbnails/2.jpg)
![Page 3: Machine Learning with Spark - Meetupfiles.meetup.com/19103884/3_ML_intro.pdf · Mercedes-Benz ML 270 turbodiesel cat CD' € 04/2000 251,000 km 120 kW hp 9 private, 1-00038 vatmontone](https://reader034.vdocuments.us/reader034/viewer/2022042218/5ec471b57de7b60a1b6d79a0/html5/thumbnails/3.jpg)
Gain wisdom around 500 BC Input/Output system
? Wisdom
Question
Wisdom
![Page 4: Machine Learning with Spark - Meetupfiles.meetup.com/19103884/3_ML_intro.pdf · Mercedes-Benz ML 270 turbodiesel cat CD' € 04/2000 251,000 km 120 kW hp 9 private, 1-00038 vatmontone](https://reader034.vdocuments.us/reader034/viewer/2022042218/5ec471b57de7b60a1b6d79a0/html5/thumbnails/4.jpg)
Gain wisdom around 1990 Input/Output system
? Wisdom
Question
![Page 5: Machine Learning with Spark - Meetupfiles.meetup.com/19103884/3_ML_intro.pdf · Mercedes-Benz ML 270 turbodiesel cat CD' € 04/2000 251,000 km 120 kW hp 9 private, 1-00038 vatmontone](https://reader034.vdocuments.us/reader034/viewer/2022042218/5ec471b57de7b60a1b6d79a0/html5/thumbnails/5.jpg)
? Wisdom
Question
Gain wisdom around 2016 Input/Output system
![Page 6: Machine Learning with Spark - Meetupfiles.meetup.com/19103884/3_ML_intro.pdf · Mercedes-Benz ML 270 turbodiesel cat CD' € 04/2000 251,000 km 120 kW hp 9 private, 1-00038 vatmontone](https://reader034.vdocuments.us/reader034/viewer/2022042218/5ec471b57de7b60a1b6d79a0/html5/thumbnails/6.jpg)
Work 1 month for 30data points
Wait 1 second for 30.000 points
![Page 7: Machine Learning with Spark - Meetupfiles.meetup.com/19103884/3_ML_intro.pdf · Mercedes-Benz ML 270 turbodiesel cat CD' € 04/2000 251,000 km 120 kW hp 9 private, 1-00038 vatmontone](https://reader034.vdocuments.us/reader034/viewer/2022042218/5ec471b57de7b60a1b6d79a0/html5/thumbnails/7.jpg)
“More data usually beats better algorithms”Anand Rajaraman (when teaching at Stanford)
http://anand.typepad.com/datawocky/2008/03/more-data-usual.html
Old Skool StatisticsOld Skool Statistics Big Data!Big Data!
Nice read!
![Page 8: Machine Learning with Spark - Meetupfiles.meetup.com/19103884/3_ML_intro.pdf · Mercedes-Benz ML 270 turbodiesel cat CD' € 04/2000 251,000 km 120 kW hp 9 private, 1-00038 vatmontone](https://reader034.vdocuments.us/reader034/viewer/2022042218/5ec471b57de7b60a1b6d79a0/html5/thumbnails/8.jpg)
![Page 9: Machine Learning with Spark - Meetupfiles.meetup.com/19103884/3_ML_intro.pdf · Mercedes-Benz ML 270 turbodiesel cat CD' € 04/2000 251,000 km 120 kW hp 9 private, 1-00038 vatmontone](https://reader034.vdocuments.us/reader034/viewer/2022042218/5ec471b57de7b60a1b6d79a0/html5/thumbnails/9.jpg)
Use Cases with Spark
![Page 10: Machine Learning with Spark - Meetupfiles.meetup.com/19103884/3_ML_intro.pdf · Mercedes-Benz ML 270 turbodiesel cat CD' € 04/2000 251,000 km 120 kW hp 9 private, 1-00038 vatmontone](https://reader034.vdocuments.us/reader034/viewer/2022042218/5ec471b57de7b60a1b6d79a0/html5/thumbnails/10.jpg)
MLlib demo recommendation engine
![Page 11: Machine Learning with Spark - Meetupfiles.meetup.com/19103884/3_ML_intro.pdf · Mercedes-Benz ML 270 turbodiesel cat CD' € 04/2000 251,000 km 120 kW hp 9 private, 1-00038 vatmontone](https://reader034.vdocuments.us/reader034/viewer/2022042218/5ec471b57de7b60a1b6d79a0/html5/thumbnails/11.jpg)
RDD
ML Model
Result
1. Create an RDD, Map/Reduce to clean the Data
2. Use a pre-build Spark ML technique to calculate a Model
3. Buy my next Car
MLlib demo car market model
![Page 12: Machine Learning with Spark - Meetupfiles.meetup.com/19103884/3_ML_intro.pdf · Mercedes-Benz ML 270 turbodiesel cat CD' € 04/2000 251,000 km 120 kW hp 9 private, 1-00038 vatmontone](https://reader034.vdocuments.us/reader034/viewer/2022042218/5ec471b57de7b60a1b6d79a0/html5/thumbnails/12.jpg)
MLlib overview
Next Meetup?Play around with
Spark, Mllib?
Kaggle Competition?