advanced analytics for the internet of things · copyright © 2015 knime.com ag input features...
TRANSCRIPT
![Page 1: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/1.jpg)
Copyright © 2015 KNIME.com AG
Advanced Analytics for the Internet of Things
Rosaria Silipo KNIME
![Page 2: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/2.jpg)
Copyright © 2015 KNIME.com AG
IoT is here …
2
Household
Energy
Wearables
Health
City
Industry
![Page 3: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/3.jpg)
Copyright © 2015 KNIME.com AG
The Challenges
• Handling very large amounts of data created over time
• Forcing sensor-equipped objects (house or city) to learn, and therefore to become smarter
![Page 4: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/4.jpg)
Copyright © 2015 KNIME.com AG
IoT
Illustration by CRISTINA BYVIK
Use Public Data Please…. Energy Consumption Prediction Energy Profiling
Restocking Strategies Geo-localization Traffic Predictions
Anomaly Detection
![Page 5: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/5.jpg)
Copyright © 2015 KNIME.com AG
![Page 6: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/6.jpg)
Copyright © 2015 KNIME.com AG
This Use Case
Capital Bikeshare in Washington DC
![Page 7: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/7.jpg)
Copyright © 2015 KNIME.com AG
![Page 8: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/8.jpg)
Copyright © 2015 KNIME.com AG
Sensors!
![Page 9: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/9.jpg)
Copyright © 2015 KNIME.com AG
The Business Challenge:
![Page 10: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/10.jpg)
Copyright © 2015 KNIME.com AG
Even MORE of a Business Challenge
• Any Station without bikes for 1 hour:
$XXXX Per Violation
• Any Station with no free slots for 1 hour:
$XXXX Per Violation
![Page 11: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/11.jpg)
Copyright © 2015 KNIME.com AG http://bikeportland.org/2013/03/10/behind-the-scenes-of-capital-bikeshare-84006
Capital Bikeshare Response
Over 3 years 307 Stations 2963 Bikes 19.4% Casual Bikers 5.9m Bike Moves
![Page 12: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/12.jpg)
Copyright © 2015 KNIME.com AG
Advanced Analytics for the Internet of Things
Pre-processing Data Visualization Predictive Analytics
![Page 13: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/13.jpg)
Copyright © 2015 KNIME.com AG
![Page 14: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/14.jpg)
Copyright © 2015 KNIME.com AG
Reading all Sensor Data
![Page 15: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/15.jpg)
Copyright © 2015 KNIME.com AG
Topology / Elevations
Weather
Holiday Schedules
Commuters
Enrich
![Page 16: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/16.jpg)
Copyright © 2015 KNIME.com AG
Elevation from Google API
Connecting to Google API and other REST services available on the Web
![Page 17: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/17.jpg)
Copyright © 2015 KNIME.com AG
Expand/Transform
![Page 18: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/18.jpg)
Copyright © 2015 KNIME.com AG
![Page 19: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/19.jpg)
Copyright © 2015 KNIME.com AG
Station and Bike Facts
Over 3 years 307 Stations 2963 Bikes 19.4% Casual Bikers 5.9m Bike Moves
![Page 20: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/20.jpg)
Copyright © 2015 KNIME.com AG
Stations: deficits and surpluses
![Page 21: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/21.jpg)
Copyright © 2015 KNIME.com AG
Top 250 Routes
![Page 22: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/22.jpg)
Copyright © 2015 KNIME.com AG
Total Bikers Number per Hour
• Registered (blue) vs. Casual (red) Bikers
![Page 23: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/23.jpg)
Copyright © 2015 KNIME.com AG
![Page 24: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/24.jpg)
Copyright © 2015 KNIME.com AG
The Goal
• Restocking alert signal
• 1 hour warning! Lag(flag-1)
![Page 25: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/25.jpg)
Copyright © 2015 KNIME.com AG
Input Features
• Weather related features
• Number of registered and casual people showing up
• Station infos (name and max. number of docks)
• Calendar infos (working day, holiday, date)
• Past infos
• Number of bikes added and removed at each hour
• Adjusted cumulative sum = number of bikes available at the station at a given hour
• Bike ratio = adjusted cumulative sum/total docks available
![Page 26: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/26.jpg)
Copyright © 2015 KNIME.com AG
Restocking Alert System
Train
Apply Evaluate
Partition
78% accuracy
![Page 27: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/27.jpg)
Copyright © 2015 KNIME.com AG
Feature Elimination Loop
Two options:
1. Use all the input features (no thinking required, just a powerful machine)
2. Select the most useful input features via the “Feature Elimination” loop
At each step, one input feature is removed - i.e. the input feature whose removal produces the smallest error increase.
![Page 28: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/28.jpg)
Copyright © 2015 KNIME.com AG
Input Attribute Impact
![Page 29: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/29.jpg)
Copyright © 2015 KNIME.com AG
Lean Restocking Alert System
The input feature subset with the smallest error (81% accuracy):
• Hour of the day
• Working day (Y/N)
• Current Bike Ratio
• Terminal (station code)
Past of bike ratio and weather infos do not seem to be relevant!
Is this because most bikers are registered members?
![Page 30: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/30.jpg)
Copyright © 2015 KNIME.com AG
Lessons Learned
• Enrich / Expand
– KNIME Transformations
– REST calls to external sources
• Explore
– Visualization with Open Street Map integration
– Network Analysis (Graph Visualization)
• Prediction
– (Lean) Restocking Alert System
– Weather influence does not seem to be important!
![Page 31: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/31.jpg)
Copyright © 2015 KNIME.com AG
Resources
• KNIME (www.knime.org) • BLOG for news, tips and tricks(www.knime.org/blog)
• FORUM for questions and answers (tech.knime.org/forum)
• EXAMPLE SERVER for example workflows
• E-LEARNING COURSE (https://www.knime.org/knime-introductory-course)
• KNIME TV channel on
• KNIME on @KNIME
• KNIME on https://www.facebook.com/KNIMEanalytics
31
![Page 32: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/32.jpg)
Copyright © 2015 KNIME.com AG
Free Book
Free Copy of KNIME Beginner’s Luck Book at KNIME Press https://www.knime.org/knimepress
Promotion Code:
SaoPaulo_KNIMEMeetup2017
![Page 33: Advanced Analytics for the Internet of Things · Copyright © 2015 KNIME.com AG Input Features •Weather related features •Number of registered and casual people showing up •Station](https://reader033.vdocuments.us/reader033/viewer/2022060502/5f1c3e2d42b66a38a508d74a/html5/thumbnails/33.jpg)
Copyright © 2015 KNIME.com AG
Whitepaper, Workflows, and Data is available on the KNIME web site: http://www.knime.com/white-papers
Where can I find all this?