analytics for iot: from sensors to decisions tom dietterich distinguished professor oregon state...
TRANSCRIPT
1
Analytics for IoT: From Sensors to Decisions
Tom Dietterich
Distinguished Professor
Oregon State University
2
The Usual View of IoT
Materials
Devices
Systems
Receiver Transmitter
420 µm
650 µm
1000 µm
550 µm
Networks
Σ DSP Processing
TransducerArray
Variable Length Delay Line
N Channels
Receive Beamformer Channels
TS
TS
TS
TS
TS
TS
TGA
ProcessingElements
∆Σ-M
∆Σ-M
∆Σ-M
3
Sensors123RF Limited
4
Spatio-Temporal Analytics Hierarchy
Sensors123RF Limited
Cleaned Data
StateVariables
Trajectories
Events & Activities
Models
5
Leadership in AnalyticsMachine Learning, Data Mining, Data Science
Hector Cotilla-SanchezTom DietterichAlan FernXiaoli FernThinh NguyenRaviv Raich
Scott Sanner (joining April 1)
Prasad TadepalliArash TermehchySinisa TodorovicWeng-Keen Wong
6
Step 1: Data Cleaning and Imputation
o Detect bad data values and broken sensors
o Interpolate good data values as needed
Sensors
Cleaned Data
Broken Sun
Shield
7
Step 2: Estimate State Variables
123RF Limited
o Location of each customero Location of each employeeo Total time customer in storeo # of items
Sensors
Cleaned Data
StateVariables
8
Step 3: Record trajectories
123RF Limited
o Trajectory of each customero Trajectory of each employeeo Trajectory of instrumented item
Sensors
Cleaned Data
StateVariables
Trajectories
9
Step 4: Event and Activity Recognition
123RF Limited
o Customer enters/exits storeo Employee enters/exits storeo Customer waiting at help desko Customer is looking for itemo Customer picks up itemo Customer puts item in basketo Customer leaves store with
unpaid merchandise
Sensors
Cleaned Data
StateVariables
Trajectories
Events & Activities
10
Step 5: Predictive Models
123RF Limited
o Predict customer demand per item
o Predict customer traffico Predict supplier delayso Predict wholesale and retail
price trends
Sensors
Cleaned Data
StateVariables
Trajectories
Events & Activities
Models
11
Bridging from Sensing to Action
Alert Sensor Needs Repair
Sensors
Cleaned Data
StateVariables
Trajectories
Events & Activities
123RF Limited
12
Bridging from Sensing to Action
Enter Store
DB Update:Increment Visit
Count
Sensors
Cleaned Data
StateVariables
Trajectories
Events & Activities
123RF Limited
13
Bridging from Sensing to Action
Alert: Employee to greet
customer
Alert: Employee to help
customer find item
Alert: Cashier needed Alert: Possible
shoplifting
Coupon Offer to cell phone
Sensors
Cleaned Data
StateVariables
Trajectories
Events & Activities
123RF Limited
14
Offline AnalyticsStore Layout
pinch pointshot spotsdead spots
Employee trainingconversion ratecustomer satisfaction
Inventory and staffingmissed sales because
customer could not find itemmissed sales because out
of stockpredictive inventory
managementpredictive staffingwhat predicts customer time
in store?
15
Beyond Retail:Personalized Medicine
StateVariables
Trajectories
Events & Activities
Patient state (glucose, heart rate, EKG)Current location
Trajectory travelledGlucose history
Unsteadiness, falls, breathing difficultiesheart attack, strokeRunning, walking, climbing stairs
ActionsMedical devicesSmart walkers; Exoskeletons
ModelsNormal state variables, gaitNormal events and activities
16
Smart Buildings
StateVariables
Trajectories
Events & Activities
Room occupancy, CO2 levelTemperature, humidity, lightingExterior weather
State history
Holidays, Special eventsRepair work, cold snap, heat waveEnergy price changes
ActionsHVAC automationAdjust mix of energy sourcesTime-shift predictable loads
Models Building temperature in responseto external weather, HVAC controls
17
Analytics for aResearch-to-Market IoT Center
Data Quality ControlComputer VisionModelingUser Interface and User ExperienceSecurity and PrivacySystem Executive and Control
18
Technology Needs: Data CleaningSensor diagnosis – Detect known sensor failure modesAnomaly detection – Detect novel sensor failures Imputation: learn and apply historical models to interpolate
missing or damaged dataData management
People:o Tom Diettericho Alan Ferno Weng-Keen Wongo Xiaoli Ferno Raviv Raicho Arash Termehchy
19
Technology Needs: Computer Vision Tracking
Pinch points Dead regions Hot spot detection
Event & activity recognition walking, standing, bending over,
looking down/up picking up item, setting down
item, placing item in basket, placing item in
bag/backpack/pocket waiting, browsing, searching for
something successfully finding item frustration, impatience shopping “mode” (“on a specific
mission”, “browsing”, “long list”) Face recognition Item recognition
People:o Sinisa Todorovico Alan Fern
20
Technology Needs:ModelingDiscovering Interesting Patterns in
Data Integration with External Data
Sources Web site visits Purchase History Third Party Customer Models, Social
Networks Shopping Apps
Optimizing Where and When Real-Time Analytics are Computed
Data Management Interactive Information Visualization
People:o Weng-Keen Wongo Xiaoli Ferno Ron Metoyero Eugene Zhango Scott Sanner
21
Technology Needs:User Interface/User ExperienceEmployee cuing (handheld? ear piece? smart watch?)Customer interaction via smart phone and kioskLong-term customer relationshipLong-term employee relationship
People:o Margaret Burnetto Chris Scaffidio Martin Erwig
22
Technology Needs:Cybersecurity & PrivacyEncryptionQuery-Specific Differential PrivacySoftware Quality and Testing Intrusion and Advanced Persistent Threat DetectionAnomaly Detection
People:o Rakesh Bobbao Amir Nayyerio Attila Yavuzo Mike Rosulek
o Danny Digo Alex Groceo Tom Diettericho Alan Ferno Weng-Keen Wong
23
Technology Needs:Control Executive / Integration
All components of the system must coordinate to ensureexcellent experience for the customerexcellent experience for employeesminimize costs (power, bandwidth, analysis)
Algorithms for controlOptimizationPlanningReinforcement Learning
People:o Ted Brekkeno Alan Ferno Prasad Tadepallio Tom Diettericho Thinh Nguyeno Yue Zhang
24
Oregon State AdvantagesTightly-integrated School of Electrical Engineering and Computer ScienceMany EECS collaborationsConsistent Federal funding
Strong entrepreneurial cultureCASS: Center for Applied Systems and SoftwareClose ties to College of Business via the Division of
Engineering and Business
Easy routes to collaboration with industry