iit kgp workshop
TRANSCRIPT
Experience certainty.
Copyright © 2011 Tata Consultancy Services Limited
Human-in-loop Cyber-physical Systems- Intelligent Infrastructure using Internet-of-Things
Arpan PalHead of ResearchInnovation Lab, Kolkata
Research Theme
3Experience certainty.
Human-in-Loop Cyber-physical Systems
Humans
Physical
Objects and
Infrastructure
Computing
Infrastructure
Physical
Context
Discovery
4Experience certainty.
Signal
Processing
Internet-of-Things - towards Intelligent Infrastructure
Sense
Extract
Analyze
Respond
Learn
Monitor
Intelligent
Infra@Home
@Building
@Vehicle@Utility
@Mobile
@Store
@Road
“Intelligent” (Cyber) “Infrastructure” (Physical)
APPLICATION SERVICES
BACK-END PLATFORM
INTERNET
GATEWAY
Sense
Extract
Analyze
Respond
Communication
Computing
5
Challenges and Solution Approach
Systems• Making Sensing Unobtrusive and Pervasive• Sensing the Human Context
Cloud-based Sensor Data Analytics Platform Use of Mobile phones and Cameras as sensing devices New Sensors for human context – EEG, 5 senses computing
Signal Processing• Preserving the Battery power of edge devices• Extracting information from ambient-noise-corrupted sensor data
Low-power computing algorithms Adaptive Signal Processing for Ambient Noise Removal
Communication• Reducing the cost of Communication while preserving reliability and security• Preserving the Privacy
Secure, Lightweight yet Reliable communication protocols over Internet Quantifying Privacy vs. Utility measures
Computing• Knowledge and Model Driven Analytics in Real-time
Distributed Computing - Bringing Edge into the Grid
Semantics driven Analytics - Reasoning and Semantic Sensor Web
Model driven Analytics - Cyber-physical system modeling / Human behavior modeling
6
Integrated Platform for Intelligent Infrastructure
People Feedback & Emotions
Social Media
Integrated Services
Sensors & IoTPlatform
Traditional Monitoring & Control Systems
Citizen Data
Smart Integration Platform
Transportation Healthcare Electricity
WaterPublic Safety Tourism
Smart Integrated Services
Sense
Analyze
Extract
Respond
Intelligence
Smart Domain Services
Community
etc.
Sense: People Activity, Appliances, Vehicles , Road, Home/Bldg, Utility Infrastructure
Detect gas leakage/water contamination : mobilize rescue team, suggest optimum route
Divert Road Traffic in case of Water Pipeline Burst
Correlate Electricity/Water /Gas consumption patterns
Intelligent Integration Platform
Integrated Intelligent Services
RIPSAC – Real-time Integrated Platform Services & Analytics for Cyber-physical Systems
7
Platform Architectural Layers
Internet
End Users &
Renderers
Administrators
Device Integration & Management Services
Analytics Services
Application Services
Storage
Messaging & Event Distribution Services
Ap
plic
atio
n S
erv
ice
s
Presentation Services
Application Support Services
OS & Device Drivers
Sensor Signal Processing
Middleware
Mid
dle
ware
(S
ecurity
/Privacy a
nd P
roto
cols
)
Edge Gateway
Sensors
Internet
Back-end
8Experience certainty.
Cyber Physical System – Research Theme
RIPSAC
Infrastructure
Simulation
New Sensing
Platforms,
5 Senses
Computing
Energy
Water
Transportation
Security
& Privacy
Distributed
Computing
Facilities
Data Integration
& Visualization
Safety
Buildings
HomeCity
Vehicles
Non revenue
water Long term network
expansion
Renewable
sources
Energy
conservations
Intelligent
Infrastructures
Sensor Informatics
for Home
Sensor Informatics for
Vehicular Monitoring
Transport planning
Electricity Grid
Optimization
algorithms
Big data
Communications &
Sensor Informatics
9Experience certainty.
Human-in-Loop Cyber-physical System – Research Theme
RIPSAC
2D/3d Surveillance -
Kinect
Physiological
Sensing –
ECG, EEG, …
Physiology
Identity
Smartphone
Sensors –
GPS,
Accelerometer,
Magentometer,
Gyropscope,
Camera, Mic
Healthcare• Elderly People Monitoring• Chronic Disease People
Monitoring
Personalized learning• Cognitive Load
Web and
Social Media
Smart
Meters
Wellness• Fitness• Stress, Fatigue
Organizational Behavior• Personal Physical Network
Discovery• Team Efficiency / Best Practice
Analysis
Retail• Customer
Behavior Study
Driving• Stress• Fatigue
Research Projects – Ubiquitous Healthcare
11
Intelligent Healthcare – Elderly / Chronic Patient Monitoring
ECG
Blood PressureMonitor
Pulse OxyMeter
Mobile phone as medical
gateway
PatientRecords
Health Center / Home
Expert Doctor
• 2 papers in SMART 2012 and MobiHoc 2012
• Requirement from NUH (Peritonial Dialysis) and
SingHealth (Elderly Care)
12
Intelligent Healthcare – Elderly / Chronic Patient Monitoring
Mobile phone as medical
Sensor
PatientRecords
Health Center / Home
Expert Doctor
• Replace Medical Sensors with Mobile Phone Sensing
• Activity and Localization using Mobile Phones
• Activity and Identification using Kinect Camera
• Social Media as a Soft sensor
• Multimodal Fusion
Identification, Localization and
Activity Detection
• 1 paper in UbiComp 2013
13
PPG based Pulse Measurement using Phone Camera
Subject1 Subject2 Subject3
Actual Detected Actual Detected Actual Detected
68 66 66 63 85 84
2.9% 4.5% 1.1%
Subject Actual Pulse
Rate
Detected Pulse Rate (20 sec
skip)
Detected Pulse Rate (automatic
detection)
PSNR (automatic
detection)
A 54 53.542 53.542 22.887
B 66 63.404 63.404 15.351
C 84 84.539 84.539 19.831
D 106 98.629 98.629 21.741
E 80 88.062 88.062 14.788
F 105 102.15 102.15 16.705
G 105 38.747 0 0
H 80 42.27 0 0
• 1 paper in Mobihoc 2013
• 1 patent filed TCS Fit4LIfe Pilot
14
Blood Pressure and ECG Monitoring from PPG
Extract PPG
Features
BP Ground
TruthCreate BP Model
Extract PPG
Features
Predict BP levels
(SP, DP, PP)
Extract ECG
Features
Create ECG
Parameters’ Model
Predict ECG
ParametersECG Parameters
BP levels
(SP, DP, PP)
Training Phase
Testing Phase
Data set Pd Ps PP-diff < 15
Standard dataset with 14 PPG features 92.9% 74.7% 77.9%
TCS recorded dataset with 14 PPG features
+ height, weight, age
99.3% 82.7% 85.5%
BP Level Pd Ps
Very Low < 50 < 70
Low 50-65 70-100
Normal 65-90 100-
135
High 90-100 135-
160
Very High > 100 >160
15
Mobile Phone based Activity Detection and Localization
Activity Detection– Uses Accelerometer Data
– Gyroscope and Magnetometer for orientation
correction
– Step Count, Stride Length Estimation
– Walking, Brisk Walking, Running Classification
Indoor Localization– Initial Referencing through GPS and
Magnetometer
– Inertial Navigation through accelerometer,
gyro, magnetometer
– Improved accuracy through Stride
Length, Kalman Filter / Particle Filter
– Tracking of non-smartphones via
Bluetooth
Location ID LOC I LOC II LOC III
Actual (ft) 2 4 6
Estimated
(ft)1.95 4.16 6.27
% Error 2.5 4.12 4.4
• 1 paper in UbiComp 2013
• 3 patents filedActivity Detection for TCS Fit4Life
Generalized indoor localization – Colleague Finder,
Retail, Emergency Evacuation
16
Kinect Based Identification and Activity Detection
Human Identification
– Skeleton Model Based / Depth based
– External Stimulus based refinement
– Accuracy of 90% for less than 10 people
database
• 2D Camera with IR
depth sensor
• Excitation by IR light
pattern
• Directional Mic.
Activity Detection
– Enter/leave a room
– Sitting / Standing / Walking / Lying down
– Going to Bathroom
– Taking MedicineLIRIS human activities dataset
• 4 papers in Fuzz IEEE 2013 , CEC 2013, UbiComp 2013, SMC 2013
• 2 patents filed
17
Social Media as a Soft-sensor for Healthcare
Support Community Discovery
– Use hidden community detection by applying NLP on
posts to create the social graph to identify the
undeclared community for a given disease
Disease onset discovery like dementia or Alzheimer's
disease or psychological disorders from social
network posts
– Search for patterns in posts to detect possible
symptoms to diseases
– E.g. - sentiment analysis on posts will give whether the
given post’s emotion is positive or negative. If the
emotions are cycling between positive and negative
extremes with some periodicity, probably the person
has bipolar disorder
• 1 paper in UbiComp 2012
Research Projects – New Sensing Techniques
19Experience certainty.
Bio-sensing for Cognitive Load
User Study
Biased and Indirect
Biological Response using EEG
Unbiased & more reliable method
• 2 papers in Fuzz IEEE 2013 , SMC 2013
• 2 patents filed
• Can be used for Usability Testing of GUI
20Experience certainty.
5 senses Computing - 3D Reconstruction with 2D images from mobiles
• Low cost solution for 3D reconstruction from multiple 2D images captured from
mobile device.
• Derive the motion information from the inbuilt sensors of the mobile phone and then
aid in increasing the accuracy of the 3D reconstruction.
• Dynamic 3D reconstruction in outdoor environment which is not possible using a
Kinect.
• 3D reconstruction of homogeneous objects
• Future research focus on the multi-modal fusion of the 3D information and the other
sensors’ data for a given object or environment
Applications
• Agro-advisory Service
• Remote Diagnostics of Machines
• Remote Healthcare
• 1 paper in VISAPP 2012
Research Projects – Communication and
Computing
22Experience certainty.
Grid with an Edge
• CPU utilization better in Condor
• Condor performs better with more nodes
• Further advantages w.r.t Heterogeneity, Versatility, Matchmaking & scheduling
• 3 papers in NGMAST 2012, PDPTA 2013, WiMob 2013
• 2 patents filed
• Flavio Bonomi et.al. MCC2012, Helsinki, Finland
23Experience certainty.
Communication Aspect- Replace HTTP
Constrained Object Access Protocol (CoAP)
Improved version of CoAP using dynamic network condition sensing
Lightweight security protocols sensor authentication and data delivery
Use suitable lightweight application protocol between edge devices and core network
• 3 papers in ICDCN 2013, ICNC 2013, WPMC 2013
• 2 patents filed
24Experience certainty.
Knowledge-driven Analytics
Ontology and Description for Sensors
• Non-standard / Multi-standard sensor descriptions across multi-domains
• Bringing all sensor data in common semantic form
• Semantic Sensor Web (OGC-SWE)
• Will provide ease of sensor discovery
Ontology and Knowledge based Reasoning
• Reasoning over continuously flowing (streaming) facts
• Reasoning over streaming data is integrated with “background knowledge” (static unchanging or slowly changing knowledge
• Spatio-temporal Queries
Semantic Analytics
• Analytics-as-a-Service
• Query language to specify processing workflow on Time series data
• 5 papers in ICST 2012 , ICNC 2013, ICUIMC 2013, ODBASE 2013, KES 2013
• 4 patents filed
25Experience certainty.
Privacy Preservation
• Information-theoretic approach for sensitivity analysis of sensor data
• Requirement based application utility measurement
• Balancing of Privacy vs. Utility
Privacy
Utility
• 1 paper in SecurIT 2012
• 2 patents filed
Privacy
Preservation
Tool
26
Lab-at-a-Glance
Research Areas• Sensor Signal Processing• 2D/ 3D Image / Video Processing • Protocols, Security and Privacy• Parallel and Distributed Computing• Stream Processing and Reasoning• System Modeling and Identification• Sematic Sensor Web• Social Media Analytics
Academic Collaborations• Singapore Management University (iCity Platform)• Indian Statistical Institute (Protocol/Privacy/Security,
Image / Video Processing)• IIT Kharagpur (Analytics, Semantic Processing)• IIT Bombay (Energy and Utilities)• Jadavpur University (Signal Processing)
Higher Studies• PhD - 4• Masters - 4
Total Researchers - 37
Thank You