real-time human posture reconstruction in wireless smart camera networks
DESCRIPTION
Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks. Chen Wu, Hamid Aghajan Wireless Sensor Network Lab, Stanford University , USA IPSN 2008. Speaker Lawrence. Outline. Background Motivation Goal Challenge Strategy for Camera Sensor Network System Overview - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/1.jpg)
Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks
Chen Wu, Hamid Aghajan Wireless Sensor Network Lab, Stanford University, USA
IPSN 2008
Speaker Lawrence
![Page 2: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/2.jpg)
Outline
• Background• Motivation• Goal• Challenge• Strategy for Camera Sensor Network• System Overview• Wireless Smart Camera (Hardware)• Human Pose Estimation (Algorithm)• Result• Conclusion
![Page 3: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/3.jpg)
Background
• Traditional Camera network for surveillance & security
• New applications of camera network for multimedia, video conference…etc
• Wireless Camera network – Scalability– Privacy preservation– Flexibility on collaboration scheme between cameras
![Page 4: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/4.jpg)
Motivation
• As pervasive sensors the cameras can free the users from wearable devices.
• Lack of real-time vision algorithm to achieve moderate complexity, robustness and scalability.
![Page 5: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/5.jpg)
Goal
• Implementation of human pose interpretation on a wireless smart camera network.
• Employing distributed processing– Real-time vision & scalability for complex vision algorithms.
![Page 6: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/6.jpg)
Challenge
• A vision sensor network poses three key challenges:
– High computation capacity for real-time performance.
– Wireless links limit image transmission (bandwidth & energy)
– Lack of established vision-based fusion mechanisms (by real time)
![Page 7: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/7.jpg)
Strategy for Camera SN
• Difference between Camera network & Distributed vision processing strategy systems.– Employ cameras as a wireless sensor network.
• Strategy:1. Video data reducing (Network bandwidth)2. Level of vision analysis to different PHY processors
![Page 8: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/8.jpg)
Strategy for Camera SN (cont.)
Central PC
SmartCamera
Level of vision analysis to different PHY processors
![Page 9: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/9.jpg)
Scalability : Spatial and functional parallelism
• Each camera video processes its own data(spatial)• Running their own function modules(functional)
Strategy for Camera SN (cont.)
![Page 10: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/10.jpg)
Smart camera communicate with the central PC through ZigBee
System Overview
LCD display
Smart camera
Different ZigBee channels
![Page 11: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/11.jpg)
Data flow in the system
System Overview (cont.)
Semaphore tech for DPRAM
P.S. DPRAM allows multiple r or w to occur at the same time.
Asynchronous
![Page 12: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/12.jpg)
Wireless Smart Camera
• Hardware Platform– VGA color image
sensor– SIMD
processor(IC3D)– Embedded
processor(8051)– ZigBee platform
![Page 13: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/13.jpg)
• Parallel arch power consumption
• LPA(320 PEs) data processing
• GCP control IC3D & DSP operations
• PE # video format, e.g., VGA(640*480)
• The main design factors of SIMD frequency & PE #
Wireless Smart Camera (cont.)
MP-SIMD
![Page 14: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/14.jpg)
• Data sharing between processors– PDRAM functions as an asynchronous connection
between IC3D and 8051– Semaphore tech to prevent mutual access
• Wireless communication– P2P structure offers direct camera to PC communication– Maximum data rate : 100 Kbit/sec
Wireless Smart Camera (cont.)
![Page 15: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/15.jpg)
• Review (Algorithm)– Goal : 2D to 3D– Ambiguity: Perspective views of the camera or self-
occlusion of human body
• Pose Estimation Approach– Top-down– Bottom-up
Human Pose Estimation
![Page 16: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/16.jpg)
• Top-down
Human Pose Estimation (cont.)
![Page 17: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/17.jpg)
• Bottom-up
Human Pose Estimation (cont.)
![Page 18: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/18.jpg)
Top-down vs Bottom-up• Top-down
– Strength• Occlusion handling• Contours & body
part association
– Weakness• Search tech
complexity(depth)• Computational
load(projection)
• Bottom-up– Strength
• Much less demands on 3D switch
– Weakness• Complex assemble• Difficult to detect
occlusions
Human Pose Estimation (cont.)
![Page 19: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/19.jpg)
• Challenges & Method– Bandwidth constraint
(100Kbits/sec)/(30frames/sec)/(8bits/Byte) ≈ 400B/frame solution: Detect body part cancroids coordinates– Limited image processing capability of the SIMD
processorsolution: Color segmentation
– Robustness with varied environment solution: Auto-balancing
filtering & combination
Human Pose Estimation (cont.)
![Page 20: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/20.jpg)
In-node processing
• Detect positions(x, y):– Head, shoulders and hands– 2Bytes for x and y
• Detect mechanism:– Face -> face color model– Head -> skin color model– Shoulders -> shirt color model (low-pass filter)
Human Pose Estimation (cont.)
![Page 21: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/21.jpg)
Human Pose Estimation (cont.)
The image processing program on IC3D
![Page 22: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/22.jpg)
Human Pose Estimation (cont.)
![Page 23: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/23.jpg)
• Processing on the central PC
– Noise filtering and 2D to 3D reconstruction
Human Pose Estimation (cont.)
![Page 24: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/24.jpg)
• Demo: Virtual ball-playing game
– Demo Video
Results
![Page 25: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/25.jpg)
Standard Deviation of detected body part coordinates in the smart cameras (in pixels) and those after noise filtering
Results (cont.)
Demo
![Page 26: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/26.jpg)
Original data from the smart cameras and data after noise filtering
Head
Left shoulder Right shoulder
Results (cont.)
![Page 27: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/27.jpg)
Results (cont.)
Left hand Right hand
![Page 28: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/28.jpg)
Conclusion
• Propose an algorithmic strategy to approach vision problems in a wireless camera sensor network
• Major aspect of the strategy:– reduce video data locally through smart camera
• Implement a prototype system of 3D human reconstruction using a wireless smart camera.
• Wireless camera networks will offer potentials for user-centric applications.
![Page 29: Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks](https://reader035.vdocuments.us/reader035/viewer/2022062315/568163e1550346895dd53c46/html5/thumbnails/29.jpg)
Thanks for listening