scanning 3 d full human bodies using kinects
TRANSCRIPT
![Page 1: Scanning 3 d full human bodies using kinects](https://reader033.vdocuments.us/reader033/viewer/2022060204/55a05d081a28ab09688b4883/html5/thumbnails/1.jpg)
By,
FensaMerry Saj
LBSITW
![Page 2: Scanning 3 d full human bodies using kinects](https://reader033.vdocuments.us/reader033/viewer/2022060204/55a05d081a28ab09688b4883/html5/thumbnails/2.jpg)
OVERVIEW Introduction
RelatedWork
System Setup
Reconstruction Approach
Results
Conclusion
References
10/5/2013 2
![Page 3: Scanning 3 d full human bodies using kinects](https://reader033.vdocuments.us/reader033/viewer/2022060204/55a05d081a28ab09688b4883/html5/thumbnails/3.jpg)
INTRODUCTION Many computer graphics applications require realistic3D models of human bodies.
Depth cameras such as Microsoft Kinects are able tocapture depth and image data at video rate.
Kinect is compact, low-price and as easy to use as avideo camera.
10/5/2013 3
![Page 4: Scanning 3 d full human bodies using kinects](https://reader033.vdocuments.us/reader033/viewer/2022060204/55a05d081a28ab09688b4883/html5/thumbnails/4.jpg)
10/5/2013 4
![Page 5: Scanning 3 d full human bodies using kinects](https://reader033.vdocuments.us/reader033/viewer/2022060204/55a05d081a28ab09688b4883/html5/thumbnails/5.jpg)
RELATED WORK
Scanning devices based on structured light or laserscan can capture human body with much high quality,but is very expensive (about $240,000).
Two main approaches employed in depth cameratechnology are:-
based on the time-of-flight principle, measuringtime delay between transmissions of a light pulse(about$8,000).
based on light coding; projecting a known infraredpattern onto the scene and determining depth based onthe pattern’s deformation.
10/5/2013 5
![Page 6: Scanning 3 d full human bodies using kinects](https://reader033.vdocuments.us/reader033/viewer/2022060204/55a05d081a28ab09688b4883/html5/thumbnails/6.jpg)
RELATED WORK(contd…)A most popular one based on light coding is the MicrosoftKinect Sensor which is at a price of only $150.
3 Main Types:-
Registration without a template.
This method requires high quality scan data & needs smallchanges in temporal coherence.
Registration with a template.
This method needs a relative accurate template & then usesthe template to fit each scan.
10/5/2013 6
![Page 7: Scanning 3 d full human bodies using kinects](https://reader033.vdocuments.us/reader033/viewer/2022060204/55a05d081a28ab09688b4883/html5/thumbnails/7.jpg)
RELATED WORK(contd…)Registration with a semi-template.
Rough template, such as the skeleton model ofarticulated object, can be utilized.
The first type requires high quality input data & iscomputationally expensive; the second one needs anaccurate template which is hard to fulfil in manyapplications.
Here, this system uses the third type.
10/5/2013 7
![Page 8: Scanning 3 d full human bodies using kinects](https://reader033.vdocuments.us/reader033/viewer/2022060204/55a05d081a28ab09688b4883/html5/thumbnails/8.jpg)
RELATED WORK( contd…)
The idea of creating a graph of pairwise alignmentsbetween scans.
First, pairwise rigid alignment is computed in thegeometric level.
Global error distribution then operates on an upper level,where errors are measured in terms of the relative rotationsand translations of pairwise alignments.
The graph methods can simultaneously minimize theerrors of all views rapidly and do not need all scan inmemory.
10/5/2013 8
![Page 9: Scanning 3 d full human bodies using kinects](https://reader033.vdocuments.us/reader033/viewer/2022060204/55a05d081a28ab09688b4883/html5/thumbnails/9.jpg)
SYSTEM SETUP Two kinects are used to capture the upper and thelower part of a human body respectively, withoutoverlapping region, from one direction.
A third kinect is used to capture the middle part of thehuman body from the opposite direction.
The distance between two sets of Kinects is about 2meters.
A turntable is put in between them.
10/5/2013 9
![Page 10: Scanning 3 d full human bodies using kinects](https://reader033.vdocuments.us/reader033/viewer/2022060204/55a05d081a28ab09688b4883/html5/thumbnails/10.jpg)
The setup of our system
10/5/2013 10
![Page 11: Scanning 3 d full human bodies using kinects](https://reader033.vdocuments.us/reader033/viewer/2022060204/55a05d081a28ab09688b4883/html5/thumbnails/11.jpg)
RECONSTRUCTION APPROACH
Denote Di={Mi,Ii},i=1,….n as the captured data, n is thenumber of captured frames, Mi is the merged mesh & Ii isthe corresponding image of the i-th frame respectively.
First, a rough template is constructed.
The template is used to deform the geometry of successiveframes pair wisely.
Global registration is performed to distribute errors in thedeformation space.
Finally, reconstructed model is generated using Poissonreconstruction method.
10/5/2013 11
![Page 12: Scanning 3 d full human bodies using kinects](https://reader033.vdocuments.us/reader033/viewer/2022060204/55a05d081a28ab09688b4883/html5/thumbnails/12.jpg)
An accurate template is unavailable.
We construct an estimated body shape as the templatemesh T1 from the first frame.
It is impossible to use this template to register eachframe by geometry fitting but it can track the pairwisedeformation of successive frames.
T1={v1^k,k=1…K; K is the number of nodes of T1(typically 50-60).
10/5/2013 12
![Page 13: Scanning 3 d full human bodies using kinects](https://reader033.vdocuments.us/reader033/viewer/2022060204/55a05d081a28ab09688b4883/html5/thumbnails/13.jpg)
Suppose Mi, i=1…n forming a cycle. fi,j denotes theregistration that can deform mesh Mi to register withmesh Mj.
To find the pairwise registration f1,2,f2,3,…fn-1,n,fn,1.
Deformation Model:
Suppose we have two meshes Mi & Mj, and templatemesh at frame i is Ti, then
10/5/2013 13
![Page 14: Scanning 3 d full human bodies using kinects](https://reader033.vdocuments.us/reader033/viewer/2022060204/55a05d081a28ab09688b4883/html5/thumbnails/14.jpg)
Pairwise registration:
For successive frames Mi and Mi+1,corresponding featurepoints are obtained by optical flow in the correspondingimages.
Projection to the first frame:
n-1 pair wise deformation is required to recover all therelative position of all frames.(refer fig.6)
The desired pairwise deformation f̂1,2,f̂2,3,…f̂n-1,n,f̂n,1should meet the following conditions:
1.It is cyclic consistent.
2.The original pairwise deformation is relatively correct, sominimize the weighted square error of the new and olddeformation.
10/5/2013 14
![Page 15: Scanning 3 d full human bodies using kinects](https://reader033.vdocuments.us/reader033/viewer/2022060204/55a05d081a28ab09688b4883/html5/thumbnails/15.jpg)
Overview of our reconstruction algorithm
10/5/2013 15
![Page 16: Scanning 3 d full human bodies using kinects](https://reader033.vdocuments.us/reader033/viewer/2022060204/55a05d081a28ab09688b4883/html5/thumbnails/16.jpg)
10/5/2013 16
![Page 17: Scanning 3 d full human bodies using kinects](https://reader033.vdocuments.us/reader033/viewer/2022060204/55a05d081a28ab09688b4883/html5/thumbnails/17.jpg)
Different 3D full human models generated by the system
10/5/2013 17
![Page 18: Scanning 3 d full human bodies using kinects](https://reader033.vdocuments.us/reader033/viewer/2022060204/55a05d081a28ab09688b4883/html5/thumbnails/18.jpg)
RESULTS Global non-rigid registration gives better result thanglobal rigid alignment.
Since the color image and depth information arecaptured simultaneously and calibrated, the colorinformation of deformed mesh is generatedautomatically.
Virtual try on.
Personalized avatar-video games,online shopping ,human computer interaction etc.
10/5/2013 18
![Page 19: Scanning 3 d full human bodies using kinects](https://reader033.vdocuments.us/reader033/viewer/2022060204/55a05d081a28ab09688b4883/html5/thumbnails/19.jpg)
Realistic virtual try on experience based on the reconstructed model.(Left)the try on results;(right)the corresponding meshes.
10/5/2013 19
![Page 20: Scanning 3 d full human bodies using kinects](https://reader033.vdocuments.us/reader033/viewer/2022060204/55a05d081a28ab09688b4883/html5/thumbnails/20.jpg)
Personalized avatar generated by our system.The motion of the human body is driven by a given skeleton motion sequence
10/5/2013 20
![Page 21: Scanning 3 d full human bodies using kinects](https://reader033.vdocuments.us/reader033/viewer/2022060204/55a05d081a28ab09688b4883/html5/thumbnails/21.jpg)
CONCLUSION The proposed method can deal with non-rigidalignment and complex occlusions.
The two stage registration algorithm is efficient and ofmemory efficiency.
The system can generate convincing 3D human bodiesat a much low price.
It has good potential for home oriented VRapplications.
10/5/2013 21
![Page 22: Scanning 3 d full human bodies using kinects](https://reader033.vdocuments.us/reader033/viewer/2022060204/55a05d081a28ab09688b4883/html5/thumbnails/22.jpg)
REFERENCES [1]Jing Tong; Jin Zhou; Ligang Liu; Zhigeng Pan; HaoYan, ”Scanning 3D Full Human Bodies UsingKinects”,Visualization and Computer Graphics, IEEETransactions on, vol.18, no.4, April 2012.
[2]Srivishnu Satyavolu,Gerd Bruder,PeteWillemsen,Frank Stenicke,”Analysis of IR-basedvirtual reality tracking using multiple Kinects”,2012IEEE Virtual Reality,2012.
10/5/2013 22
![Page 23: Scanning 3 d full human bodies using kinects](https://reader033.vdocuments.us/reader033/viewer/2022060204/55a05d081a28ab09688b4883/html5/thumbnails/23.jpg)
10/5/2013 23
![Page 24: Scanning 3 d full human bodies using kinects](https://reader033.vdocuments.us/reader033/viewer/2022060204/55a05d081a28ab09688b4883/html5/thumbnails/24.jpg)
10/5/2013 24