development of a system to reproduce the drainage from tsujun bridge for environment education...
TRANSCRIPT
Development of a system to reproduce the drainage from Tsujun Bridge
for environment education
Hikari TakeharaKumamoto National College of Technology
Background
• Tsujun Bridge– an aqueduct bridge made of stones in Kumamoto, Japan– a representative structure in water environment
surrounding Shiraito plateau– This drains water to remove sediment in its water pipe
The drainage from Tsujun Bridge
– The drainage from Tsujun Bridge is utilized for education of water environment in Kumamoto
Problem and Situation
• Problem– Tsujun Bridge has been given damages whenever it drains
water– The reason Leaking water from Tsujun Bridge
• Situation– A number of the drainage from Tsujun Bridge has
decreased for protecting Tsujun Bridge– We cannot always watch the drainage from Tsujun Bridge
Purpose
• Creating a contents for learning the water environment surrounding Shiraito plateau– at any time– without giving any damages to Tsujun Bridge
• We developed a system which reproduces the drainage scene from Tsujun Bridge using Mixed Reality (MR) technology
Mixed Reality (MR)
• MR technology– Synthesize image of real environment (actually visible
landscape) and image of virtual environment (Computer Graphics: CG)
– Making the image as if there is a teapot on table
image of real environment
image of virtual environment
synthesized image
Overview of system
• Capturing the camera image• Geometric registration• Synthesizing camera image and 3DCG model
Program flow
Camera Calibration
Capturing camera image
Geometric Registration
Synthesizing the CG and camera image
Once
Each frame
Program flow
Camera Calibration
Capturing camera image
Geometric Registration
Synthesizing the CG and camera image
Once
Each frame
Camera Calibration
• Intrinsic parameters which express how a camera projects a real space (object) on the projection plane are obtained
object
projection plane
Camera’sfocus
Camera Calibration
•
Cameracoordinate
Imagecoordinate
100
0
0
0
0
vfk
ufk
v
u
A
Intrinsic parametersmatrix A
Program flow
Camera Calibration
Capturing camera image
Geometric Registration
Synthesizing the CG and camera image
At once
Each frame
intrinsic parameters
Capturing the camera image
• Capturing the camera image was used with OpenCV
• OpenCV(Open source Computer Vision library)– Library for Computer Vision
Program flow
Camera Calibration
Capturing camera image
Geometric Registration
Synthesizing the CG and camera image
At once
Each frame
intrinsic parameters
camera image
Geometric registration
• Geometric registration– to relate the position and orientation of camera with the coordinate
system (world coordinate system) which is set at will in real environment
– This relation is expressed by extrinsic parameters
Worldcoordinate
C
Cameracoordinate
M
XW
YW
ZW
Projection plane
XC
YC
ZC
Extrinsic parameters
• Extrinsic parameters– expressed as 3x4 matrix using rotation matrix R and translation vector T– This study assumed that the position of camera is fixed and that the
orientation of camera can be changed freely– So, T is not need to be obtained, and R is only need to be obtained
Extrinsic parametersmatrix M
Extrinsic parameters
• Extrinsic parameters– R matrix is obtained using Zhang’s method with homography matrix
• Homography matrix H– transforms an image of a plane taken from one perspective into an
image taken from the other perspective– expressed as 3x3 matrix
XC
YC
ZCXC’
YC’
ZC’
vH
u’
v’
A plane
u
Homography matrix
• Homography matrix– Obtaining this matrix needs more than 4 corresponding
points between 2 images from each other perspectives
– As the method of obtaining corresponding points, the method based on natural features is adopted
– As the method of extracting natural features, SURF is used
Program flow
Camera Calibration
Capturing camera image
Geometric Registration
Synthesizing the CG and camera image
At once
Each frame
intrinsic parameters
camera image
extrinsic parameters
Synthesizing the CG and camera image
• Camera image and 3DCG model of drainage were synthesized on the basis of the intrinsic and extrinsic parameters using OpenGL
• OpenGL (Open source Graphics Library)– Library for 3D graphics
Intrinsicparameters
Extrinsicparameters
3DCG model of drainage
Camera image
Synthesized image
Result of reproducing drainage scene
Result
• The drainage scene was reproduced to the correct position at the most part.
• An average of execution time– 0.247 [s]
• An average of Estimation error– 6.758 [pixel]
Conclusion
• In the future– The execution speed of this system will be improve using
other method of obtaining corresponding feature points such as the optical flow
– This system will be available even if the position of camera will be freely
– This system will be operated in actual Tsujun Bridge, and will be utilized for learning water environment
End
SURF
• SURF (Speeded Up Robust Features)– The method extracts robust feature points for scaling and
rotation of image and describes feature quantity– In following image, the corresponding points is obtained
using SURF