The Study and Improvement of Augmented Reality ased on Feature aching

Ruobing Yang Media School, Linyi University

Linyi, China yrb7103@sohu.com

Abstract—Augmented reality is an emerging technology of Virtual Reality , which has a great development and application prospects. Augmented Reality involves knowledge about sensors, image recognition, computer vision, human-computer interaction, virtual reality, and many other areas. The key technologies include displaying, registration and tracking, interactive etc. Among them, registration and tracking is the most important technology. In Augmented reality system, the traditional methods of registration and tracking technology has the problems of not accurate, low efficient. In this paper, improved feature matching algorithm is used in augmented reality system. Experimental results show that, compared with traditional methods, improved algorithm can significantly improve the efficiency of registration and tracking.

Keywords- Augmented Reality; Feature maching; registration; ARToolKit

I. INTRODUCTION Augmented reality (referred to as AR), also known as

mixed reality, which combine technologies of sensors, virtual reality, image recognition, human-computer interaction techniques. In recent years, augmented reality technology is widely used in medical, machinery manufacturing and maintenance, entertainment, military, heritage, landscape display and many other fields.

In Ref.1, defines AR as systems that have the following three characteristics[1]:

1) Combines real and virtual 2) Interactive in real time 3) Registered in 3-D Currently, Several popular Augmented reality system

are ARtoolkit [2], ARSTudio [2] and ARtag [3]. ARtoolkit , which developed by Dr. Kato of

Hiroshima City University, open source, is the most widely used AR development package. It has good efficiency in the implementation and tracking. But there is a fatal weakness of higher false recognition rate, because it’s using a simple template matching method.

ARSTudio is developed by Malik of Carleton University in Canada. The system introduces the process of tracking markers. When the marker was partially blocked, using of other visible corner information, you can

still determine the camera external parameters, which to some extent, enhance the robustness of the system.

ARtag is an AR library developed in recent years, which made a lot of improvements on the basis of its ARtoolkit, to solve the problems of ARtoolkit. But it is not conducive to research and extension, because the library is not open source.

In this paper, based on the results of previous studies, the traditional L-K tracking algorithm is improved and using in Augmented reality system. Experimental results show that the improved L-K algorithm can solve the traditional problems, and improve efficiency of augmented reality system.

II. AUGMENTED REALITY

A. The advantages of AR After several years of development, augmented reality

has become a very important branch of the field of Virtual Reality. Compared to the traditional VR technology, augmented reality has obvious advantages. The main advantages of augmented reality technology are as follows:

� Augmented reality has better sense of reality. Virtual reality technology is to simulate the real world in computer, giving people an immersive feel. VR technology emphasize that virtual environment is the leading role, and its realistic of the real world depends on the degree of simulation. However, augmented reality technology is organic integration of real world and the virtual environment. Therefore, augmented reality has a better sense of reality.

� Augmented reality has better interaction. As VR technology emphasize that virtual environment is the leading role, users are in a passive position in VR environment. However, augmented reality emphasize that the organic integration of virtual environment and real world, users can actively participate. Therefore, augmented reality has better interactivity.

___________________________________ 978-1-4244-9698-3/11/$26.00 ©2011 IEEE

B. The basic flow of Augmented Reality The basic principle of augmented reality is: through the

sensor, the virtual scene and the real world organic integrated, creating a realistic scene, which can greatly enhance the immersive and interaction of virtual environment [5]. Therefore, augmented reality technology has great application and development prospects.

The basic procedure of Augmented reality is: first, calculate the Affine transformation of virtual model to camera plane, according to the location of the camera and marker information in the real world. Then, draw virtual model on the basis of affine transformation matrix. Finally, organicly combinate the video of real world and virtual model, displayed on the terminal display. The concrete flow of augmented reality is divided into the following four steps [5]:

(1) Get real-world information It is through input devices such as image sensors

collect and enter to get real-world information mainly. (2)Analyze the real world and camera position

information This step is mainly through image recognition

technologies, to analyze real world and the camera position information, and get the information of virtual model’s location.

(3) Generated virtual model Generate a virtual model with the location of the

virtual model. (4)Integrate virtual model into to the video Finally, organic integrate the virtual model into the

video and display on the terminal display. The basic workflow chart of Augmented reality is

shown in Figure 1.

Figure 1. Workflow of Augmented Reality

C. The key technology of Augmented Reality Augmented Reality involves sensors, image

recognition, computer vision, human-computer interaction, virtual reality, and many other areas of knowledge. The key technologies of Augmented Reality include display, registration and tracking, interactive etc [39]. The following detailed analysis these various techniques.

� Display technology is mainly refers to the output device of augmented reality. The main function is to display the organic integration data of real world with the virtual model. The current display

technology is divided into transmission helmet display, projection displays and handheld display.

� Registration and tracking technology is based on the current real-world spatial data, using image recognition technology, get the virtual model’s correct position and orientation in the real world, so that the virtual model exactly match the real world. Registration and tracking is the core technology of augmented reality, and the key to determine the performance of augmented reality system. Achieve integration of the virtual model and real-world is on the premise of registration and tracking technology. Registration and tracking technology mainly includes computer vision, optical system, Global Positioning System, ultrasonic wave, magnetic field, etc. Currently, the technology based on computer vision is most widely used, mainly includes marker-based and natural features-based. Among them, the natural feature-based technology is not mature. Therefore, in the current augmented reality systems, the main use registration and tracking technology is marker-based.

� Good interaction is a major advantage of augmented reality. Interactive technology is the interaction between the user in real scene and interaction virtual objects. Augmented reality system, through input devices such as sensors to obtain data, technical analyze instruction sent to the virtual object by registration and tracking technology, and make corresponding feedback results. Interactive technology using space points, command and special tools and other ways to implement interaction between user and virtual objects.

D. The principles of registration and tracking Cameras capture images in essence is the use of

transformation process from real scene coordinate system to the camera coordinate system, transforming the images of three dimensional objects in real scene into two dimensional image and displaying on the camera's imaging plane. Based on the known camera parameters, determined the transfer relationship by coordinate transformation between the camera coordinate system and the real scene coordinate system, thus calculate the position and posture of camera in real scene real-time. This is the registration and tracking process of augmented reality system[7].

Figure 2. relationship between marker coordinate and camera

coordinate

The relationship between marker coordinate system and camera coordinate system is shown in Figure 2. Equation (1) shows the process of calculating the transformation matrix from mark coordinate system to camera coordinate system by image analysis. Where, (Xc, Yc, Zc) is coordinate for the camera coordinate system, (Xm, Ym, Zm) is coordinates for the marker coordinate system. Tcm is the transformation matrix coordinate transformation from the marker coordinate system to the camera coordinate system.

����

�

�

����

�

�

����

�

�

����

�

�

��

���

�

����

�

�

����

�

�

����

�

�

����

�

�

����

�

�

����

�

�

1110

11

1333

44434241

34333231

24232221

14131211

m

m

m

cmm

m

m

m

m

m

c

c

c

ZYX

TZYX

WV

ZYX

VVVVVVVVVVVVVVVV

ZYX

(1)

E. Introduce of ARToolKit ARToolKit is a set of development libraries which is

written by C/C++ language and OpenGL, based on it, we can enhance the realistic application example. ARToolKit was originally developed by Kato Hongbo Shi, and supported by the University of Washington HIT Lab (Human Interface Technology Laboratory), New Zealand, Canterbury University, HIT Lab NZ Laboratory (Human Interface Technology Laboratory New Zealand) and the company of ARToolworks [10]. ARToolKit uses computer vision techniques, achieving the registration and tracking of real world by calculating the position and direction of the camera relative to the marker. By

merging virtual objects and video of the real world video at the position of marker, ARToolKit achieve the effect of augmented reality.

III. IMPROVED L-K ALGORITHM L-K algorithm is brought by Lucas and Kanada in

1981 when they solving the optical flow tracking[7]. L-K algorithm's basic purpose is, for given two images, to calculate all the pixels in a region, finding the optimal value of similarity measure in two images. Suppose I (x) and T (x) denote the pixel gray value of the points x in two images, W (x; p) represent image transformation.

Calculation of Hessian Matrix is relative complex, will affect the overall performance of augmented reality system. So, considering changing the objective function of current image and the template, turn the Hessian matrix into definite value, which you can calculate in advance. This greatly reduces the need for calculating the Hessian matrix each iteration in the original algorithm, meet real time requirements [8].

The process of improved L-K algorithm is as follows: (1)Calculate the gradient map T� of image templates; (2)Calculate the Jacobian matrix

pW�� on (x;0);

(3)Calculate the steepest descent map pWT��

�

(4)Calculate the Hessian matrix H iteration using (4); (5)For the image I, calculate I (W (x; p)) using W (x;

p) transforms; (6)Calculate the difference image I (W (x; p))-T (x); (7)Calculate �)());(( xTpxWI

pWT

T

x��

�

���

���

�� ;

(8) Calculate Δ p using (3); (9) Update W (x; p) using (2) ; Iteration stop when ���� pT .

1);();();( ��� pxWpxWpxW (2)

�)());((1 xTpxWIpWTHp

T

x��

�

���

���

�� �� (3)

Hessian matrix H is:

��

���

���

���

���

���

�� pWT

pWTH

T

x (4)

IV. EXPERIMENTAL RESULTS AND PERFORMANCE ANALYSIS

Based on ARToolKit, in accordance with its development process, this paper achieved a basic augmented reality system. Which the module of registration and tracking using improved tracking module L-K algorithm, identifies using the ARToolKit own letter identity, as shown in Figure 3. The virtual object model, is using designed 3DS MAX model of car and dog. Augmented reality rendering subsystem shown in Figure 4.

Figure 3. Marker of ARToolKit

The experimental results show that the improved L-K algorithm is stable and accurate, and has better real-time, can meet most of the augmented reality system for the requirements of registration and tracking.

(a)

(b)

Figure 4. Effect chart of Augmented reality system

V. SUMMARY AND OUTLOOK On the basis of ARtoolkit, an improved L-K algorithm

is used to achieve registration and tracking in this paper. Compared with traditional augmented reality system, improved L-K algorithm is more accurate, more efficient. However, for the shortcoming of identify is not accurate

when meet the case that markers are blocked, this article does not take into account, which is need to solve in next step.

VI. ACKNOWLEDGMENT We thanks to ICSESS for helping us greatly improve

the paper.

REFERENCES [1] Azuma RT, “A survey of augmented reality,” In

Presence:Teleoperators and Virtual Environments, CA, vol. 6, pp. 355 385, 1997.

[2] Kato, H. and M. Billinghurst, “Marker Tracking and HMD Calibration for a Video-based Augmented Reality Conferencing System,” in 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR99). San Francisco, CA, 1999.

[3] Malik H, Roth G, Mcdonald C, “Robust 2D Tracking for Real-time Augmented Reality,” Proceedings of Vision Interface, Calgary, Alberta, Canada, 2002.

[4] B. Lucas and T. Kanade, “An iterative image registration technique with an application to stere vision,” in International Joint Conference on Artificial Intelligence, pp. 674–679, 1981.

[5] Y. Z. Fan, “Study on Augmented Reality Application”. Hangzhou, China: Zhe jiang University, 2008.

[6] Bhatnagar D K, “Position trackers for Head Mounted Display systems:A survey,” 1993, pp.1-19.

[7] S. Baker and I. Matthews, “Lucas-Kanade 20 years on: A unifying framework,” International Journal of Computer Vision, pp. 221–255, March 2004.

[8] P. Ren, “The technology of Vision-based tracking and registration in Augmented reality”. Chengdou, China: University of Electronic Science and Technology of China, 2008.

[9] M. Fiala. Artag, “An improved marker system based on artoolkit,” In National Research Council Publication NRC 47166/ERB-1111, 2004.

[10] HIT Lab ”Introduction to ARToolKit [EB/OL],” http://www. hitl.washington. edu/artoolkit/ documentation/userintro.htm 2010