presentation object recognition and tracking project
TRANSCRIPT
Autonomous Vehicle For Object Autonomous Vehicle For Object TrackingTracking
Group Members:- Prathamesh Joshi [15]Group Members:- Prathamesh Joshi [15]
Anirudh Panchal [31]Anirudh Panchal [31]
Project Guide:- Mr Kiran BhandariProject Guide:- Mr Kiran Bhandari
Goal Goal
““Build a mobile robot platform, including Build a mobile robot platform, including mechanics and electronics, implement and mechanics and electronics, implement and test a purely vision based image matching test a purely vision based image matching algorithm under real-time constraints.”algorithm under real-time constraints.”
Object matching using SIFTObject matching using SIFT
SIFT- Scale Invariant Feature TransformSIFT- Scale Invariant Feature Transform
To extract distinctive invariant features for reliable To extract distinctive invariant features for reliable matching –object recognition matching –object recognition
The features areThe features are ––invariant to scaleinvariant to scale ––invariant to rotationinvariant to rotation ––robust to (partial invariant to) affine distortionrobust to (partial invariant to) affine distortion ––robust to change in 3D viewpointrobust to change in 3D viewpoint ––robust to noiserobust to noise ––robust to change in illuminationrobust to change in illumination
SIFT SystemSIFT System
Extract features from reference images in Extract features from reference images in a databasea database
Extract features from a new given imageExtract features from a new given image
Select candidates -the new image key-Select candidates -the new image key-points (features) and match them with the points (features) and match them with the features in the databasefeatures in the database
Major Stages of Computing Major Stages of Computing Image FeaturesImage Features
1.1. Scale Space Extrema DetectionScale Space Extrema Detection
22. Key-point Localization. Key-point Localization
3.3. Orientation assignment of key-points Orientation assignment of key-points
44. Calculation of Descriptor vector of key-. Calculation of Descriptor vector of key-pointspoints
Convolving the image I(x,y) with a variable scale Convolving the image I(x,y) with a variable scale Gaussian Gaussian GG((xx,,yy,σ ).,σ ).
LL((xx, , yy,σ ) =,σ ) =GG((xx,,yy,σ )*,σ )*II((xx, , yy)………………………(1))………………………(1)
To efficiently detect stable key-points difference To efficiently detect stable key-points difference of Gaussians are calculated of Gaussians are calculated
DoGDoG((xx, , yy,σ ) = ,σ ) = LL((xx, , yy,,kkσ)− σ)− LL((xx,,yy,σ)………….. (2),σ)………….. (2)
Scale space imagesScale space images
…
first octave
…
…
second octave
…
…
third octave
…
fourth octave
…
…
Difference-of-Gaussian imagesDifference-of-Gaussian images
…
first octave
…
…
second octave
…
…
third octave
…
fourth octave
…
…
Finding extremaFinding extrema
Sample point is selected only if it is a Sample point is selected only if it is a minimum or a maximum of these pointsminimum or a maximum of these points
DoG scale spaceExtrema in this image
FilteringFiltering
For each candidate keypoint: Keypoints with low contrast are removed Responses along edges are eliminated
Orientation assignmentOrientation assignment
))),1(),1(/())1,()1,(((2tan),(
))1,()1,(()),1(),1((),( 22
yxLyxLyxLyxLayx
yxLyxLyxLyxLyxm
−−+−−+=−−++−−+=
θ
DescriptorDescriptor
Descriptor has 3 dimensions Descriptor has 3 dimensions (x,y,(x,y,θθ)) Orientation histogram of gradient magnitudesOrientation histogram of gradient magnitudes Position and orientation of each gradient Position and orientation of each gradient
sample rotated relative to keypoint orientationsample rotated relative to keypoint orientation
Recognition using SIFT features
Compute SIFT features on the input image
Match these features to the SIFT feature database
Each keypoint species 4 parameters: 2D location,scale, and orientation.
Depth measurement in real-time frames
Using Lagrange’s InterpolationUsing Lagrange’s Interpolation
Observation Table for Depth Observation Table for Depth DetectionDetection
The Lagrange’s interpolation Polynomial reduces to the following equation
F(x) = 0.0004 x2-0.2283 x+44.3429
This method has some advantages : a) Using only a single camera for the depth
finding. b) Having no direct dependency on the
camera parameters like focal length and etc. c) Having uncomplicated calculations. d) Requiring no auxiliary devices. d) Having a constant response time,
because of having a fixed amount of calculations;
f) This method can be used for both stationary and moving targets.
HARDWAREHARDWARE
The hardware consists of :-The hardware consists of :-
Wireless Video Camera.Wireless Video Camera. RS-232 cable.RS-232 cable. PCB board.PCB board. Robot Platform equiped with DC motor.Robot Platform equiped with DC motor.
Wireless Video CameraWireless Video Camera
Consists of Tx and Rx unit.Consists of Tx and Rx unit. Operates at 1.2Ghz.Operates at 1.2Ghz. Compatible with PC via TV tuner card.Compatible with PC via TV tuner card. Requires 9V for camera and 12V for Rx module.Requires 9V for camera and 12V for Rx module.
CIRCUIT DIAGRAMCIRCUIT DIAGRAM
PCB BOARD AND LAYOUTPCB BOARD AND LAYOUT
OperationOperation
Data is sent by serial module of Data is sent by serial module of Roborealm to the PCB board via RS-232.Roborealm to the PCB board via RS-232.
Data is processed by the microcontroller Data is processed by the microcontroller and control signals are given to the motors and control signals are given to the motors depending upon the input data.depending upon the input data.
Snap of VehicleSnap of Vehicle PlatformPlatform
ConclusionConclusion
Drawbacks:Drawbacks: Algorithm is computationally intensive.
False triggering.
Varying speed of the moving object.
Object may go outside the vision range of the camera.
Lack of integration between RoboRealm and Matlab.
ReferencesReferences A Moving Object Tracked by A Mobile Robot with Real-Time A Moving Object Tracked by A Mobile Robot with Real-Time
Obstacles Avoidance Capacity --Chung-Hao Chen, Chang Cheng, Obstacles Avoidance Capacity --Chung-Hao Chen, Chang Cheng, David Page, Andreas Koschan, and Mongi Abidi. David Page, Andreas Koschan, and Mongi Abidi.
A Vision System for IIT Kanpur Mirosot Robot Soccer Team CS497 A Vision System for IIT Kanpur Mirosot Robot Soccer Team CS497 Special Topics in Computer Science Semester II, 2002-03 Manu Special Topics in Computer Science Semester II, 2002-03 Manu Chhabra-99211.Chhabra-99211.
AA New Method for Depth Detection Using Interpolation Functions New Method for Depth Detection Using Interpolation Functions Mahdi Mirzabaki Azad University of Tabriz, Computer Engineering Mahdi Mirzabaki Azad University of Tabriz, Computer Engineering DepartementDepartement
DepthFinder, A Real-time Depth Detection System for Aided DepthFinder, A Real-time Depth Detection System for Aided DrivingDriving
Yi Lu Murphey, Jie Chen1, Jacob Crossman, Jianxin Zhang, Paul Yi Lu Murphey, Jie Chen1, Jacob Crossman, Jianxin Zhang, Paul Richardson and Larry. Richardson and Larry.
REFERENCESREFERENCES
A Real-time Image Recognition System for Tiny A Real-time Image Recognition System for Tiny Autonomous Mobile RobotsAutonomous Mobile Robots
Stefan Mahlknecht ,Roland Oberhammer ,Gregor Novak.Stefan Mahlknecht ,Roland Oberhammer ,Gregor Novak. Multipurpose Control System and Mobile Robot Multipurpose Control System and Mobile Robot
Development, for Control Development, for Control Motion Detection and Object Tracking in Image Motion Detection and Object Tracking in Image
Sequences---Zoran ˇZivkovi´.Sequences---Zoran ˇZivkovi´. Object Recognition from Local Scale-Invariant Object Recognition from Local Scale-Invariant
Features--David G. Lowe.Features--David G. Lowe. An Image Identification Algorithm using Scale Invariant An Image Identification Algorithm using Scale Invariant
Feature Detection Feature Detection
THANK YOUTHANK YOU