image processing ip cores © evs srl, may 30, 2005
Post on 20-Dec-2015
222 views
TRANSCRIPT
Confidential © 2005 eVS Srl. 2
Cooperation
Vision, Image Processing and Sound Lab
University of Verona
Department of Computer Science
Ultimodule
Confidential © 2005 eVS Srl. 3
Research Areas
Image Processing • Image Restoration
• Feature Extraction (Edge, Corner, Region, Gradient)
Computer Vision • Mosaics and Video summarization
• Model-Based Tracking
• Stereopsis and 3D reconstruction
Pattern Recognition • Statistical techniques
• Clustering and classifications of sequences
• Object Recognition
Confidential © 2005 eVS Srl. 4
Applications
Security and Monitoring• Video surveillance
• Intrusion detection
Industrial • Active video surveillance
• Robotic arm motion
• Measurement of points and distances
Marine• Auto docking
• Support to navigation (stabilization)
• Hazard detection
• Active video surveillance
Automotive• Active security
• Autonomous guided vehicles
Confidential © 2005 eVS Srl. 5
Sensing The Real World
Stereo sensor to reconstruct the 3rd dimension
• Robustness to brightness changes on scene
• Real time performance
Output Image
• robust RGBZ• resolution up to 512x512• up to 100 fps
Confidential © 2005 eVS Srl. 6
Image Processing Engine
• The image processing engine implements most of the algorithms required when an application is asked to rely on vision to make decisions and control
• Real time performance
• High computational power
• Self-contained modules
• Compact configuration
• Common interface
• High modularity for differentcontexts
• Optical (2D) and range (3D)image support
ImageProcessing Engine
RGBZ RGBZ*
+ programmable linear and non linear filters+ warping+ edges, corners, segments and curvesdetection+ segmentation+ tracking of points through a sequence ofimages
Confidential © 2005 eVS Srl. 7
Stereopsis 1
The computational stereopsis is the process used to obtain the depth information from a pair of images coming from two cameras that see the same scene from two different points of view.
We can distinguish two sub-problems:
Corresponding points detection consists in finding which points in the left and right images are projection of the same point of the scene. The images must be rectified in order to ensure that corresponding points belong to the same horizontal line. Rectification reduces the algorithm complexity.
3D reconstruction: once the correspondences, the relative position of cameras, and the internal sensor parameters are known, the 3D position in the scene can be calculated for each point of the two 2D images.
Confidential © 2005 eVS Srl. 8
LeftLeftviewview
DisparityDisparitybrigh = closebrigh = close
Stereopsis 2
RightRightviewview
DisparitDisparityy
hot = closehot = close
Confidential © 2005 eVS Srl. 9
RGBZ sensor
Left image rectification
Right image rectification
Stereomatching
Zreconstruction
RGBZimage
Stereo Post processing
Confidential © 2005 eVS Srl. 10
Project Overview
Filtering
Segmentation
Corner
Edge
Gradient
Warping
Motionestimation
Image Processing Engine:a possible configuration
RGBZSensor Post-
processing
Confidential © 2005 eVS Srl. 11
Filtering
Noisy image (Gaussian noise σ = 0.01) Filtered image (5x5 Gaussian filter σ = 1)
Filtering
• The noise is reduced by convolving the intensity image with a gaussian kernel (linear filtering)
• The amount of filtering can be controlled by changing the coefficients of the convolution mask
Confidential © 2005 eVS Srl. 12
Segmentation
Histogram
threshold
Segmented imageInput image
• Segmentation is usually one of the first steps in image analysis
• The purpose of image segmentation is to subdivide an image into meaningful, non-overlapping regions
• Our implementation produces as a result a binary image which distinguishes between background and foreground
Confidential © 2005 eVS Srl. 13
Edge detection
EdgeDetection
Original image Edge
• An edge is a set of connected pixels that lie on the boundary between two regions
• To detect an edge we apply a threshold to the magnitude of image gradient
• Gradient is computed by convolving the image with a 3x3 Prewitt mask, and edges are thinned applying non-maximum suppression
Confidential © 2005 eVS Srl. 14
Corner detection
• Corner points are detected by a significant change of the gradient values along two directions
• The core receives in input the image gradients and the required number of corners, and outputs the coordinates and the degree of confidence ofthe extracted corners.
• Corners are easy to track
• Once the position of cornersis known along the video sequence, many informations on camera motion can be retrived.
Confidential © 2005 eVS Srl. 15
• Warping consists in applying a geometrical spatial transformation T:R2→R2 to the image coordinates, and re-sampling the grid
• Some features of our image warping module:
• affine transformation
• backward mapping
• bilinear interpolation
Warping
Confidential © 2005 eVS Srl. 16
Summary 1
SPACE Slices BRAM DSP48s
Filtering 184 2 3
Edge-detection 1195 5 0
Corner-detection 1799 14 8
Warping 422 0 4
Segmentation 858 2 4
Rectification 1312 3 0
Stereo n.a. n.a. n.a.
Performances estimation on Xilinx Virtex4 LX60 with 8 bit depth 512x512 input images
Confidential © 2005 eVS Srl. 17
Summary 2
TIME Max Freq (MHz) Pixel Rate (MHz) Frame rate (fps)
Filtering 197 49 200
Edge-detection 53 9 36
Corner-detection 42 21 86
Warping 10 26 105
Segmentation 91 91 370
Rectification 209 52 212
Stereo n.a. n.a. n.a.
Performances estimation on Xilinx Virtex4 LX60 with 8 bit depth 512x512 input images
Confidential © 2005 eVS Srl. 18
Work in progress
• Motion estimation: frame to frame geometric image transformation computation
• Video stabilization: unwanted motion compensation
• 2D Feature Tracker: correlation-based tracker of a region of interest
• Hough Transform: straight lines detection
Confidential © 2005 eVS Srl. 19
Development tools
• Matlab:• Algorithm design• Prototyping• High level verification
• Xilinx System Generator for DSP 7.1:• Hardware design• Simulation• Hardware co-simulation (via JTAG)• Initial resources estimation
• Xilinx ISE 7.1:• Implementation• Integration
• Mentor ModelSim SE 6.0:• Simulation
Confidential © 2005 eVS Srl. 20
Development boards
• 3 Xilinx Spartan3 Starter Kit Boards• 200,000 gates Xilinx Spartan3 XC3S200FT256 FPGA
• 2Mbit Xilinx XCF02S Platform Flash
• 1M-byte of Fast Asynchronous SRAM
• 1 ML310• Virtex-II Pro XC2VP30
• 256MB DDR memory
• 512MB CompactFlash card
• multiple PCI slots
• 1 SCM40• NEC VR4131 MIPS
• DDR SDRAM
• 200,000 gates Spartan2
Confidential © 2005 eVS Srl. 21
Contacts
eVS is:
• Prof. Vittorio Murino: [email protected]• Dott. Roberto Marzotto: [email protected]• Dott. Alessandro Negrente: [email protected]• Dott. Marco Monguzzi: [email protected]
• Phone / fax: +39 045 802 7027• Web: www.evsys.net