hi_lite scott fukuda chad kawakami. background ► the darpa grand challenge ► the defense advance...

Hi_LiteHi_Lite

Scott FukudaScott Fukuda

Chad KawakamiChad Kawakami

BackgroundBackground

► The DARPA Grand ChallengeThe DARPA Grand Challenge► The Defense Advance Research Project The Defense Advance Research Project

Agency (DARPA) established a contest in Agency (DARPA) established a contest in 2004 to set a new level of accomplishment 2004 to set a new level of accomplishment for autonomous land vehicles. This was in for autonomous land vehicles. This was in response to a goal set by Congress that by response to a goal set by Congress that by 2015 twenty percent of U.S. military vehicles 2015 twenty percent of U.S. military vehicles on a battlefield would be unmanned. In The on a battlefield would be unmanned. In The Grand Challenge, contestants vied for a one Grand Challenge, contestants vied for a one million dollar prize by racing for 150 miles.million dollar prize by racing for 150 miles.

HI_Lite VehicleHI_Lite Vehicle

►Modified from a Columbia Par Car Modified from a Columbia Par Car electric scout vehicleelectric scout vehicle

►Akamai Research LLC hosted and led Akamai Research LLC hosted and led the Hawaii effort with volunteers and the Hawaii effort with volunteers and sponsorship from e-Vehicles of Hawaii.sponsorship from e-Vehicles of Hawaii.

► In only three months the car was born. In only three months the car was born. ►HI_Lite stayed in consideration up to the HI_Lite stayed in consideration up to the

semi finals of the DARPA challenge.semi finals of the DARPA challenge.

HI_Lite VehicleHI_Lite Vehicle

Current statusCurrent status

►Vehicle is currently dismantled.Vehicle is currently dismantled.►No longer bounded by the DARPA No longer bounded by the DARPA

contest rules.contest rules.►Looking to improve operational Looking to improve operational

capabilities/ strategies and robotic capabilities/ strategies and robotic controls, integrate wireless feedback controls, integrate wireless feedback and multiple remote controllers, and multiple remote controllers, including human operators, and allow including human operators, and allow for “urban” environments. for “urban” environments.

Our ProjectOur Project

►To explore the capabilities of the To explore the capabilities of the vehicle’s “eyes” and derive a primitive vehicle’s “eyes” and derive a primitive useful “environment describing” useful “environment describing” system.system.

►Coupling simple optical devices to Coupling simple optical devices to create an enhanced “environment create an enhanced “environment describing” system.describing” system.

Block DiagramBlock Diagram

Software

Small Vision SystemMatlab (or C++)

Hardware

Videre Design digital stereo Camera Videre Design digital stereo Camera headhead

Laser PointersLaser PointersLensesLenses

Block Diagram - HardwareBlock Diagram - Hardware

► Videre Design Variable baseline digital stereo camera Videre Design Variable baseline digital stereo camera headhead Provides left and right video/images for depth/3D informationProvides left and right video/images for depth/3D information

► Laser PointersLaser Pointers Create a 3D laser grid and illuminate grid on objects within Create a 3D laser grid and illuminate grid on objects within

the camera's view to calibrate depth informationthe camera's view to calibrate depth information► If necessary, external "pre-processing" will be done If necessary, external "pre-processing" will be done

using color lenses as filters to enable the camera to using color lenses as filters to enable the camera to see the lasers as lines instead of pointssee the lasers as lines instead of points

► If necessary, use the SICK Laser Range Detector to If necessary, use the SICK Laser Range Detector to establish numerical calibration points and establish numerical calibration points and calibrate/tune the stereo camera using the SICK's calibrate/tune the stereo camera using the SICK's datadata

Block Diagram - SoftwareBlock Diagram - Software

► Small Vision System – Videre bundled softwareSmall Vision System – Videre bundled software Record stereo imagesRecord stereo images Rectify the images to account for distortionRectify the images to account for distortion Perform stereo correlation to compute a range Perform stereo correlation to compute a range

image (depth contours)image (depth contours)

► Matlab - interpret data and provide useful Matlab - interpret data and provide useful outputsoutputs Object detection calculations using edge detection Object detection calculations using edge detection

and boundary tracing on disparity imagesand boundary tracing on disparity images Correlate objects with numerical distance points Correlate objects with numerical distance points

from laser gridfrom laser grid

Software – In DepthSoftware – In Depth

►Object DetectionObject Detection►PhilsophyPhilsophy

Simpler = Better!Simpler = Better! Why?Why?

►We don’t need to record or identify objects we We don’t need to record or identify objects we “see”, we only need to avoid hitting objects“see”, we only need to avoid hitting objects

Software – In DepthSoftware – In Depth

►Object DetectionObject Detection►AlgorithmAlgorithm

SVS library performs disparity calculations SVS library performs disparity calculations which give us depth information that can which give us depth information that can be converted to a 3D image be converted to a 3D image

Software – In DepthSoftware – In Depth

►Using the disparity image, look for Using the disparity image, look for edges that will distinguish object and edges that will distinguish object and trace the boundaries around each trace the boundaries around each objectobject

Software – In DepthSoftware – In Depth

►To ensure a complete object, look for To ensure a complete object, look for cohesion in objects depth by observing cohesion in objects depth by observing correlation between the pixels in the correlation between the pixels in the disparity image since the disparity disparity image since the disparity image uses the brightness of the image uses the brightness of the pixels to show depth (brighter = pixels to show depth (brighter = closer)closer)

Software – In DepthSoftware – In Depth

►Separate objects, then output object Separate objects, then output object information and distance (determined information and distance (determined from laser grid)from laser grid)

Software – In DepthSoftware – In Depth

►Potential problems with algorithm:Potential problems with algorithm: Tiny objectsTiny objects Overlaying objectsOverlaying objects Oddly shaped objectsOddly shaped objects

Test BedTest Bed

5 Feet

5 1/2 Feet

6 Feet

5 Feet

5 1/2 Feet

6 Feet

Test Bed 2Test Bed 2

Filters

Laser “array”

distance

5 Feet

5 1/2 Feet

6 Feet

Test Bed 3Test Bed 3

Filters

Laser “array”

5 Feet

5 1/2 Feet

6 Feet

Test Bed 4Test Bed 4

Filters

Laser “array”

Work to be doneWork to be done

► Implement useful simple algorithm for Implement useful simple algorithm for object detectionobject detection

►Test effects of coupling devicesTest effects of coupling devices►Move from image to video functionalityMove from image to video functionality►Testing, testing, testing!Testing, testing, testing!

Hi-Lite Gant Chart Scott Fukuda and Chad Kawakami              

  18-Feb 25-Feb 4-Mar 11-Mar 18-Mar 25-Mar 1-Apr 8-Apr 15-Apr 22-Apr 29-Apr

Tests                      

Setup Test Bed 1                      

Run Test Bed 1                      

Setup Test Bed 2                      

Run Test Bed 2                    

Data                      

organize                      

algorithm                      

Exploratory                      

additional Tests                      

QUESTIONS?QUESTIONS?