ball detection and tracking using color featuresdsie10/presentations/session 5/ball detection...

Ball Detection and Tracking using Color Features

Catarina B. Santiago catarina.santiago@fe.up.pt

Armando Sousa asousa@fe.up.pt

Luís Paulo Reis lpreis@fe.up.pt

Luísa Estriga lestriga@fade.up.pt

Presentation Outline

• Motivation

• Challenges

• Image Processing

• Test Platform

• Results

• Conclusions

• Future Work

2

Motivation

• Interest by the sports community to monitorgames and training sessions

• Championships do not allow intrusive system

• Image processing techniques

o Deal with a huge amount of information/data

o Analyse the position and path of the ball

3Motivation Challenges Processing Test Plat. Results Conclusions Future

Challenges

• Develop a system to collect images from a movingball

o Able to detect a ball based on color features (using blobnotion)

o Perform the tracking of the ball

o Can in future be applied to a real handball game

4Motivation Challenges Processing Test Plat. Results Conclusions Future

• Color calibration

5

o Physical flood

o Colour growth

Color ball subspace

ColorCalibration

Motivation Challenges Processing Test Plat. Results Conclusions Future

RGB cube

HSL expansion

• Background Subtraction

o Conditional subtraction based on the color difference betweenthe image under analysis and the background image

o Background is dynamically updated in each frame

6

Background Subtraction

ColorCalibration

Motivation Challenges Processing Test Plat. Results Conclusions Future

• Color Detection

o Compare each pixel color (binning)

• color ball subspace => pixel in the image is replaced by the ballidentifier

• color ball subspace => pixel keeps the original color

Image with the

color detected

Color ball

subspace

7

Background Subtraction

ColorDetection

Color Calibration

Motivation Challenges Processing Test Plat. Results Conclusions Future

Original

image

• Blob Aggregation and Characterization

1. Scan per line to join pixels that belong to the ball

2. Join lines that belong to the same color blob

xmin xmax

y

xmin xmax

y

1 1 1 1 1 1

1 1 1 1 1 1

1 1 1

5 5 5

5 5 5 5 6 7 7

5 5 5 8 8 8 8

1 1 1 1 2 2

3 3 3 3 3 3

4 4 4

5 5 5

5 5 5 5 6 7 7

6 6 6 8 8 8 8

8

Background Subtraction

Color Detection

Blob Aggreg. and Charact.

Color Calibration

Motivation Challenges Processing Test Plat. Results Conclusions Future

• Blob Aggregation and Characterization

o Minimum and maximum x and y

o Blob area (bounding box)

o Blob centre of mass (converted into world coordinates)

o Blob density

9

Background Subtraction

Color Detection

Blob Aggreg. and Charact.

Color Calibration

Motivation Challenges Processing Test Plat. Results Conclusions Future

• Tracking the Ball

o The centre of mass of the ball is saved into a file which allows to determine statistics

Tracking algorithm

• Previous blob

characteristics

• Maximum

speed

likely

position area

Background Subtraction

Color Detection

Blob Aggreg. and Charact.

Tracking

10

Color Calibration

Motivation Challenges Processing Test Plat. Results Conclusions Future

Test Platform

• Test set mounted in the laboratory

o GigEthernet camera

o 2 sample footages of 7 seconds with fast and slow ball movements

o MJPEG encoding

o Used image size 412x708

o 30 frames per second

11Motivation Challenges Processing Test Plat. Results Conclusions Future

Results

• Impact of the color calibration expansion

12Motivation Challenges Processing Test Plat. Results Conclusions Future

Results

• Ball detection

o Detection rates

o Detection under severe light conditions

13

1st set 2nd set

98% 100%

Motivation Challenges Processing Test Plat. Results Conclusions Future

Results

• Ball Tracking

o 1st set – red, green and blue

o 2nd set – rose and purple

o Each frame took ~33ms to be processed laptop computer with 1MB L2 cache and powered by an Intel T2130 processor running at 1.86GHz

14Motivation Challenges Processing Test Plat. Results Conclusions Future

Conclusions

• System for detecting and tracking a ball (lab tested)

• Image processing using blob notion is a powerful and fast tool

• Color calibration has impact on the ball detection

• Ball detection rate: 98% (1st set) and 100% (2nd set)

• Ball tracking is possible

• On going work for handball tracking on a sports hall

15Motivation Challenges Processing Test Plat. Results Conclusions Future

Future Work

• Adapt the actual setup to a real environment

o Mount the system in a sports hall (FADEUP)

o Use it in real game situations (barrel distortion effects, other objects, varying light conditions)

• Improve overall performance

o Artificial Intelligence

o Kalman filter

• Develop a better interface for the data analysis

• Support system to aid and backup the teacher/coach

16Motivation Challenges Processing Test Plat. Results Conclusions Future

Ball Detection and Tracking using Color Features

17

THANK YOU!

Questions?