Trajectory-Based Ball Detection and Tracking with Aid of Homography

in Broadcast Tennis Video

Xinguo Yu, Nianjuan Jiang, Ee Luang Ang

Present by komod

Visual Communications and Image Processing 2007

Proc. of SPIE-IS&T Electronic Imaging, SPIE Vol. 6508

Introduction

• The ball is the most important object in tennis (and in many kind of sports)

• Very challenging problem– Camera motion– presence of many ball-like objects– small size and the high speed of the ball– Object-indistinguishable

Introduction

• Method– Trajectory-based

• the ball is the “most active” object in tennis video• previous work ： A Trajectory-based ball detection

and tracking algorithm in broadcast tennis video, Proc. of ICIP

– Homography• Goal

– find projection locations of the ball on the ground

– find landing positions

Introduction

Introduction

Feature Point Extraction• Court Segmentation

– Find the court color range and paint all the pixels in this range with a single color

– find the lines separating the audience from the playing field

• detecting the change pattern of color for each row and column of the image

– paint the audience area in the court color.

Feature Point Extraction

• Straight Line Detection– gridding Hough transform

• Court Fitting – Detect the net and use it as reference– find the intersection of lines

Homography Acquisition

• Standard Frame– whose lookat is the cluster center of all

lookats of all the frames in the considered clip

– The lookat of frame is a point in the real world that corresponds to the center of the frame

Homography Acquisition

• Disparity Measure of Two Court Images– For i = 1 to 9

• Measure Function– Let Cstd be the court in the standard frame and Ctrn

denote the transformed court from the segmented court in frame F

– For given H and F

Homography Acquisition

• Initial Matrix– transforms an image point X' (x1', y2', 1) to a

point X (x1, y2, 1) in another image– X = HX‘

• Tuning of Homography– The homograph matrix computed based on

feature points

– A small hough space enclosing it

Homography Acquisition

• Tuning procedure

• Frame transform

Ball Location In Hitting Frame

• Hitting frame detection– Find the sound emitted by the racket

hitting• M. Xu et al, Creating audio keywords for

event detection in soccer video, In Proc. of ICME

• Hitting racket detection– Maybe player tracking

Ball Candidate Detection

• Object segmentation from standard frame• Four sieve are used for non-ball object removal

– Court Sieve Θ1

• filter out audience area• filter out court lines

– Ball Size Sieve Θ2

• filter out the objects out of the ball-size range• homography from ground model to standard frame• use a range of allowable ball sizes (estimate error)

– Ball Color Sieve Θ3

• filter out the objects with too few ball color pixels

– Shape Sieve Θ4

• filter out objects out of the range of width-to-height ratio• 2.5 is suggested in previous paper

– Each sieve is a Boolean function on domain Ο(F)

– The set of remaining objects is C(F)• C(F) = {o : o ∈O(F), Θi(o)=1 for i = 1 to 4}

• Candidate Classification– Three features are use

• Size, color, and distance from other objects

– The ball-candidates are classified into 3 Categories

Ball Candidate Detection

Candidate Trajectory Generation

• No detail explanation in this paper– X. Yu et al, Trajectory-based ball detection and

tracking of broadcast soccer video, IEEE Transactions on Multimedia, issue 6, 2006.

• Candidate Feature Plots (CFPs)– CFP-y– CFP-l

• The algorithm is actually works on the CFP-l which are 3-D plots

Candidate Trajectory Generation

Trajectory Processing

• Trajectory Confidence Index– Let T be a candidate trajectory

– and λ1,λ2,…,λm, be all properties of trajectory T

– confidence index Ω(T)

Trajectory Processing

• Trajectory Discrimination

Trajectory Processing

• Ball Projection Location– y = an3 + bn2 + cn + d.

• Ball Land Detection– form a ball position function against

frame number i, y = f(i)– find the maximum of f '(i) between each

pair of hittings

Experimental Results

• 5 clips

• extracted from mpeg2 704x576

• average time for acquiring ball candidates

• ALGnew for a frame is 86.15s on a P4/1.7Ghz PC with 512MB RAM

• ALGold is 19.21s

Experimental Results

BPL

Experimental Results

Experimental Results

• average discrepancy of all detected balls from the groundtruth

previous result

Experimental Results• frames with inserted 3D projected virtual content

• Homography in home surveillance video

Conclusion and Future Works

• The previous algorithm mainly alleviated the challenges raised by causes besides camera motion

• The algorithm presented in this paper additionally counteracts the challenges brought to us by the camera motion

• The contributions of this paper are two-fold– it develops a procedure to robustly acquire an accurate

homograph matrix of each frame– it forms an improved version of ball detection and tracking

algorithm• Two future works

– evolve the algorithm into an end-to-end system for ball detection and tracking of broadcast tennis video

– analyze the tactics of players and winning-patterns, and hence produce rich indexing of broadcast tennis video by making use of the ball position

Any Question?

Thank You

Experimental Results

• 7 segments, total 120 s, mpeg1 video, Men’s Final of FRENCH OPEN 2003

back