Live video streaming and analysis in indoor soccer with a quadcopter

Filipe Trocado FerreiraMSc Electrical Engineering-Automation

Jaime Cardoso (PhD)Advisor

Hélder Oliveira (PhD)Co-Advisor

MotivationLive Soccer Analysis

◦ Big Investments in technical and tactical preparation

◦ Individual and team performance data collected from video

The Problem◦ Complex implementation

◦ Still a lot of human intervention

◦ System not suitable for the big majority of the teams

Our Solution◦ Low cost architecture based on an unnamed

air vehicle with a camera onboard

Main Goals Video sequences from Indoor Soccer Games using an Ar.Drone

2.0 Collect and Filtering low-level data as:

◦ Players and Ball positions and trajectories in both image and world coordinate system

◦ Team Identification

◦ Frame-to-Frame and Image-to-Pitch transformations

◦ Occlusion detection and resolution

High-Level data:◦ Ball Possession

◦ Field Occupation

◦ Offensive/Defensive trends

◦ …

Flight Control Architecture for Automatic Image Recording (optional Goal):◦ Position Stabilization

◦ Ball following and avoidance

Initial Framework

Video Sequences

Motion Compensation

Stable Images

Camera CalibrationH

Calibrated Images

Player and ballDetection

Temporal and SpatialFiltering

Raw Player coordinates

Player Coordinates

High-Level Data Interpretation

Video sequences Image sequences recorded by an Ar.Drone 2.0 (manual

flight control)

Main issues:◦ Camera stabilization and motion compensation

Camera Stabilization Calculation of Frame-to-Frame affine transformation Using Point Feature Matching (FAST Features)

Main issues:◦ Can not handle big oscillations

Camera Calibration Correction of “Barrel” Effect Calculation of the perspective trasnformation (min. of 4 pair of

points requested)

Main issues:◦ Long term drift

◦ There is the need of a periodic perspective recalibration

◦ Need of a precise geometric model for each playground

Player Detection Non static background and a lot of lines of different colors on the

playground do not allow basic segmentation methods (as background subtraction or color segmentation)

HOG people detection: Histogram Oriented Gradients features and a trained SVM classifier for detecting players in an upright pose.

Main Issues:◦ Fails when players are running, tackling and in cases of occlusions.

◦ Low False Positives but a lot of missed Positives

Some Raw Results…

What comes next?Spatial and Temporal Filtering:

◦ False Positive handling◦ Dynamic Model (Kalman Filter/Particles Filter)◦ Team/Player Identification

Initial Framework Improvements:◦ Dynamic Calibration◦ Creation of a SVM classifier for HOG player detector

Ground truth annotationHigh-Level Data Interpretation:

◦ Basic Statistics with filtered player positions◦ Complex Team Information◦ Relation with players and ball possession

Time to shoot some questions!