tennis real play: an interactive tennis game with models from real videos jui-hsin lai et al., ntu...
TRANSCRIPT
TENNIS
REAL P
LAY:
AN
INTE
RACTIVE T
ENNIS
GAME WIT
H MODELS
FROM R
EAL VID
EOS
J UI -
HS
I N L
AI
ET
AL . ,
NT
U
AC
M M
M 2
01
1
THE CHALLENGES
1.Player Database- Auto Video Analysis- Auto Clip Collection
2.Player Rendering- Realtime Clip Editing- Player Strategy Analysis
3.The Game System- Background/Foreground Rendering- Realtime Processing
THE CHALLENGES
1.Player Database- Auto Video Analysis- Auto Clip Collection
2.Player Rendering- Realtime Clip Editing- Player Strategy Analysis
3.The Game System- Background/Foreground Rendering- Realtime Processing
SOLUTION: FIDUCIAL COORDINATE
Fiducial (adj.) 基準的
(m0~m7: homography parameters, can be determined by trigonometrics)
PLAYER DB: HIT STATISTICS
Jui-Hsin Lai et al., “Tennis Video 2.0: A New Presentation of Sports Videos with Content Separation and Rendering”, JVCI 2011
1. Forehand volley
2. Backhand volley
3. Forehand stroke
4. Backhand stroke
5. Drop shot
PLAYER DB: BEHAVIOR MODEL
• The final piece of the DB that combines other pieces
• Players speed, hit trajectory, reaction time…etc.
• Can create ‘virtual ’matches between players (like fantasy baseball!)
kcir:
In other words, the gamer actually plays minor role in this ‘game’.
Whether a player can catch up (speed/direction); how does one choose to hit (force/angle), these are pre-determined.
THE CHALLENGES
1.Player Database- Auto Video Analysis- Auto Clip Collection
2.Player Rendering- Realtime Clip Editing- Player Strategy Analysis
3.The Game System- Background/Foreground Rendering- Realtime Processing
PLAYER RENDERING: CLIP SELECTION
If the player decides (how?) to go from point A to point B:
The system automatically choose the “suitable” clips and seam them together.
Suitable: 1. shortens the distance 2. runs along the path 3. min # of clips
FROM “MOTION” TO “HIT”
Shape/texture similarity between the frame of the clip
The Player DB
kcir:
So the swing of the gamer is meaningless.
What only matters is the timing of swing.
FROM “MOTION” TO “HIT”
In order to look good:
They want to use “view morphing” in between the transitionPROBLEM: YOU NEED
‘LABELS’ BEFORE YOU CAN MORPHSolution: [JCV09] and [JCV04] on
Auto Labeling for morphing
THE CHALLENGES
1.Player Database- Auto Video Analysis- Auto Clip Collection
2.Player Rendering- Realtime Clip Editing- Player Strategy Analysis
3.The Game System- Background/Foreground Rendering- Realtime Processing
EVALUATION
• Computation Analysis
• Prediction of Game Result - Virtual Match
• Subjective Evaluation- Compares to Wii and xbox- Interactivity- Immersive Experience
COMPUTATION ANALYSIS
Intel i7 2.6 GHz CPU with 4GB RAM
Rendering resolution: 720 x 480
=> Frame rate: 30 fps (standard mpeg1 requirement)
Basis
You can offline-computed these two
SUBJECTIVE EXPERIMENTS
20 undergrads
5-point scoring system
Q1. Interaction
Q2. Immersive Experience
Q3. Interesting
Q4. Innovative application (huh?)
Q5. Willing to play TRP after watching videos?
SUBJECTIVE EXP: COMPARES TO OTHERS
Participants played TRP, Wii tennis, and TP3 xbox
Q6. Entertainment levels
Q7. Realism of visual effects
Q8. Interactiveness (didn't you just ask this in Q1?)
Q9. Preference
RESULTS
Using Wii as the comparison basis (score=3)
Q1. Interaction
Q2. Immersive Experience
Q3. Interesting
Q4. Innovative application (huh?)
Q5. Willing to play TRP after watching videos?
Q6. Entertainment levels
Q7. Realism of visual effects
Q8. Interactiveness
Q9. Preference
CONCLUSION
1. The first work to integrate video-based rendering and interactive sports game running in realtime
2. Without motion capture system, construct game from video without much human assistance
3. Game results can reflect the match results in real world and predict match results
COMMENTS
1. The value of this work is its auto process of video->game, which includes:- video analysis- auto editing of the clips- realtime rendering- computational complexity to overcome with
2. The game itself, although put in the title, is just a bonus
3. How can this be better than Wii !?!?!?- gamer cannot control how they hit. The hitting result is determined by the player DB. (Williams always wins)- gamer don't have to chase the ball. The player will instantly know where to go. (this is cheating!)- You will always win if you know which player to pick….And you call this FUN? Poor consumers