time lapse and stabilising a sequence of images (gstreamer conference 2011)
DESCRIPTION
By Guillaume Emont. The topic for this talk was encountered when trying to make a time lapse film with a camera attached to a tethered helium balloon high above. This talk will introduce the issues of stabilising a sequence of images. It will mainly focus on "discreet" sequences of images (time lapses, stop motion) but will have many bits relevant to more typical continous-ish videos. It will introduce some methods to solve that problem, and various benefits of having these methods in GStreamer, so that they are available to solve other related problems (such as following an object, movement detection, automated compositing or 3D reconstruction). Of course, there will be cool demos, hopefully featuring stunning aerial pictures getting transformed into a smooth video. More information at http://emont.org/blog/2011/11/10/gstreamer-and-opencv-for-image-stabilisation/.TRANSCRIPT
Image Stabilisation
Guillaume EmontIgalia
Image StabilisationHoliday pictures
that picture is not by me(it is © 2011 Ugo Riboni)
...but I was there!
with a plan!
What the balloon sees
Let's go higher
Higher!
HIGHER!
I said: HIIIGHEEEER!
Ok, that will do
We wanted to take pictures for 7 days
The balloon agreed to 4 hours and a half
There was an intimate ceremonywith family and friends
Nice pics! ...now what?
Time lapse!
-- [timelapse-theory.png] [fit] [text-align=center] <span font="20">(image from wikipedia under GFDL)</span> -- [timelapse-theory.png] [fit] [text-align=center] Crazier: one frame per minute!
Removing the training wheels:No tripod.
(luckily, we have OpenCV)
Stabilisation in post-processing:
1. detect the movement (optical flow)
Optical flow: position changes of features
Stabilisation in post-processing:
2. compensate
Compensation:
OpenCV provides nice tools(but we won't talk about them here)
Optical flow: a few algorithms
Horn-Schunck
Lucas-Kanade
gst-launch-0.10 filesrc location=shaky-hippo.ogg ! decodebin ! ffmpegcolorspace ! opticalflowcorrector algorithm=1 ! ffmpegcolorspace ! xvimagesink
Err, yeah but...What about the balloon pictures?
Horn-Schunck
Lucas-Kanade
SpeededUpRobustFeature
SURF
Woohoo!
Bonus track
TODO:Check out the optical flow work by David Jordan (used for slow motion)and try to apply it to stabilisation
TODO:better compensation (tesselation?)
TODO:try more algorithms (BRIEF and ORB)
TODO:automatic optimal choice of algorithm and parameters
TODO:reimplement in C and put in -bad?
TODO:for videos: movement prediction/averaging(follow the intended movement of the cameraman)
questions?
Image credits:training wheels: CC BY-NC-SA 2.0 Dale Basler http://www.flickr.com/photos/baslercast/surf: CC BY-NC-SA 2.0 Guilherme Meneghelli http://www.flickr.com/people/iberere/last one: CC BY-NC-SA 2.0 Martin Ibert http://www.flickr.com/photos/mar_ibert/
Thanks! http://www.gitorious.org/gststabilizerhttp://balloonfreaks.mooo.com/http://emont.org/blog/http://www.igalia.com/[email protected]