Harper Langston

Simple Tracking of 3D RigidSimple Tracking of 3D RigidBodies in FluidsBodies in FluidsSimple Tracking of 3D RigidSimple Tracking of 3D RigidBodies in FluidsBodies in Fluids

MotivationMotivationMotivationMotivation

• Ability to track objects moving in fluids may Ability to track objects moving in fluids may provide insight into properties of the fluidprovide insight into properties of the fluid

• Verify numerical simulationsVerify numerical simulations

• Study aquatic dynamics (current work in Study aquatic dynamics (current work in Math Department for example)Math Department for example)

• Create nice animations, recreate scene with Create nice animations, recreate scene with moving camera in 3D, etc.moving camera in 3D, etc.

Simple SetupSimple SetupSimple SetupSimple Setup

• Two cameras with Two cameras with sloppy calibration (front sloppy calibration (front view/side view)view/side view)

• One cheap fish tank One cheap fish tank bought at Petcobought at Petco

• Multi-colored balls, Multi-colored balls, sliced open to allow sliced open to allow desirable amount of desirable amount of water in and a water in and a cardboard rampcardboard ramp

First ExperimentsFirst ExperimentsFirst ExperimentsFirst Experiments

• Track a single ball, Track a single ball, using Lucas-Kanade using Lucas-Kanade tracker and see how it tracker and see how it lookslooks

• Pretty good as can be Pretty good as can be seen from video seen from video (slowed down 10 times (slowed down 10 times on bottom – good way on bottom – good way of faking higher of faking higher viscosity or heavier viscosity or heavier object)object)

Getting More ComplicatedGetting More ComplicatedGetting More ComplicatedGetting More Complicated

• With the single ball, there was very little With the single ball, there was very little movement in the ‘x’-direction of camera #2, movement in the ‘x’-direction of camera #2, or in the global ‘z’-direction. Once we add or in the global ‘z’-direction. Once we add the ‘ramp’, it gets more complicatedthe ‘ramp’, it gets more complicated

• Sloppy calibration: Use 1-in marks on side Sloppy calibration: Use 1-in marks on side of fish tank as well as knowledge of camera of fish tank as well as knowledge of camera location and size of tank. Further, measure location and size of tank. Further, measure radii of spheres in both camera views to find radii of spheres in both camera views to find a scale factor. Choose camera #1 as ‘true’ a scale factor. Choose camera #1 as ‘true’ values, and scale camera #2’s values, and scale camera #2’s measurements accordingly.measurements accordingly.

Multiple Objects ContinuedMultiple Objects ContinuedMultiple Objects ContinuedMultiple Objects Continued

• Simply using Lucas-Simply using Lucas-Kanade doesn’t cut it. Kanade doesn’t cut it. It would be good to use It would be good to use Condensation, but let’s Condensation, but let’s see if we can get away see if we can get away without it.without it.

• To distinguish colored To distinguish colored items, use ‘Skin-Color items, use ‘Skin-Color Tracker’, computing Tracker’, computing HSV to isolate the HSV to isolate the layer for each color.layer for each color.

ContinuingContinuingContinuingContinuing

• Now, it is easier to use Lucas-Kanade iteratively to track Now, it is easier to use Lucas-Kanade iteratively to track each object. Of course there is some degradation and each object. Of course there is some degradation and noise, so use a Kalman filter to clean it up! May takes time noise, so use a Kalman filter to clean it up! May takes time for each motion to find right parameters but is smooth.for each motion to find right parameters but is smooth.

Let’s put it togetherLet’s put it togetherLet’s put it togetherLet’s put it together

• First, view the animation I captured from the frontFirst, view the animation I captured from the front

ResultResultResultResult

• With Skin-Color Tracker, Lucas-Kanade and With Skin-Color Tracker, Lucas-Kanade and Kalman Filter (Without purple ball)Kalman Filter (Without purple ball)

How the Animation is ProducedHow the Animation is ProducedHow the Animation is ProducedHow the Animation is Produced

• Treat each ball as a single-jointed object and Treat each ball as a single-jointed object and create a biovision format file (create a biovision format file (http://www.cs.wisc.edu/graphics/Courses/cs-838-1999/Jeff/BVH.htmlhttp://www.cs.wisc.edu/graphics/Courses/cs-838-1999/Jeff/BVH.html))

• Allows us to consider extensions to multi-jointed Allows us to consider extensions to multi-jointed dynamic setups possiblydynamic setups possibly

• The Bvh file is imported into Maya using Maya’s The Bvh file is imported into Maya using Maya’s MEL scripting language to create key frames for MEL scripting language to create key frames for the ‘skeleton’the ‘skeleton’

• Physical geometric object is created in Maya and Physical geometric object is created in Maya and bound to the one-jointed skeletonbound to the one-jointed skeleton

• Water effects and lighting, etc.!Water effects and lighting, etc.!

Continuing OnContinuing OnContinuing OnContinuing On

• Spent time working on adding Lucas Kanade with Spent time working on adding Lucas Kanade with rotations [Bergen et al. 1992, “Three-Frame Algorithm for rotations [Bergen et al. 1992, “Three-Frame Algorithm for Estimating Two-Component Image Motion”]. Like we Estimating Two-Component Image Motion”]. Like we discussed in class, this allows us to find any affine discussed in class, this allows us to find any affine transformationtransformation

Is Lucas-Kanade with Rotations, Is Lucas-Kanade with Rotations, Scaling and Shearing Worth ItScaling and Shearing Worth ItIs Lucas-Kanade with Rotations, Is Lucas-Kanade with Rotations, Scaling and Shearing Worth ItScaling and Shearing Worth It

• Clearly, the last animation had many Clearly, the last animation had many obstructions. Putting aside the clear need obstructions. Putting aside the clear need for a less crude setup, the Condensation for a less crude setup, the Condensation algorithm would be a better choice for algorithm would be a better choice for scenes cluttered with potentially many scenes cluttered with potentially many ‘particles’‘particles’

• Extension could also include [Koller-Meier, Extension could also include [Koller-Meier, Ade, ‘Tracking Multiple Objects Using the Ade, ‘Tracking Multiple Objects Using the Condensation Algorithm”]Condensation Algorithm”]