MatchmovingThe Invisible Art of Camera Tracking

Second Edition

MatchmovingThe Invisible Art of Camera Tracking

Second Edition

T i m D o b b e r t

Acquisitions Editor: Mariann BarsoloDevelopment Editor: Sara BarryTechnical Editor: Michael RaphaelovichProduction Editor: Eric CharbonneauCopy Editor: Sharon WilkeyEditorial Manager: Pete GaughanProduction Manager: Tim TateVice President and Executive Group Publisher: Richard SwadleyVice President and Publisher: Neil EddeBook Designer: Franz BaumhacklCompositors: Cody Gates and Kate Kaminski, Happenstance Type-O-RamaProofreader: Rebecca RiderIndexer: Ted LauxProject Coordinator, Cover: Katherine CrockerCover Designer: Ryan SneedCover Image: © Tim Dobbert

Copyright © 2013 by John Wiley & Sons, Inc., Indianapolis, Indiana

Published simultaneously in Canada

ISBN: 978-1-118-35205-2

ISBN: 978-1-118-46123-5 (ebk.)

ISBN: 978-1-118-52966-9 (ebk.)

ISBN: 978-1-118-52927-0 (ebk.)

No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, with-out either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at www.wiley.com/go/permissions.

Limit of Liability/Disclaimer of Warranty: The publisher and the author make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation warranties of fitness for a par-ticular purpose. No warranty may be created or extended by sales or promotional materials. The advice and strategies contained herein may not be suitable for every situation. This work is sold with the understanding that the publisher is not engaged in rendering legal, accounting, or other professional services. If professional assistance is required, the services of a competent professional person should be sought. Neither the publisher nor the author shall be liable for damages arising herefrom. The fact that an organization or Web site is referred to in this work as a citation and/or a potential source of further information does not mean that the author or the publisher endorses the information the organiza-tion or Web site may provide or recommendations it may make. Further, readers should be aware that Internet Web sites listed in this work may have changed or disappeared between when this work was written and when it is read.

For general information on our other products and services or to obtain technical support, please contact our Customer Care Department within the U.S. at (877) 762-2974, outside the U.S. at (317) 572-3993 or fax (317) 572-4002.

Wiley publishes in a variety of print and electronic formats and by print-on-demand. Some material included with standard print versions of this book may not be included in e-books or in print-on-demand. If this book refers to media such as a CD or DVD that is not included in the version you purchased, you may download this material at http://booksupport.wiley.com. For more information about Wiley products, visit www.wiley.com.

Library of Congress Control Number: 2012948914

TRADEMARKS: Wiley, the Wiley logo, and the Sybex logo are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates, in the United States and other countries, and may not be used without written permission. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc. is not associated with any product or vendor mentioned in this book.

10 9 8 7 6 5 4 3 2 1

Dear Reader,Thank you for choosing Matchmoving: The Invisible Art of Camera Tracking,

Second Edition. This book is part of a family of premium-quality Sybex books, all of which are written by outstanding authors who combine practical experience with a gift for teaching.

Sybex was founded in 1976. More than 30 years later, we’re still committed to producing consistently exceptional books. With each of our titles, we’re working hard to set a new standard for the industry. From the paper we print on to the authors we work with, our goal is to bring you the best books available.

I hope you see all that reflected in these pages. I’d be very interested to hear your comments and get your feedback on how we’re doing. Feel free to let me know what you think about this or any other Sybex book by sending me an email at nedde@wiley.com. If you think you’ve found a technical error in this book, please visit http://sybex.custhelp .com. Customer feedback is critical to our efforts at Sybex.

Best regards,

Neil EddeVice President and PublisherSybex, an Imprint of Wiley

For Desiree, without whom this book would not have been possible.

AcknowledgmentsI’d like to send out a thanks to everyone at Sybex who was involved with this book, especially Mariann Barsolo, Pete Gaughan, Sara Barry, Eric Charbonneau, Michael Raphaelovich, Sharon Wilkey, Cody Gates, and Kate Kaminski. You helped bring focus to my ideas, and your encouragement was greatly appreciated.

I’d like to thank Sean Wagstaff for his early support and for kicking me in the pants a little to help get this book off the ground. Thanks to the folks at Andersson Technologies and The Pixel Farm, especially Russ Andersson and Daryl Shail for their support. Thanks also to Jeff Saltzman, Joe Stevenson, and Brian Cantwell at Industrial Light & Magic for all of their feedback and suggestions, as well as Jesse Kim and the Academy of Art University Spring ’12 matchmoving class and Tony Rizo for their help with the Second Edition content.

I’d particularly like to thank everyone who contributed some of the fantas-tic images and sequences used in this book: Greg Downing, Alex Lindsay at Pixel Corps, Jorge Mendoza of Litigation Animation, Quan Tran, and Trevor Tuttle.

And special thanks to my wife, Desiree, whose patience and support never flagged for a moment during the writing of both editions of this book. I can’t express how much I appreciate everything you’ve done. And special thanks too, to my daugh-ters, Sabina and Alex, who gave up a lot of their “daddy-time” so that this book could happen.

Tim Dobbert has been working in the visual effects industry for more than 10 years. He began as an animator/generalist at Flat Earth Productions and Foundation Imaging, working on projects such as Roughnecks: Starship Troopers Chronicles and Max Steel.

In 2000, he began working as a CG artist for Realviz, the company that created MatchMover Pro software. In this capacity, in addition to gaining a deeper knowledge of how matchmoving programs work, he traveled throughout the world helping companies establish their matchmoving pipelines and training them on the software. During this time, he created the first matchmoving classes at the Academy of Art University in San Francisco (and continued to teach them for another 10 years afterward).

In 2004, he created the matchmoving department at an up-and-coming visual effects studio known as The Orphanage. Over the next four years at that company, he worked on such films as Hellboy, Sky Captain and the World of Tomorrow, Harry Potter and the Goblet of Fire, Pirates of the Caribbean: Dead Man’s Chest, Sin City, and Iron Man. While there, he wrote the first edition of this book, which was the first book ever written exclusively on the subject of matchmoving.

In 2009, he joined Industrial Light & Magic, where he has worked on films such as Avatar and Pirates of the Caribbean: On Stranger Tides.

About the Author

ContentsIntroduction xxi

Chapter 1 The Basics of Matchmoving 1

Exploring a Typical Matchmove . . . . . . . . . . . . . . . . . . . . . . . . 2

Moving from 3D to 2D and Back Again. . . . . . . . . . . . . . . . . . 4

Understanding the Matchmoving Process . . . . . . . . . . . . . . . . . 5Evaluating the Footage 5Applying Information 6Defining the Camera 6Fitting the Set 7Testing the Matchmove 9Delivering the Scene 9

Matchmoving in the Production Pipeline . . . . . . . . . . . . . . . . . 9

Perspective Matching Tutorial. . . . . . . . . . . . . . . . . . . . . . . . . 10Gathering the Data 11Setting Up the Camera 13Adding Rough Geometry and Refining the Camera 15Creating a Camera Rig 16

Evaluating and Adjusting the Camera. . . . . . . . . . . . . . . . . . . 18

Chapter 2 Matchmoving Software 23

Using Matchmoving Programs . . . . . . . . . . . . . . . . . . . . . . . . 24

Introducing Photogrammetry . . . . . . . . . . . . . . . . . . . . . . . . . 24Calibration 25The Optics of Photography 26Projection 28Building a Better Camera 31Parallax: A Matchmover’s Best Friend 38

Understanding How Matchmoving Programs Work. . . . . . . . 40

Requirements for Matchmoving Programs 41

Chapter 3 2D Tracking 43

Understanding the 2D Tracking Process . . . . . . . . . . . . . . . . . 44

Exploring the Anatomy of a 2D Track . . . . . . . . . . . . . . . . . . 45

Track Placement: Making Every Track Count . . . . . . . . . . . . 47Sample the 3D Space 47Maintain the Minimum Number of Tracks 48Track Stationary Objects 49Track “True” 3D Positions 50

Co

nt

en

ts

u

xiv

Getting Tracks to Stick . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

Following Hints for Effective 2D Tracking . . . . . . . . . . . . . . . 52

Handling Plate Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54Motion Blur 55Soft Focus or Rack Focus 56Occlusion 56Noise 58Low-Contrast Features 58Lack of Trackable Features 59

Optimizing the Plate for 2D Tracking. . . . . . . . . . . . . . . . . . . 61

2D Tracking Tutorial. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62Evaluating the Shot 63Placing the First Tracks in SynthEyes 63Choosing Your Tracks 66

Adding the Remaining Tracks 66

Chapter 4 3D Calibration 69

Understanding “Good” Calibration . . . . . . . . . . . . . . . . . . . . 70Evaluating Error 70

Calibrating Your Cameras. . . . . . . . . . . . . . . . . . . . . . . . . . . . 72Finding the Right Fit 72Calibration Tutorial 73

Evaluating the Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77Evaluating the 3D Nulls 77Evaluating the 3D Space 78Rendering the Matchmove 79When Good Solutions Go Bad 80

Determining Camera Moves . . . . . . . . . . . . . . . . . . . . . . . . . . 81Dolly/Crane/Moving Camera 81Pan/Tilt Shots 82Slight Dollies/Translation 82Lock-offs 83Zoom Shots 83Shaky Footage/Handheld Footage 84

Setting Up a Coordinate System . . . . . . . . . . . . . . . . . . . . . . . 84

Coordinate System Tutorial. . . . . . . . . . . . . . . . . . . . . . . . . . . 85

Exporting the Scene 88

Chapter 5 Automatic Tracking 91

Tracking on Autopilot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

Editing Automatic Tracks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

Using Masks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

u

Co

nt

en

ts

xv

Reaping the Benefits of Automatic Tracking . . . . . . . . . . . . . . 97Noise Reduction 97Environment Definition 98

Implementing the Automatic Tracking Process . . . . . . . . . . . . 99

Automatic Tracking Tutorial 99

Chapter 6 Cameras 107

How Film Cameras Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

Lenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110Focal Length 111Focus 113Nodality 113

Film Backs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115Film Back and Focal Length 115Digital Cameras 117CMOS, CCD, and Rolling Shutter 119Camera Information 121

Format. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

Lens Distortion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123Anamorphic Distortion 127

Capture and Digitization. . . . . . . . . . . . . . . . . . . . . . . . . . . . 128Video 128Film 129HD 130

Keeping It All Straight. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

Chapter 7 Set Fitting 133

Fitting the Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

Checking the Matchmove . . . . . . . . . . . . . . . . . . . . . . . . . . . 135Proxy Geometry 135Checkerboards 137

Fitting the Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138Moving the Camera 138Scaling the Camera 139Creating Camera Rigs 141

Set-Fitting Tutorial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141Examining the Exported Scene 142Adding Test Geometry 143Moving the Camera 144Texturing the Objects 149Performing the First Tests 150Placing the Matchmove in the Set 152

Co

nt

en

ts

u

xvi

Altering the Coordinate System. . . . . . . . . . . . . . . . . . . . . . . 156

Delivering the Scene . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

Tutorial Continued 157

Chapter 8 Being on Set 159

Getting the Right Information . . . . . . . . . . . . . . . . . . . . . . . 160

Getting Camera Information. . . . . . . . . . . . . . . . . . . . . . . . . 162Before the Shoot 162During the Shoot 162After the Shoot 165

Marking Up the Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165Tracking Markers for a Green Screen or Blue Screen 166Tracking Markers for Set Pieces 167Tracking Markers for Exterior Scenes 168

Working with Others on Set . . . . . . . . . . . . . . . . . . . . . . . . . 168

Building a Matchmoving Kit . . . . . . . . . . . . . . . . . . . . . . . . . 172

Shooting Distortion Grids . . . . . . . . . . . . . . . . . . . . . . . . . . . 173

Chapter 9 Matchamation 175

Understanding the Basic Technique. . . . . . . . . . . . . . . . . . . . 176Establishing the Distance to the Camera 176Setting the Initial Pose 177Using Nonlinear Animation Techniques 178Analyzing the Movement 178Making the First Pass 179Making Additional Passes/Tweaks 180

Making Model Changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

Working with Imperfect Data . . . . . . . . . . . . . . . . . . . . . . . . 181

Matchamation Tutorial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182Setting Up the Camera 182Creating the Proxy Object 184Creating a Matchamation Rig 185Analyzing the Motion 186Making the First Pass 188Making the Second Pass 191

Making Additional Passes 194

Chapter 10 Troubleshooting and Advanced Techniques 197

Performing Effective Troubleshooting. . . . . . . . . . . . . . . . . . 198

Following a Troubleshooting Checklist. . . . . . . . . . . . . . . . . 199Check the Camera’s Motion 199

u

Co

nt

en

ts

xvii

Check Set Placement and Fitting 201Scrutinize 2D Tracking Closely 202Check for Plate Issues 202

Diagnosing the Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203No nulls are visible after calibration. 204Nulls are all in a line or a single point. 204The camera path jumps suddenly to another position or is extremely jagged and chaotic. 204Everything lines up on the first frame but quickly slips off. 205Noise or jitter appears in the camera’s motion path. 205Objects appear to tear away from the plate feature during areas of heavy motion blur. 206

Using Advanced Tools and Tricks . . . . . . . . . . . . . . . . . . . . . 206Using Advanced 2D Tracking Tools 206Using Constraints 207Working with Reference Photos 209Forcing a Calibration 210Faking the Focal Length 210

Using Survey Data 211

Chapter 11 Object Tracking 215

The Object-Tracking Process. . . . . . . . . . . . . . . . . . . . . . . . . 216Understanding Object Tracking vs. Matchamation 216Following the Object-Tracking Workflow 216Converting a Camera Track to an Object Track 217

Object-Tracking Tutorial . . . . . . . . . . . . . . . . . . . . . . . . . . . 219

Common Problems with Object Tracking . . . . . . . . . . . . . . . 2292D Tracking Problems 229Difficult Object Motion 230Object Not Truly Rigid 231Tracking Human Faces 232

Geometry Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

Geometry-Tracking Tutorial . . . . . . . . . . . . . . . . . . . . . . . . . 234

Common Problems with Geometry Tracking . . . . . . . . . . . . 237

Advanced Object Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . 237Moving Camera/Moving Object 238

Tracking Deforming Objects 239

Chapter 12 Multipurposing Matchmoving Data 241

Creating Special Cameras . . . . . . . . . . . . . . . . . . . . . . . . . . . 242Extending or Enhancing Matchmoves 242Blending Matchmoves 243Blending Cameras in Maya 245Object Track as a Camera Track 251

Co

nt

en

ts

u

xviii

Matchmoves for Paint Work: “Patchmoves”. . . . . . . . . . . . . 252

Matchmoves for Compositors . . . . . . . . . . . . . . . . . . . . . . . . 253

Matchmoves for TD Work . . . . . . . . . . . . . . . . . . . . . . . . . . 254

Chapter 13 Stereo 257

Understanding Stereoscopy . . . . . . . . . . . . . . . . . . . . . . . . . . 258Interocular Distance 259Convergence 261How IO and Convergence Affect Stereo Depth 264Adjusting Stereo Parameters in Postproduction 265Stereo Postconversion 266

Filming Stereo Footage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266Side-By-Side Rigs 266Beam-Splitter Rigs 267Dual-Lens Cameras 269

Viewing Stereo Footage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269Anaglyphs 270Polarized Lens Systems 271Modern Stereo Projection 271Active Shutter Systems 272

Matchmoving Stereo Footage . . . . . . . . . . . . . . . . . . . . . . . . 2722D Tracking 273Solving 273Set Fitting 275Evaluating the Matchmove 275

Overcoming Common Problems . . . . . . . . . . . . . . . . . . . . . . 276Problems with 2D Tracking 276Problems with 3D Tracking 277

Focal-Length Issues 277

Appendix A Resources 279

Appendix B Matchmoving Software and Useful Links 281

Matchmoving Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282

Other Useful Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283

Books. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283

Websites. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284

Appendix C Film Formats 285

16 mm Formats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286

35 mm Formats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286

u

Co

nt

en

ts

xix

Large Formats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287

Digital Formats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287

Calculating the Film Back for Cropped Images . . . . . . . . . . 288

Glossary 289

Index 299

I remember getting my first glimpses of the sequences in Hellboy that

my team of matchmovers was about to work on. One that stood out

in particular was the opening sequence, in which the villains were

attempting to open the gates of hell with a special machine. The

sequence had all the elements of a great action film: a driving rain-

storm, lightning, explosions, people running everywhere, and a lot of

fast-moving camera work. I realized immediately that this would be

an incredible sequence, but also a very difficult one from the vantage

point of a matchmover.Experienced matchmovers look at effects shots with a different eye than other visual

effects artists because their job is unique. Their main goal is to decipher the clues in the live-action footage and create a 3D scene that animators can use to place their effects in the shot. It is an important job, even though it is not readily visible in the final shot. Actually, the only time a matchmove is visible in the shot is if it is a bad matchmove. I often tell my students, “If you’ve done your job right, no one should ever know you’ve done your job at all.”

In a lot of ways, matchmoving, or camera tracking as it’s sometimes called, is really just like solving a puzzle. Sometimes the puzzle can be fairly straightforward, and other times it can be extremely difficult. But it is this problem-solving aspect of matchmoving that has kept me working in this specialized area of visual effects for so long.

Many visual effects artists understand the need for matchmoving, but very few know what’s really happening “under the hood,” and that is part of the reason for this book. There is a strong technical aspect to matchmoving, but there is an art to getting a tough matchmove to work. The art of matchmoving becomes easier the more you do it, but interesting technical challenges always remain around every corner.

Who Should Read This BookI’ve written this book for two groups of people: visual effects artists and students.

Visual effects artists who are not matchmovers by trade are occasionally tasked with matchmoving in addition to their other duties as animators or technical directors. Generally, these folks are given some matchmoving software and a few tutorials and sent off to matchmove the shot. Being highly talented visual effects artists, they quickly figure out how the software works and throw their shot at it. If they’re lucky, it’s an easy shot,

Introduction

Int

ro

du

ct

Ion

u

xxii

and they achieve satisfactory results. But often they find that their footage is consider-ably more difficult than the tutorial footage and they are able to arrive at a solution only through a lengthy trial-and-error process. Much of the pain of matchmoving can be elimi-nated by knowing how the matchmoving software achieves its results. Relatively simple techniques can speed up the process, eliminate the guesswork, and help the matchmover deliver a quality solution to artists further down the production pipeline.

The second type of person is the 3D animation or compositing student. There are the obvious benefits to knowing how to accurately track 3D elements into your footage, but there are other more compelling reasons to master these skills. I firmly believe that matchmoving is a great way to break into the visual effects industry. This industry is a highly competitive one, and with little or no production experience, many students must bang their heads tirelessly on the doors of studios in search of a first job.

Many of my own students have landed their first jobs in the matchmoving depart-ments of high-profile visual effects companies. There is one simple reason for this: most visual effects artists don’t want to be matchmovers! They’d much rather be character animators, effects technical directors, or compositors. And who can blame them? It’s nice to be able to point to a cool character or explosion and say, “I did that!” But no one ever says, “Did you notice the brilliant matchmoves on that show? I didn’t see one single slip.” Matchmoving may not be the most glamorous job in visual effects, but there will always be a strong demand for knowledgeable matchmovers.

I’ve written this book to give visual effects artists the tools and techniques to help make their encounters with matchmoving as easy and pain-free and possible. And if you’re one of the few who decide to make a career of matchmoving, this book will be a great starting point for your studies. This book won’t tell you what every button and menu does for each matchmoving program—I’ll leave that to the user guides. It will, however, show you how those programs work and even walk you through the process.

I’ve designed this book to be as non-program-specific as possible, apart from the obvious need for matchmove-specific software. It was really tough to decide which match-moving programs to use for the tutorials, because there are so many great options for matchmovers these days. Because space was limited, I decided to pare it down to Andersson Technologies’ SynthEyes, The Pixel Farm’s PFTrack, and Vicon’s boujou. But for the most part, the techniques I describe here are compatible with all 3D animation and matchmoving software packages and do not require special knowledge of any particular programs.

What’s New in the Second EditionWhen I wrote the first edition of this book, I had some nagging doubts as to whether the book would be useful to folks interested in matchmoving, or whether there would even be an audience for such a book. In the seven years since, I have been fortunate enough to meet many of my readers and have received dozens of emails from all over the world, and the feedback I’ve received has been overwhelmingly positive.

u

Int

ro

du

ct

Ion

xxiii

In recent years, the matchmoving landscape has changed quite a bit and so, when I was asked to do a second edition, I was excited to be able to get the book back up-to-date. In this edition, I’ve taken the feedback and questions I’ve received over the years, and used them to expand the content and concepts covered in the book in a number of ways. I’ve tweaked the tutorials and updated them to current software and added a lot of new material.

Matchmoving programs such as SynthEyes and PFTrack that were relative new-comers during the writing of the first edition have become the program of choice at many facilities. So in the second edition, I’ve included them in many of the tutorials.

Perhaps the most prominent change is that I’ve added two completely new chapters. Object tracking, which garnered only a brief mention in the first edition, now has its own chapter that allows you to examine this complex subject in detail. I’ve also devoted an entire chapter to one of the newest phenomena to affect matchmovers—stereo.

What’s InsideMatchmoving: The Invisible Art of Camera Tracking is divided into 13 chapters:

Chapter 1, “The Basics of Matchmoving” A basic overview of what matchmoving is and how it fits into a typical visual effects pipeline. There’s also a tutorial for matching the perspec-tive of a still image.

Chapter 2, “Matchmoving Software” Chapters 2 through 5 all deal with matchmoving programs and how they work. This chapter lays the groundwork by explaining the processes used in photogrammetry and how they are implemented in these programs.

Chapter 3, “2D Tracking” As one of the first steps in a matchmove, 2D tracking can make or break a solution. This chapter breaks down 2D tracking and shows ways to make yourself a more effective tracker.

Chapter 4, “3D Calibration” Calibrating, or solving, can be one of the more frustrating parts of getting a good matchmove. In this chapter, I shed light on this sometimes mysterious process and provide ways to help get the best solution possible.

Chapter 5, “Automatic Tracking” Most matchmoving programs these days have automatic 2D tracking capabilities. In this chapter, I show how automatic tracking is used and how it differs from other forms of 2D tracking.

Chapter 6, “Cameras” This is a chapter entirely devoted to real-world cameras, because that is a primary concern for matchmovers. It covers everything about cameras that is impor-tant to a matchmover, including how cameras are similar or different from their virtual counterparts in animation programs.

Chapter 7, “Set Fitting” Getting the right camera is only half the battle with a typical match-move, because you still must fit the camera into any existing scenes you may have. This chapter shows you how to quickly and accurately place your cameras into your 3D scene.

Int

ro

du

ct

Ion

u

xxiv

Chapter 8, “Being on Set” If you’re ever asked to prep a set for matchmoving, this is an important chapter for you to read. I cover the roles and responsibilities of a matchmover on a film shoot as well as how to get the best information possible.

Chapter 9, “Matchamation” Matchmovers sometimes have to match more than just cameras. This chapter shows you how to match CG objects to moving objects in the plate.

Chapter 10, “Troubleshooting and Advanced Techniques” This chapter covers ways to help solve dif-ficult shots or shots that are just giving you problems. It includes a troubleshooting check-list and tips to coaxing a good solution out of mediocre data.

Chapter 11, “Object Tracking” This chapter covers the increasingly common process involved in tracking an object’s motion. It covers both regular object tracking and one of the newer tools in a matchmover’s quiver, geometry tracking.

Chapter 12, “Multipurposing Matchmoving Data” There are lots of ways to use matchmoving data besides calculating cameras. This chapter shows you some of the possibilities.

Chapter 13, “Stereo” 3D films have become the rage again, and this chapter show you how 3D films work and, more important, how to matchmove stereo footage. It includes a com-plete overview of the stereo process and the basic workflow of matchmoving in stereo.

I have provided some useful documents, a complete list of which you can find in Appendix A, “Resources.” I’ve also included a list of current matchmoving programs in Appendix B, “Matchmoving Software and Useful Links,” for those interested in know-ing more about them. Appendix C, “Film Formats,” is a detailed reference of camera formats, including film-back measurements and special considerations necessary when tracking digital formats. Finally, matchmoving comes with its fair share of jargon, so I’ve listed key words in a glossary at the back of the book.

As more and more directors add CG elements to their projects, there will be a greater need for matchmovers. Although matchmoving is truly an invisible art, I hope that this book will help bring to light many of the techniques that have languished in anonymity for so long.

How to Contact the Author If you’d like to contact me with feedback or questions, write to ectokon@gmail.com.

MatchmovingThe Invisible Art of Camera Tracking

Second Edition

u

Th

e B

as

ics

of

Ma

Tc

hM

ov

ing

1

1

The Basics of Matchmoving

Anytime computer-generated imagery (CGI)

needs to be placed into a live-action sequence,

or vice versa, a matchmove is required. But

what exactly is matchmoving? Matchmoving

is the process of matching CG elements into

live-action footage. As a result, it’s a crucial

part of many visual effects shots. Despite its

importance, it is completely invisible in the final

shot—that is, if it’s done right.

In this chapter, I explain the key steps of a

matchmove and how matchmovers work with

the rest of the visual effects team. I’ve also

included a tutorial that will help you become com-

fortable working with cameras and perspective.

Chapter ContentsExploring a Typical MatchmoveMoving from 3D to 2D and Back AgainUnderstanding the Matchmoving ProcessMatchmoving in the Production PipelinePerspective Matching Tutorial

Ch

ap

te

r 1

: T

he

Ba

sic

s o

f M

aT

ch

Mo

vin

g

u

2

Exploring a Typical MatchmoveTo better understand what matchmovers do, let’s consider a typical visual effects shot. Let’s say the director has called for a cg creature to crash out of a building’s window and run across the street and into an alley. Because the monster will need to interact with the window, the visual effects supervisor decides that in addition to the monster, the window-shattering effect should also be done on the computer.

on the day of shoot, the director makes artistic decisions as to how to shoot the scene and eventually decides on a camera position. There is an opening in the building where the window should be, although the panes of glass are missing. The director and the cameraman practice the camera move a few times and watch the video playback to see how it looks. When they are filming, they move the camera as though it were fol-lowing the monster crashing through the window and running across the street, even though the monster isn’t there. extras react to the imaginary beast, and props around the window are rigged with monofilament string (fishing line) to be pulled down on cue as though they were knocked over. When the director is happy with the shot, the film is sent off to be digitized and then given to the visual effects artists to add the monster.

When the visual effects studio receives the digitized sequence (known as a plate), an animator will need to animate the creature and a technical director (TD) will need to create the glass-shattering effect. and, of course, a matchmover will have to match-move the plate.

The visual effects artists’ goals are to make their 3D elements look as realistic as the scene that was filmed. The animator needs to make the creature move as though it were really crashing through a window, and the TD needs to make the window shat-ter like a real window. The matchmover needs to figure out where the camera was and how it was moving when the scene was filmed.

Matchmovers play an important role in this case, because in order for the crea-ture and window to appear matched realistically with the scene, they need to make sure that the cg objects are “filmed” the same way with their cg camera as the real set was filmed with the real camera. consider the window that needs to shatter—if the perspective of the window doesn’t match the perspective in the plate, the cg window will look out of place. furthermore, if the real camera moves to the left but the cg window stays put, everyone will know it’s a fake.

in our example effects shot, the on-set matchmover or data wrangler measures key items on the set. for example, she measures the size of the opening of the window as well as its height off the ground. she measures the distance across the street and the size of the opening to the alleyway. she draws a rough picture of the set and makes notes about positions of certain props and lights that might be useful to know. she also measures how high the camera is off the ground, what lens is used, and how far the camera is from the window.