1

Keying introduction and resources

Note: When a background is not of a consistent and distinctive color, you can’t remove the background with keying effects. Under these conditions, you may need to use rotoscoping—the

manual drawing or painting on individual frames to isolate a foreground object from its background. About keying: color keys, luminance keys, and difference keys

Keying is defining transparency by a particular color value or luminance value in an image. When you key out a value, all pixels that have colors or luminance values similar to that value become

transparent. Keying makes it easy to replace a background, which is especially useful when you work with objects too complex to mask easily. When you place a keyed layer over another layer, the result

forms a composite, in which the background is visible wherever the keyed layer is transparent. You often see composites made with keying techniques in movies, for example, when an actor

appears to dangle from a helicopter or float in outer space. To create this effect, the actor is filmed in an appropriate position against a solid-color background screen. The background color is then keyed out and the scene with the actor is composited over a new background.

The technique of keying out a background of a consistent color is often called bluescreening or greenscreening, although you don’t have to use a blue or green screen; you can use any solid color

for a background. Red screens are often used for shooting non-human objects, such as minia ture models of cars and space ships. Magenta screens have been used for keying work in some feature films renowned for their visual effects. Other common terms for this kind of keying are color

keying and chroma keying. Difference keying works differently from color keying. Difference keying defines transparency with respect to a particular baseline background image. Instead of keying out a single-color screen,

you can key out an arbitrary background. To use difference keying, you must have at least one frame that contains only the background; other frames are compared to this frame, and the

background pixels are made transparent, leaving the foreground objects. Noise, grain, and other subtle variations can make difference keying very difficult to use in practice. Keying effects, including Keylight

After Effects includes several built-in keying effects, as well as the Academy Award-winning Keylight effect, which excels at professional-quality color keying. Keylight is not included with

the trial version of After Effects CS5. Keylight is included in the trial version of After Effects CS5.5 and later. Note: Though the color keying effects built into After Effects can be useful for some purposes, you

should try keying with Keylight before attempting to use these built-in keying effects. Some keying effects—such as the Color Key effect and the Luma Key effect—have been superseded by more

modern effects like Keylight. Note: Keep in mind that generating a high-quality key can require the application of mult ip le keying effects in sequence and careful modification of their properties, especially if the footage

was shot without considering the requirements of the compositor. Shooting and acquiring footage for keying

Keying footage that was not acquired correctly is much more difficult than is keying footage that was acquired with keying in mind. For best results, use Adobe OnLocation™ to monitor color and lighting as you acquire footage for color keying.

Light your color screen uniformly, and keep it free of wrinkles. Start with the highest-quality materials you can gather, such as film that you scan and

digitize. Use uncompressed footage (or, at least, files with the least possible amount of

compression). Many compression algorithms, especially the algorithms used in DV, HDV,

and Motion JPEG, discard subtle variations in blue—which may be necessary to create a

2

good key from a bluescreen. Use footage with the least color subsampling possible —for

example, 4:2:2 rather than 4:2:0 or 4:1:1. Tips for keying with After Effects

Noise and compression artifacts can cause problems for keying, especially difference keying. Often, applying a slight blur before keying can reduce noise and compression artifacts enough to improve keying results. For example, blurring the blue channel for DV

footage can smooth out noise in a bluescreen. Use a garbage matte to roughly outline your subject so that you don’t have to waste time

keying out parts of the background far from the foreground subject. Use a hold-out matte to roughly protect areas that are of a similar color to the background

from being keyed out.

To help you view transparency, temporarily change the background color of the composition, or include a background layer behind the layer you are keying out. As you

apply the keying effect to the layer in the foreground, the composition background (or a background layer) shows through, making it easy to view transparent areas.

For evenly lit footage, adjust keying controls on only one frame. Choose the most intricate

frame of the scene, one involving fine detail such as hair and transparent or semitransparent objects, such as smoke or glass. If the lighting is constant, the same settings you apply to

the first frame are applied to all subsequent frames. If lighting changes, you may need to adjust keying controls for other frames. Place keyframes for the first set of keying properties at the start of the scene. If you are setting keyframes for one property only, use Linear

interpolation. For footage that requires keyframes for multiple interacting properties, use Hold interpolation. If you set keyframes for keying properties, you may want to check the results frame by frame. Intermediate keying values may appear, producing unexpected

results. To key well-lit footage shot against a color screen, start with the Color Difference Key.

Add the Spill Suppressor to remove traces of the key color, and then use one or more of the other Matte effects, if necessary. If you are not satisfied with the results, try starting again with the Linear Color Key.

To key well-lit footage shot against multiple colors or unevenly lit footage shot against a bluescreen or greenscreen, start with the Color Range key. Add the Spill Suppressor and

other effects to refine the matte. If you are not completely satisfied with the results, try starting with or adding the Linear Color Key.

To key dark areas or shadows, use the Extract Key on the Luminance channel.

To make a static background scene transparent, use the Difference Matte Key. Add the Simple Choker and other effects as needed to refine the matte.

After you have used a key to create transparency, use Matte effects to remove traces of key color and create clean edges.

Blurring the alpha channel after keying can soften the edges of the matte, which can

improve compositing results. Use a garbage matte

A garbage matte (or junk matte) removes unneeded portions of the scene, resulting in a rough area that contains only the subject that you want to keep. When you are working with a poorly lit or uneven color screen (for example, a bluescreen or greenscreen), sketching a garbage matte around

the subject can greatly reduce the amount of work that you have to do in keying out the background. However, if you spend a lot of time making a perfect garbage matte that exactly outlines the

subject—essentially rotoscoping—you lose the time-saving advantage of keying. 1. Create a mask to roughly outline a subject. 2. Apply one or more keying effects to mask out the remainder of the background.

3. Apply Matte effects as necessary to fine-tune the matte.

3

Use a hold-out matte

Use a hold-out matte (also known as a hold-back matte) to patch a scene to which a keying effect has been applied.

A hold-out matte is a masked-out portion of a duplicate of a layer that you have keyed. The duplicate is masked to include only the area of the image that contains the key color that you want to preserve as opaque. The hold-out matte is then placed directly on top of the keyed layer.

Example of using a hold-out matte A.

Original bluescreen image. The background for the number is also blue.

B. After keying, the background for the number is also transparent.

C. Hold-out matte containing the part of the image you want to remain opaque

D.

When the hold-out matte is placed on top of the keyed image, the background for the number is now opaque.

1. Duplicate the layer containing the color screen. 2. Apply keying effects and Matte effects to the original layer to create transparency. 3. On the duplicate layer, create masks to mask out everything in the image except the area

that you want to preserve. 4. Make sure that the copy (the hold-out matte) is positioned directly on top of the keyed layer.

Note: Don’t change Transform properties of only one of the layers after making the duplicate; keep the layers moving together. Consider parenting one to the other.

Matte Choker effect

The Matte Choker effect repeats a sequence of choking and spreading the matte to fill undesired

holes (transparent areas) in opaque regions. The repetition is necessary because the entire matte must be choked and spread; the spreading fills the hole, but the edges of the matte must be choked back to preserve the matte shape.

4

The process of choking and spreading occurs in two stages, each with its own set of identical

controls. Typically, stage two does the opposite of stage one. After a specified number of back-and-forth adjustments (which Matte Choker handles automatically), the hole is filled.

This effect works with 8-bpc and 16-bpc color.

Original (upper-left) shows areas of unwanted transparency after using Color Key (lower-left) that are removed with Matte Choker (lower-right).

You can use the Matte Choker effect to only blur the alpha channel. To use this effect as an alpha-channel blur, set Gray Level Softness to 100%.

Close a hole in a matte

1. Select the layer, and choose Effect > Matte > Matte Choker. 2. Set stage-one controls (the first three properties) to spread the matte as far as possible

without altering its shape, as follows: Geometric Softness

Specifies (in pixels) the largest spread or choke. Choke Sets the amount of choke. Negative values spread the matte; positive values choke it.

Gray Level Softness Specifies how soft to make the edges of the matte. At 0%, the matte edges contain only

fully opaque and fully transparent values. At 100%, the matte edges have a full range of gray values but may appear blurred.

3. Set stage-two controls (the next three properties) to choke the matte by the same amount

as the amount by which you spread it in stage one. 4. (Optional) Use the Iterations property to specify how many times After Effects repeats the

spread-and-choke sequence. You may need to try a few different settings so that the sequence is repeated as many times as necessary to close any unwanted holes.

Simple Choker effect

The Simple Choker effect shrinks or expands the edges of a matte in small increments to create a cleaner matte. The Final Output view displays the image with the effect applied, and the Matte

view provides a black-and-white view of the image with black areas indicating transparency and white areas indicating opacity. Choke Matte sets the amount of choke. Negative values spread the matte; positive values choke it.

This effect works with 8-bpc, 16-bpc, and 32-bpc color.

5

Original (upper-left) contains unwanted edges after keying (lower-left) that are removed with

Simple Choker (lower-right).