After Effects CS5 saw several significant new features added, including Roto Brush and mocha for AE, which will be the focus of our next couple of posts.

(CS5 also saw the introduction of a couple of features that have since been discontinued, including Forge Freeform and Photoshop Repousse. If you’re still using an old version of After Effects and would like to see the movies on Freeform and Repousse, let us know in the comments and we’ll include them in this series.)

Roto Brush provides a software-assisted way to create an alpha channel to cut out actor or other feature from the background – even when it has soft edges. When it works, it provides as substantial time savings over masking by hand. However, it is not an instant gratification effect, and can be very frustrating to use and get good results from. We were among the beta testers for this feature, and worked closely with the developers to come up with the recommended workflow, which is covered in the six movies below.

The last (seventh) movie covers later changes in terminology for some of the parameters in Roto Brush, so make sure you watch that. In short:

• Smooth is now Feather
• Feather is now Contrast
• Choke is now Shift Edge

1) Overall Workflow

This explains how Roto Brush works, and lays out the steps to follow to get the best results when using Roto Brush:

2) Creating a Base Frame

Roto Brush needs a starting point to extrapolate from. This movie describes how to select a good base frame candidate, as well as how to create the best base frame to inform Roto Brush’s processing:

3) Propagating the Base Frame

Once you have good Base Frame, Roto Brush “propagates” that information to frames before and after the Base to update the Segmentation Boundary as the subject moves. Here’s how to optimize the Propagation settings, and make corrections when Roto Brush goes astray:

4) Propagation Gotchas

Several things can throw Roto Brush off, including objects that enter or leave the frame. Here’s how to deal with those:

5) Refining the Matte

Once you have a good Segmentation Boundary that follows your object from frame to frame, you need to refine the resulting matte (alpha channel), including dealing with motion blur, color contamination, and “chatter” as Roto Brush gets indecisive about defining an edge:

6) Application Ideas

So now you can more easily separate a foreground object from its background. What you are going to do with that? Here are some ideas:

Roto Brush Changes!!!

When Adobe added Refine Edges to the Roto Brush tool in the initial version of After Effects CC (after the movies above were created), they changed some of the terminology inside the effects dialog. This will update you on those changes.

These movies previously appeared on Lynda.com/LinkedIn Learning. They’ve retired this course from their library, so we’re making the movies from it available publicly for free. You can either scan our page on ProVideo Coalition to see the other free movies we’ve posted over the last couple of years, or check out the Crish Design channel on YouTube.