Compositing: Science and Nature

Compositing Science and Nature

What exactly is happening in a simple A over B composite? You're just laying one image over another, right? Nothing to itwas obvious from the first time you used Photoshop, right?

For most people, the basic compositing operation is a completely intuitive process that is rarely questioned, but to deal with the crucial part of an A over B compositethe edge detail of layer Ait helps to know what is going on, not only in the world of software but also in the real world as well. The two worlds do not operate the same, but it is the job of the digital world to re-create, as faithfully as possible, what is happening in the natural world.

Bitmap Alpha

A bitmap selection channel is one in which each pixel is either fully opaque or fully transparent. This is the type of selection generated by the Magic Wand tool in Photoshop. You can feature or blur the resulting edge, but the default selection has no semitransparent pixels.

This type of selection has its placesay, isolating pixels of one particular color range to change thembut it does not reflect how nature works. An edge made up of pixels which are either fully opaque or invisible cannot describe a curve or angle smoothly, and even a straight line with no edge transparency looks unnatural in an image whose goal is realism (Figure 3.5).

Figure 3.5. Using 400% magnification, you can see clearly how ugly a curved or angled shape is if it can be described by only bitmap pixels, pixels that are either fully off or fully on.

Feathered Alpha

But although it's easy enough to see that a bitmap edge does not occur in nature, it's hard to imagine that a feathered alpha does occurhard objects don't have semi-transparent edges, do they? Of course not. Look at the edge of this page, or anything in the foreground of your field of vision. Do you see semitransparent edge areas? No.

Not exactly, anyhowbut on the other hand, what you see isn't razor sharp in its hardness, either. It so happens that semi-transparent edge pixels are the best digital approximation we have for overlapping edges in nature (Figure 3.6) because they solve two problems in translating the world of objects to the world of pixels:

• They come closer to describing organic curves.
• They approximate the physics of light as it bends around objects.

Figure 3.6. Ahh, that's better. With a proper alpha channel and a one-pixel feather adjustment, even at 400%, the edge does a much better job of describing a smooth, soft curve.

Say what? The first point is easy enough to see and appreciate, but the second one is a doozy, an observation that can be traced back a century to Einstein's annus mirabilus, in which he demonstrated how mass and light are interrelated. Light is bentslightlyby any amount of mass that it passes. The greater the mass, the higher the amount of bending, right up to the extreme of black holes from which light cannot escape. With most objects, your eye sees a slight, subtle effect of the background light actually bending around the foreground object.

You can study this effect close-up in a digital photo with no compositing whatsoever (Figures 3.7a and b). In the digital image, you can see areas at the edge of objects that become a wash of color combining the foreground and background. This is caused by the influence of the mass of that object itself on the light (color) coming from behind it.

Figures 3.7a and b. Consider a simple digital photo that is not composited (a). When you look closely, you can see that there is a natural softness at the edges of the sign despite that it is in sharp focus and hard-edged. The effect is even more evident close up, if you compare the sharpness within the sign itself to the softness of the edge (b). (Images courtesy of Nathan Moody.)

Opacity

But we're by no means finished discovering surprises with basic image combination; the way that Opacity edits work in After Effects often takes people by surprise as well, although they can work with it for years without coming face to face with what is counterintuitive about it.

Close-Up: Geek Alert: The Compositing Formula

When you layer a raster image with semi-transparent alpha over an opaque background image, here's what happens: The foreground pixel values are multiplied by the percentage of transparency, which, if not fully opaque, reduces their value. The background pixels are multiplied by the percentage of opacity (the inverse of opacity), and the two values are added together to produce the composite. Expressed as a formula, it looks like

(Fg * A) + ((1-A)*Bg) = Comp

With real RGB pixel data of R: 185, G: 144, B: 207 in the foreground and R: 80, G: 94, B: 47 in the background, calculating one edge pixel only might look like

[(185, 144, 207) x .6] + [.4 x (80, 94, 47)] = (143, 124, 143)

The result is a weighted blend between the brightness of the foreground and the darker background.

Other effects compositing programs, such as Shake, do not take this operation for granted the way that After Effects and Photoshop do. You can't simply drag one image over another in a layer stackyou must apply an Over function to create this interaction. Is there a difference? Not until you add to the discussion the operations that go along with an Over, in particular premultiplication, which is detailed later in this chapter.

Say you have two identical layers, no alpha/transparency information for either layer. Set each layer to 50% Opacity, and the result should look exactly like either one of the layers at 100%, right?

Wrong (Figure 3.8)! If you've ever heard of Zeno's Paradox, it describes a phenomenon that is something like how opacity is calculated in After Effects, but for good reason (where as Zeno's actual paradox is pure folly).

Figure 3.8. A series of white solids overlaid with Normal blending at 50% over a black solid at 100%. Each layer only adds 50% to the delta (difference) between current opacity and 100%, so only the center begins to approach pure white (but registers 91.25%). This is consistent with Photoshop, but not with many other compositing applications.

A lead developer on the After Effects team once described the program's opacity calculations as follows: Imagine you have a light which is 1, and place a 50% transparent filter (say, a sheet of vellum) in front of it. Half the total light is permitted through the vellum (0.5 * 1 = 0.5). Put another 50% transparent sheet of vellum on top of that. Now half of half the light shows through (0.5 * 0.5 = 0.25). You can theoretically repeat ad infinitum without reaching 0% light transmission, at least in a pure digital environment.

Notes

Zeno's Paradox goes something like this: Suppose I wish to cross the room. First, of course, I must cover half the distance. Then, I must cover half the remaining distance. Then, I must cover half the remaining distance. Then I must cover half the remaining distance, and so on forever. The consequence is that I can never get to the other side of the room.

Hence, and in some tangential relationship to Zeno's Paradox, After Effects mimics how transparency behaves in the real world. This is not how opacity settings are handled in many alternative compositing applications, and so it often takes users of such programs as Shake by surprise, but the operation is by design.