Fundamentals of Audio and Video Programming for Games (Pro-Developer)

In order to optimize the technical quality of your audio and video, you must have a clear understanding of the purpose and destination of the content. The quality decisions that you make must be driven by the creative intent of a work or a production. For example, to obtain a gritty quality, you may want a music track that contains audio distortion and unmotivated background noises. In fact, to achieve this effect, a total lack of distortion and noise may be exactly what you do not want. You must also be aware of the limitations imposed by your destination medium. For example, you have more flexibility with the wider dynamic range of CDs than you do with low-bit-rate streaming audio.

Optimizing is all about matching and consistency. It is more important that a clip match the quality of other clips and images within a certain work, than to have the best possible quality, because it is inconsistency that the viewer or listener will notice before a lack of quality. As stated earlier, the point of optimizing technical quality is to make it transparent, so that it does not distract the audience from the artistic aspect. For example, a high-quality , realistic, gunshot sound effect may sound bland and out of place in an action game with very stylized , animated characters . A sound that is nothing like a gunshot may actually work better, such as a distorted recording of hitting a large metal sheet with a baseball bat.

Matching means comparing the quality throughout a piece to a standard, especially your artistic standard. For example, you can ask yourself if the technical quality of a particular shot matches the quality standard that you have established for a project. The other standard is imposed by the medium, for example, a particular game platform. The limitations of a medium also impose a standard. For example, standard-definition video has a limited contrast ratio, requiring that a scene be lit more evenly in order to pick up all the detail in the dark areas.

Matching and consistency become most apparent in the editing room and in the game environment when you create transitions between video clips or attempt to mix sounds from different sources. For example, a user is likely to notice when you cut from a dark, bluish image to one that is bright and evenly colored, and that sudden shift may draw the viewer s attention away from your story or a point you want to make; in other words, that shift may take the viewer outside the experience that you are trying to create. Even though both images may be of technically good quality, it may be more important that they match and the transition appears transparent and seamless.

You should decide on your standard of technical quality, and then adhere to that standard. Maintain control, so that the technical quality of your production supports your vision throughout the creative process.

Quality in the Big Picture

In general, your original or raw , material should be as clean as possible. Then you can take the raw material and add effects, mix and composite it with other material, or modify properties to create a final product. For example, you should start with a clean voice recording, and then add effects in postproduction to create the sound that you want.

There are five opportunities during the life of a digital media production when you can optimize technical quality, as illustrated in Figure 13.3.

Figure 13.3: Five steps of the production process.

At each point in the production process, you have different options and reasons for optimizing. For example, when shooting with a camcorder, you will optimize in different ways than when you are in the editing phase.

You can optimize quality at the following stages in the production process:

1. Production. This stage involves recording audio and video. Production is the best time to think about technical quality, because your raw material sets the standard for the entire production.

2. Acquisition. This is when you convert or copy content to a hard disk. The process is also called capturing or digitizing. At this point, there are steps that you can take to ensure that the transfer retains as much of the original quality as possible. If you are acquiring digitally, the content is not processed because the data is transferred directly. Therefore, for digital acquisition, you do not have the opportunity to either improve or degrade quality.

3. Clean-up. If your raw, digital media data contains elements that should be removed or changed, it is often easier to work with the content before postproduction. If you start the editing process with clean audio and video, you can concentrate on the artistic aspects of your production rather than fight technical problems. Also, it may be easier at this point to identify content that must be rerecorded than to wait until after you start editing or mixing.

4. Postproduction. This stage involves editing, mixing, and processing the raw material into a final product. Assuming that you have done as much as possible to optimize the quality of your raw files, the technical optimization that you perform in postproduction will most likely focus on matching and consistency: tweaking the quality of your audio and video so that the audience will not notice the transitions, and so that your audio elements work well together.

5. Mastering. In this stage, you work with the final product before it is distributed to your audience. After your content has been edited, effects and processing applied, and the transitions smoothed out, your production should require very little additional work. Typically, any modifications that you make at this point affect the product as a whole, and are often necessitated by your destination medium. For example, you may need to raise the audio level of your entire video or decrease the dynamic range by using an audio compressor before recording the content to a CD. You may also need to make overall corrections to the color . If the content is to be used in a game, the audio quality of the many individual files must match. If files are to be played concurrently, audio levels must be adjusted so that they mix properly.

In the next section, we will look at the tools used for creating and working with audio and video.