Fundamentals of Audio and Video Programming for Games (Pro-Developer)
The tools used to create and work with content are divided between those used in the following stages:
-
Production
-
Postproduction
Production tools should record and store good-quality sound and images. Postproduction tools should enable you to create your final product from raw audio and video content, and to work with the technical quality to ensure that the final product is consistent with your artistic vision.
Production Tools
In the production stage, you capture images and sounds on a recording device that saves them electronically as analog signals or digital streams to a medium, such as videotape. A typical recording device for video is a camcorder, so named because it consists of a camera and videotape recorder (VTR) in one unit. Camcorders also include a microphone and playback capabilities, meaning that a camcorder is a complete video production tool in one package.
There are many types of camcorders, for many budget sizes and purposes, from manufacturers such as Sony, JVC, Panasonic, and Canon. Camcorders are the most popular video-recording device, and there is a wide range of quality among all the brands and models. Again, it is best to start production with the best technical quality because it is not possible to add quality later. For example, if you shoot low-resolution video with an inexpensive camcorder, you cannot simply add resolution later. You might be able to add sharpness with an editor plug-in, but you cannot restore detail that was not recorded.
The following describes some aspects that affect technical quality to consider when choosing a camcorder for your production:
-
Number of chips. A microchip called a CCD and related electronic circuitry convert an image to an electrical signal. Consumer camcorders typically use one CCD chip. Camcorders created for professional use have three CCDs: one for each of the primary colors: red, green, and blue. With three CCDs, the camcorder produces images with truer colors and more color detail. Figure 13.4 shows examples of camcorders from Sony and Canon that lie somewhere between consumer and professional. They include three CCD chips, record in the MiniDV digital format, and provide high-quality audio and video at a price well below that of camcorders that are used in broadcast and professional video production.
-
Horizontal resolution. All camcorders and televisions of a given television standard have the same vertical resolution, which is measured by the number of lines that are used to scan one frame of video. The National Television Standards Committee (NTSC) standard consists of 525 lines. Horizontal resolution is also measured in number of lines, but it varies with the technical quality of a video source and describes the amount of sharpness and detail in an image. A professional camcorder produces images that exceed 700 lines of resolution, where a VHS videocassette recorder (VCR) has a resolution of only 240 lines.
-
Tape format. The type of format and tape used to record video affect horizontal resolution, color quality, and other video properties. Format describes not only the tape, but the hardware and system used for recording and playing the video and audio. A low-quality format, such as a consumer format, is more likely to produce an inferior image because the format was designed to minimize costs, and often size . For example, video recorded on 8 mm consumer tape is more likely to contain drop-outs than professional BetaCam tape, because the format itself specifies narrower tape and smaller, lighter moving parts . In general, the quality is better with digital tape formats, such as MiniDV and DVCam, and digital video can be transferred directly to a hard disk, thereby eliminating the conversion step that lowers resolution.
-
Manual control. All camcorders include a number of automatic functions, including auto-focus, auto-iris, auto white balance and others. Camcorders used by professional camera operators also include the ability to override the automatic functions, because the sensors that control the functions might not produce the desired effect. By having these manual options, you therefore have more control over technical quality.
Audio can be recorded in a number of ways. If audio is synchronized with video, it can be recorded directly on the videotape with a camcorder. If audio is to be recorded without video, it is typically recorded on an audio tape recorder, or directly to a hard disk on a computer using a digital audio workstation (DAW). One of the most popular audio recorders uses digital audio tape (DAT). One advantage of a DAT recorder is that the audio can be transferred digitally to a computer, with no loss of audio quality from the original. Not only do professional DAT machines record with very high quality, but the tape can also embed time code. Many professional video and film productions record their audio on DAT recorders, and then later synchronize the audio to film or video during postproduction. Figure 13.5 shows a portable DAT recorder from Tascam. Sony and Fostex are other companies that manufacture DAT recorders .
To pick up sound, you can use a high-quality condenser microphone. If multiple microphones are required, an audio mixer is used to blend the audio before it goes to tape. Shure, Sony and Sennheiser are several companies that manufacture good-quality microphones. Figure 13.6 shows a lavalier microphone from Tram that you can conceal underneath clothing.
The following list describes the general properties of microphones and recorders that determine their technical quality:
-
Internal noise. The electronic components in microphones and recorders produce a certain amount of noise, as measured by the device s signal-to-noise ratio specification.
-
Dynamic range. A recorder or mixer with a high signal-to-noise ratio enables you to record with a wide dynamic range, which is the difference between the quietest sounds you can record and the loudest sounds. A wide dynamic range enables you to record with more headroom , which is the range between the loudest sound and digital distortion. In other words, if you use a noisy recorder, you will have to record at a higher level to avoid hearing the noise, so the recorder will have less headroom, and distortion will be more likely.
-
Analog-to-digital converter. The quality of the hardware device that converts the analog signal to a data stream and back again is measured by how accurately it makes the conversion, in other words, how close the digital sound is to the original. A converter that performs oversampling is capable of producing a smoother, more accurate digital waveform because it takes more samples of the analog signal during conversion.
-
Transport speed and inconsistencies. The speed of the tape in a recorder must be accurate and consistent. In an analog recorder, you can hear variations in speed. In a digital recorder, the consistency of the tape speed is not as important because the data is buffered. However, the consistency of the clocking source is important. A digital recorder synchronizes the delivery of its data stream to a digital clock source. If the source is unstable, the device will produce a type of distortion called clock jitter .
Postproduction Tools
After production, the content is acquired or digitized to a computer hard disk or a digital disk recorder (DDR). After it is has been digitized, many of the tools used to create the final product are also used to improve the technical quality or clean up the audio and video. The tools are for the most part software (programs, plug-ins, or filters) that modify the content data.
To ensure that you can effectively manage technical quality, the first set of tools you need in a postproduction workstation is a high-quality video display and audio speakers. After installing the display and speakers , make sure that they are adjusted to a known standard.
To correctly adjust audio levels, use a mixer or audio program that provides accurate VU meters. A good-quality VU meter changes rapidly enough to correctly represent how the audio level fluctuates. It also indicates at what level distortion occurs. Always use the meter when capturing analog audio content, to make sure that levels are within the desired dynamic range without distorting. Figure 13.7 shows the VU meters in a DAW.
To accurately adjust video properties, you can use a software waveform monitor and vectorscope . These displays work similarly to the hardware devices with the same names . The waveform monitor enables you to correctly adjust luminance and setup; the vectorscope lets you adjust chrominance and hue (phase). If this software is not available, you will have to judge video properties by viewing the content on the video display.
If you transfer digitally, for example from a camcorder or DAT recorder, the data will not change as it is being written to the hard disk. Therefore, you cannot adjust audio and video properties during acquisition.
After the content has been stored on a hard disk, you can use the following types of programs to edit, process, and convert your content:
Video
-
Video editors. The primary purpose of a video editor is to provide you with a way to edit video content into short clips, and then arrange the clips on a timeline to create a finished video. However, most high-quality video-editing programs also include many features for processing video, which enable you to add effects and different types of transitions between clips, and to composite or layer clips. The features are often included as plug-ins or filters, and a number of them enable you to work with the technical quality of your video, such as color correction. Windows-based video editors are available from such companies as Adobe and Avid. Figure 13.8 shows the Vegas video-editing program from Sonic Foundry.
-
Digital effects programs. There are a number of professional-level programs that enable you to perform advanced digital compositing and apply effects, similarly to the way that you can work with still images in a graphics program. If your video requires this level of work, there are a host of third-party providers that you can contact. One such program is AfterEffects from Adobe.
Audio
-
Digital audio workstation (DAW). With these programs, you view the waveform display of an audio file and perform detailed editing and processing. The programs also include an audio-capture function and most employ good-quality VU meters. The programs typically include a number of processing features and enable you to use processing plug-ins, such as a noise reduction plug-in. You can also copy or cut and paste portions of the waveform. For fine control, many enable you to actually redraw portions of the waveform. This is often the best way to eliminate short transient clicks and pops. WaveLab from Steinberg, Sound Forge from Sonic Foundry and Cool Edit Pro from Syntrillium are examples of DAWs.
-
Multitrack programs. Use an audio workstation to make fine adjustments to a file, and a multitrack program to layer, synchronize, and mix multiple files and clips. Many multitrack programs enable you to move back and forth between it and a workstation, so that you can work with multiple tracks, and then open a piece of audio in a workstation to perform detailed work, and then move the file back to the multitrack program. Multitrack programs also include plug-ins so you can apply audio processing in real-time to one or more tracks. Steinberg and Sonic Foundry are examples of companies that offer multitrack programs for Windows-based computers.
Mastering
There are a number of tools that you can use to prepare finished content for distribution and to master content to CDs and DVDs. These tools enable you to design the layout of your media, and encode or create final distributable files. Most of these tools also include features for processing digital media for final distribution. Cleaner Pro from Discreet is an example of a tool for encoding multiple files for publishing streaming media files.
Summary
In this chapter, we explored methods and tools that you can use to optimize technical quality. The next chapter builds on the concepts and tools described in this chapter by showing you how to improve or maintain technical quality at each stage of the production process.