Fundamentals of Audio and Video Programming for Games (Pro-Developer)

DirectSound s most impressive abilities lie in the world of 3-D sound. Even a simple sound, such as an engine running, can wow your users if it appears to come from a moving car or plane, and especially if it appears to go over, under, or around the user to match the movement of the vehicle.

Enter the world of 3-D sound by going to the AVBook\bin directory and firing up the Rumpus 3D SFX tool. Due to the number of new concepts that this tool features, we have divided its autopsy into two chapters: this chapter covers 3-D sound, while Chapter 4 examines special and environmental effects. If you are new to 3-D sound, get ready to be impressed.

The Rumpus 3D SFX Tool

Like the Cacophony tool that flexes the 2-D sound capabilities of DirectSound, the Rumpus tool gives its 3-D capabilities a workout. There are a lot of similarities between these two tools, which are based on a very similar UI.

After you have clicked on the executable file, the dialog box shown in Figure 3.1 appears. The most notable difference between this tool and the Cacophony tool is the green grid on the Rumpus 3D SFX dialog box, which represents your 3-D world. The center of this world, marked by a white circle, is the listener. The listener is simply the location, and the direction that they are facing , of the person that is hearing the sound effects produced by DirectSound. In this sense, the 3-D features of DirectSound assume a first person model; it is assumed that the user of the application is playing a role within the application itself. In other words, the user is represented by the listener: they are not a spectator of the action but an integral part of it. The listener can move around in the same 3-D space as the rest of the objects in the game, although for our first few samples we will keep the listener stationary so that you can see what s going on.

Figure 3.1: Rumpus 3-D SFX main dialog box.

As you enter sounds into your 3-D world, numbers will appear on the grid (1 through 10) that represent the stationary or moving object with which the sound is associated. The grid only shows the movement of the objects in the x- and z-coordinates (so the grid represents a flat field). The y-coordinate (height) is not represented on the grid, however, the height of the object is modeled in all the underlying code, and affects the sound output. The Rumpus tool s graphics show the movement as if it was all on a flat surface, but the sound output is full 3-D.

For your first test, click the Load Rumpus 3D SFX File button, and then select the  TESTONE.rfx file. Now click Play. What you should hear is the sound of a train, first coming towards you, then passing on your left, and then variously bouncing around the 3-D world, passing both in front of and behind you. Not a bad effect for one simple wave file.

You should also see a small number 1 moving around the grid, marking the position of the sound, with the listener remaining stationary in the center.

When you are done listening to this first effect, click Stop. Now load up the Rumpus file  TESTTWO.rfx . This file simply expands on the first test, with the train now more fully represented by multiple sounds (those of the locomotive, numerous train cars , and so on). This gives a more realistic effect of the various sounds that would be heard along the length of a passing train.

Now click on the  TESTTHREE.rfx file. This is an identical scenario to  TESTTWO.rfx , except that now the listener is on the train and traveling with it. Note that the round white circle representing the listener moves along with the numbers representing the train. By the fairly simple act of giving the listener an identical movement to the train, you get quite a different sound experience.

Also try running the  CarShop.rfx file, which is an example of a file where all the sound slots are filled (see Plate 2).

Once you re done playing with these files, click Stop, and take a look at the UI. The ten time slots, the start time, the sound name , the Add or Edit Sound button, the Delete Sound button, the Play and Stop buttons, and the Load, Save and New buttons should all be familiar to you from the Cacophony tool. The Special Effects column is not used in the samples described in this chapter, but is discussed in Chapter 4. The Random Sound column shows the reference number and playing frequency of a sound that is to be triggered randomly by the initial sound. This is not a feature of DirectSound, but a feature of the Rumpus tool, and shows how effects such as claps of thunder can be generated by a consistent sound of steady rainfall. We ll discuss this in more detail during the discussion of the Add or Edit Sound dialog box later in this chapter.

The first three DirectSound parameters to look at are the Doppler factor, distance factor, and rolloff factor, currently all shown as 1.0 in the appropriate edit boxes of the main dialog box. These three factors are universal to all 3-D sounds. The value 1.0 means that the default values (those mirroring the natural world) are applied.

The Doppler Effect

The Doppler effect is certainly the most well-known audio effect, and is shown in Figure 3.2.

Figure 3.2: Doppler effect.

If a fast-moving object is moving towards you, it will appear to compress the sound waves that it is emitting, so that its sound will appear to have a higher frequency than it actually does. After passing you, the frequency will make a sudden change as the movement of the object away from you appears to expand the sound waves that the object is emitting. In this case, the object s sound appears to have a lower frequency than it actually does. If the object is not moving quickly, the Doppler effect is fairly unnoticeable.

Setting a value of 1.0 in the Rumpus tool for the Doppler factor applies the natural Doppler effect. If this value is increased, for example, to 2.0, then the effect is doubled , and appears to be twice as dramatic. On the other hand, if the Doppler factor is reduced, for example, to 0.5, then the effect is halved and will be less noticeable.

The theory behind providing these factors is that you can apply an exaggeration factor to overdramatize your sound effects, so typically, you would increase the Doppler factor rather than lower it. Try setting different values, and then running the  OnePlane.rfx file.

The Distance Factor

The distance factor is simply the ratio of your application s modeling unit for your 3-D world to DirectSound s modeling unit of meters. A distance factor of 1.0 means that all measurements are in meters, for a 1:1 relationship. If your modeling world is in feet (this is usually determined by what you have done in D3D graphics), then change the entry in the Distance Factor edit box to 0.3048. With the value set at 1.0, the 3-D world represented by our grid is a square that measures 100 meters by 100 meters .

The Rolloff Factor

The rolloff factor determines how quickly sounds diminish as they radiate from their source. Zero is the minimum value for rolloff factor; however, if this value is set to 0, then the sounds are not attenuated at all. A value of 1.0 means that the value mirrors the natural world. A value between 0 and 1.0 means that the sounds will diminish, or attenuate, at a lesser rate than normal, for example, a value of 0.5 for Rolloff Factor means that all sounds attenuate at half the normal rate. The maximum value is 10, which will attenuate the sounds rather quickly. See the discussion later in this chapter on minimum distance to see how to adjust rolloff for individual sounds.

Note that when you save Rumpus files, the settings for Doppler Factor, Distance Factor, and Rolloff Factor are not saved. You must adjust these universal factors each time.

Now, click on the Add or Edit Sound button.

The Add or Edit Sound Dialog Box

There is very little similarity between the dialog box in Figure 3.3 and the one with the same title in the Cacophony tool. However, one identical feature is the Sound File button. Click this button to load in a wave file. Now, note the sets of sliders on the left side of the Add or Edit Sound dialog box. The outermost set of three coordinate sliders places the object in the 3-D world. Note that the default position is dead center and at ground level (x=50, y=0, z=50). By moving these three coordinate sliders, you can place an object anywhere on the grid.

Figure 3.3: Add or Edit Sound dialog box for 3-D sounds.

The inner set of sliders gives the sound a velocity, with the default being no movement. If you have not changed the distance factor in the main dialog box, then speeds are in meters per second. Given that you cannot see height movement in the grid, try loading a sound, and then altering just the x- and z-coordinate positions and velocities. Note that a negative z velocity moves the sound farther down the grid.

You can also set the start and stop times for a sound, select the check boxes to activate one or both of these times. Like the Cacophony tool, the entire playing time is 100 seconds, but the granularity is in tenths of a second (rather than eighths), so times can be entered accordingly .

To the right of the start and stop times are the Random Sound Number (N) and the Frequency (F) boxes. If the N box is 0, then no random sound is heard. If N is a number between 1 and 10, then the sound in that numbered slot will be initiated, on average, the number of times per 100 seconds that is entered in the F box. For example, a value of 5 in the F box indicates that the random sound is fired off, on average, 5 times per 100 seconds, or once every 20 seconds. This feature is especially effective with natural effects such as adding a clap of thunder to steady rainfall, or the sound of creaks to a rocking ship, or squeaks to a moving piece of machinery.

Minimum Distance

Now look at the upper-right corner of the dialog box, at the Minimum Distance box. The theory behind this slightly confusing setting is based on the audible range of the human ear.

If the object producing this sound is within the minimum distance, measured in meters, then the sound will no longer increase in volume, but will remain at its maximum. By default, this value is 1, meaning that anywhere within one meter from the listener, the sound will be at its maximum volume. However, for very loud noises (for example, the roar of a jet engine, an explosion, or the blast of just sitting too close to the speakers at a rock concert) this distance could be 10 or 100 meters.

Figure 3.4: Relationship between minimum distance and volume.

Figure 3.4 shows that the minimum distance not only determines the radius around the listener within which the volume does not change, but also how the sound attenuates over distance. A value of 1 means that at 2 meters, the sound will be at half volume, and at 4 meters, at quarter volume. A value of 0.1 means that the sound will be at half volume at 0.2 meters. Similarly, a very loud object, with a minimum distance of 50 meters, will be at half volume 100 meters away, quarter volume at 200 meters, and so on. Remember that this value is combined with the global rolloff factor to give the final attenuation of a sound.

There is also a value in the DirectSound SDK that sets the maximum distance at which a sound can be heard. This turns out to not be very useful, is not represented in the Rumpus tool, and can usually be ignored.

Ambient Sounds

An ambient sound is not a 3-D sound, but a sound that is heard everywhere in the world at equal volume. Typically, this represents sounds such as rainfall, although many games implement background sounds for settings such as railyards, airports, natural environments, and so on, that work very well. One of the tricks to ambient sound is to play them at such low volumes that the human ear is incapable of determining whether it is a continuous sound or a short sample that is looping, yet is able to identify the sound as coming from a certain source.

Selecting the Ambient Sound check box disables most of the other entries in the dialog box because no 3-D positioning or movement applies. The Outside Volume setting gives an ambient sound a volume as a percentage of the recorded volume; typically, this is 100 percent.

As a piece of advice to game developers, ambient sounds and 3-D sounds work very well together.

Sound Cones

The sound cone provides an interesting set of features to DirectSound, although you should judge whether the 3-D-graphics world that your players are moving in justifies such precision in the sound output. The theory is that sounds do not emanate at equal volume from their point source, but tend to have a direction, as shown in Figure 3.5.

Figure 3.5: Inside and outside sound cones.

For example, when you speak, the sound is at a greater volume in front of you than behind. The Inside Cone setting gives the full diameter in degrees of the cone where full volume applies. So if this value is 90, then full volume occurs 45 degrees to either side of the direction of the sound source. The Outside Cone setting gives the full diameter of the cone outside of which the Outside Volume applies. Anywhere between the inside and outside cone settings, the sound volume is attenuated evenly. For example, if the Inside Cone is set to 90, the Outside Cone set to 180, and the Outside Volume set to 50, then the sound is at full volume up to 45 degrees from the facing of the source. The sound then attenuates evenly from full volume at 45 degrees down to half volume at 90 degrees from the facing of the source, after which the sound is at a consistent half volume.

Another feature of sound cones is that they can be made to face in any direction relative to the source object. The three radio buttons shown in Figure 3.3 provide the most obvious directional options; the first (the default) simply points the sound cone down the z-axis, so that an object placed at the top of the grid will have a cone facing towards the listener in the center of the grid (which, incidentally, has a default facing of up the z-axis).

Probably more useful are either the forward-facing or rear-facing sound cones selected by the other two radio buttons. Human speech is an example where a forward-facing cone is appropriate, while a rocket engine would require a rear-facing cone. There are also examples where a sound cone faces out to the side of an object or continuously rotates (such as an emergency vehicle s siren). The DirectSound SDK allows you to implement this (as does the Concertina framework code described in Chapter 8), but the Rumpus tool does not provide this option.

Experiment with adding sound cones to the train and plane Rumpus files to get a good idea of the concept of directional sound cones.

For now, ignore the Special Effects group box in this dialog box. We have a lot of 3-D code to dissect without making it more complicated by adding special effects. We ll examine these effects in great detail in the next two chapters.

The Rumpus tool is similar to the Cacophony tool in that clicking OK will add the sound and all the settings to the list in the main dialog box, while clicking Cancel will ignore any changes that you ve made.

Now, back in the main Rumpus dialog box, click on the Edit Listener button.

The Listener

Figure 3.6: Listener in a 3-D world.

In the Rumpus tool, and also in the DirectSound SDK itself, the listener can be given a similar set of 3-D position properties to those given to sounds, as shown in Figure 3.6. After clicking on the Edit Listener button, you get the Add or Edit Sound dialog box, but with only the position and movement sliders available. By default, the listener is in the center of the grid, and not moving. However, the listener can be placed anywhere within the grid, and can be set to move, and perhaps given the same position and speed as one of the sounds. The listener is not attached to a sound in any way, but can simply be given identical position and movement parameters. For example, to simulate a person traveling on a train, the listener is given the same starting position and same velocity as one of the railroad cars of the train.

Any listener information that you enter is saved in a Rumpus file, along with the sounds and their settings.

The Rumpus Files

Now might be a good time to go through and play all the Rumpus files located in the Rumpus SFX files directory (except for those provided in the Special Effects sub-directory, which will be explained in the next chapter).

When saving a Rumpus file, only the non-zero sound settings are saved. Similar to the Cacophony (.cac) files, the file is saved in text format, but this time with an .rfx file name extension, so that it can conceivably be edited in Microsoft Notepad or another text file editor. However, since the .rfx files have a large number of settings, they are not as understandable as the .cac files.

If you want a multichannel sound (for example, 5.1, 6.1, 7.1) to play using the Rumpus tool, then load it in as an ambient sound. Try loading the Rumpus file  Paddling.rfx for an example of a 6.1 sound. The multichannel wave files need to be loaded as ambient (2-D) sounds because no 3-D processing can be applied to them.

Since multichannel wave files are usually very large, you will notice a considerable start latency when running the  Paddling.rfx file. In order to use multichannel files effectively, you should stream them in (see Chapter 6), rather than load them into static buffers. Compare the start latency when running  Paddling.rfx in the Rumpus tool with the start latency when loading  PaddlingInCircles.wav into the Circular Streams sample of Chapter 6. Streaming is definitely the way to go with very large files.

If you want to create your own multichannel wave files, see Appendix B on the multichannel Audio Tool (MCAudio.exe) and use it to make 5.1 or 6.1 files. It is not an easy task to make meaningful multichannel sounds; you will probably find that you need the assistance of a good sound-editing tool.

However, perhaps a better use of multichannel output is that some audio cards will process the 3-D positioning and output the sound correctly out of the six or seven speakers.

This completes our discussion on the Rumpus user interface. Next, we ll load the Rumpus 3D SFX project, and delve into the code.