Fundamentals of Audio and Video Programming for Games (Pro-Developer)

Using Visual Studio, open the High5 project file, in the AVBook\Audio projects directory, and then open the  High5.cpp file.

Timing

For the High5 sample, timing is simply used to delay an overlapping sound by a small and random fraction of a second in the playOverlappingSounds function. As a general rule, especially for games involving animation, timing is extremely important. It is one of the first things that you should address when designing an application. You must reconcile the variables of refresh rate for the screen and the simulated time that the application code is running to, and the variants of these that depend on the speed that the client machine can run your application. Since timing issues affect the structure of such an application so fundamentally, it can be a tough problem if it is not addressed early on.

The first functions in the source file handle timing. The basic timing functions in the Microsoft Win32 SDK do not have the precision we require “ each tick takes around 55 milliseconds , which is pretty useless for our purposes. Many developers, and indeed many Microsoft SDKs, instead use the timing functions that are available in the Microsoft Windows Multimedia SDK.

This is an ancient Microsoft SDK long slated for retirement, which is still around due to the usefulness of a few of its functions. The Multimedia SDK includes many early attempts at video reproduction on a computer, in the form of the Video for Windows (VfW) functions, which are also still in use despite their age. However, it does include a number of timing functions that provide fairly good millisecond accuracy. The most frequently used is the timeGetTime function.

The timeGetTime function returns the number of milliseconds since you started the Windows operating system. The functions in the  High5.cpp code that wrap this function are initMSTimer and getMSTime . The initMSTimer function stores the first time that the call is made, and the getMSTime function returns the difference between another call and the first call. This means that you can always get an accurate millisecond count since your application started. If you do a search for the use of this function, you will find it used throughout many Microsoft SDKs, since it is a reliable timing function.

There is a remote chance of a problem with timeGetTime : as the millisecond count is a DWORD value, it will wrap back to 0 every 49 days or so. For almost all applications, this problem falls in the won t fix category, but if you are a purist, you might want to amend the getMSTime function to cope with this case (by maintaining a count of 49-day iterations).

The Multimedia SDK run-time bits are included as part of Windows; to use them you need only add the header file, mmsystem.h, to your source files, and include the winmm.lib file for the linker.

Random Numbers

No game program is complete without some random numbers to add chaos to order. Randomness of some sort is essential in ensuring that every puff of smoke, clank of a sword, roll of a dice, or bang of a cannon are not identical each time that they occur.

The rand function returns a random number between 0 and the global variable RAND_MAX defined in the header file stdlib.h. However, to simulate a dice roll, the numbers required are clearly 1 through 6. To achieve this range, simply use the modulus operator (%) to return the remainder of the returned random number after division by six, and add one to change the result from 0 to 5 to 1 to 6. Purists will notice a tiny flaw in this math, in that if RAND_MAX does not evenly divide by six, then there is a small distortion in the results, however one that is too small for us to care about.

The function srand should be called to give the random number generation a variable seed, so that the same sequence of numbers is not given out. The most common way of doing this is with the following call.

srand( (unsigned)time(NULL));

This results in a different stream of random numbers each time that the program is run. For debugging purposes, when you might not want this behavior in order to try to repeat a bug, either set the parameter of srand to 1, or comment out the statement, and exactly the same sequence of numbers will be generated every time. Be sure to uncomment the line or change the parameter back when you are ready for prime time.

MFC or Win32

You will note that most DirectX samples are built up from Win32 calls and do not use MFC. Microsoft Foundation Classes (MFC) have turned out, in our opinion, to be somewhat more popular outside of the company than within. You can use MFC to build your applications, and can use the wizards available in Visual Studio to help you get started. However, our samples will follow the convention of coding directly to Win32, so you will not see any projects containing the output from MFC wizards.

There can be tricky issues if you start a programming project using MFC, and then wish to expunge it from the code. For example, the run-time library in the Properties dialog box (in the C/C++ Code generation section) usually needs to be single-threaded for a Win32 application, but multithreaded for an MFC-based application.

Our recommendation for audio/video projects is that if you are not a whiz in MFC, then stick with Win32 and leave learning MFC for a rainy day. A very rainy day.

Having said all that, notice that our WinMain function, the entry point to our program, simply calls InitCommonControls , then initializes random numbers, stores the instance handle in a handy global, creates a path to the directory containing the wave files, and then fires off the dialog box to do the rest of the work. Delegation at its best.

int APIENTRY WinMain( HINSTANCE hInstance, HINSTANCE hPrevInstance, LPSTR lpCmdLine, int nCmdShow) { // Initialize the common control dll. InitCommonControls(); // Use the time to seed the random number generator. srand( (unsigned)time( NULL ) ); // Store the value of the Instance handle. g_hInst = hInstance ; // Get the sound directory containing the dice sounds. GetCurrentDirectory(MAX_PATH, g_soundDir); g_soundDir[2] = 0; // Delete all but the drive. strcat(g_soundDir,"\AVBook\Audio\Boardgames\"); // Run everything else from the dialog box. DialogBox(hInstance, MAKEINTRESOURCE( IDD_HIGH5_DIALOG ), NULL, DlgProc ); return 0 ; }

Graphics

The High5 sample shows the kind of graphical masterpiece that would have looked fine in the age of games that involved space invaders and ping pong.

For a graphics-based game, which most games are, the rule is to push the hardware . If you are going to write a 3-D-animated game, and haven t delved into the world of pixel and vertex shaders, then you might want to consider becoming an expert in these skills. However, for our sound samples, we wish to minimize the graphics code in order to isolate the audio programming techniques. You will not learn much about graphics in this book, although it is interesting how passable a user interface can be developed from dialog boxes and character strings. Note that the sixFaces structure uses o s and the UI uses multiline edit dialog boxes to mock up the faces of dice.

In other words, dialog-based programs often provide a good base for tool development.

Callbacks

Callback functions are one of the great building blocks of Windows programming. There are many books on this subject, such as Programming Windows , by Charles Petzold, and it is a good idea to have such a reference available if you plan to use Windows features in your programming. The callback for the High5 sample is suitably straightforward.

int CALLBACK DlgProc( HWND hwndDlg, UINT uMsg, WPARAM wParam, LPARAM lParam ) { HRESULT hr.= S_OK ; switch( uMsg ) { case WM_INITDIALOG: OnInitDialog(hwndDlg); return TRUE ; case WM_COMMAND: switch (LOWORD(wParam)) { case IDC_RADIO1: SetDlgItemText(hwndDlg, IDC_ROLL6, "" ); g_Dice = 1; return TRUE; case IDC_RADIO2: g_Dice = 2; return TRUE; case IDC_RADIO3: g_Dice = 3; return TRUE; case IDC_RADIO4: g_Dice = 4; return TRUE; case IDC_RADIO5: g_Dice = 5; return TRUE; case IDC_ROLL: initMSTimer(); EnablePlayUI( hwndDlg, FALSE ); return TRUE; case IDCANCEL: EndDialog( hwndDlg, wParam ) ; return TRUE ; } case WM_TIMER: OnTimer( hwndDlg ); break; case WM_DESTROY: // Clean up everything. KillTimer( hwndDlg, 0 ); closeDirectSound(); break; } return FALSE ; }

Most of the callback function is self-explanatory, so we ll begin by looking at the OnInitDialog function, which is called when the WM_INITDIALOG message is received by the callback.

VOID OnInitDialog( HWND hwndDlg ) { HICON hIcon = NULL; g_hwndDialog = hwndDlg; // Load the application icon. hIcon = LoadIcon( g_hInst, MAKEINTRESOURCE( IDI_ICON1 ) ); if( hIcon ) { SendMessage( hwndDlg, WM_SETICON, ICON_SMALL, (LPARAM)hIcon ); SendMessage( hwndDlg, WM_SETICON, ICON_BIG, (LPARAM)hIcon ); } // Load the dialog defaults. CheckRadioButton(hwndDlg, IDC_RADIO1, IDC_RADIO5, IDC_RADIO1); g_Dice = 1; // Create a timer, so we can check for when the sound buffer is stopped. SetTimer( hwndDlg, 0, 250, NULL ); // Initialize DirectSound. initDirectSound( hwndDlg ); }

There is some pretty standard code for the first few lines of this function, then the dialog box defaults are set, and then there is call to SetTimer . This function will set a Windows timer going, and will send the Windows message WM_TIMER every time that the timeout value is reached. The first parameter is the ubiquitous handle to the dialog box, the second is an identifier (in case you want to use multiple timers), the third parameter is the timeout value, and the last one can be a procedure name that is called when the timeout value is reached. Windows sends the WM_TIMER message if this value is NULL , which is the most common use of this function. Note that the timeout value is in milliseconds, so the WM_TIMER message is going to be sent every 250 milliseconds (one quarter of a second).

The timer is deleted in the callback function when the dialog box is destroyed with a call to KillTimer , using the identifier (0) to indicate which timer is to go.

The next function we ll examine is the OnTimer function, remembering that this is called every 250 milliseconds.

VOID OnTimer( HWND hwndDlg ) { int result[5]; int d; int total; // If the shaking sound is required, then load and play it. if (rollStatus == status_shaking) { if (loadSound("DieShake.wav", g_Dice - 1)) playOverlappingSounds(g_Dice - 1); rollStatus = status_starting; } else // If the die roll sound is required, then load and play it. if (rollStatus == status_starting AND (g_Dice == 1 OR testSoundStopped() OR getMSTime() >= 1000)) { if (loadSound("DieRoll.wav", g_Dice)) playOverlappingSounds(g_Dice); rollStatus = status_rolling; } else // If the dice are rolling, then check to see if it is time // to present the results. if (rollStatus == status_rolling AND (testSoundStopped() OR getMSTime() >= 2000)) { total = 0; // For each die, retrieve a random number. for (d=0; d<5; d++) { if (d < g_Dice) result[d] = dieRoll(6); else result[d] = 0; total += result[d]; } // Display the numbers. SetDlgItemText(hwndDlg, IDC_ROLL1, sixFaces[ result[0] ] ); SetDlgItemText(hwndDlg, IDC_ROLL2, sixFaces[ result[1] ] ); SetDlgItemText(hwndDlg, IDC_ROLL3, sixFaces[ result[2] ] ); SetDlgItemText(hwndDlg, IDC_ROLL4, sixFaces[ result[3] ] ); SetDlgItemText(hwndDlg, IDC_ROLL5, sixFaces[ result[4] ] ); // If more than one die, display a total. if (g_Dice > 1) { SetDlgItemInt(hwndDlg, IDC_ROLL6, total, false); } // Update the UI controls to show the sound as stopped. EnablePlayUI( hwndDlg, TRUE ); } }

The first test checks a flag to see if the shaking sound should be started. If so, a call is made to loadSound , with two parameters: the name of the wave file to load and the maximum number of duplicate sound buffers that might be needed.

Notice that we are loading in the wave file each and every time that it is required. This is hardly efficient, but we do this so we don t have to address the creation of multiple CSound objects just yet. We have one CSound object, which we will be describing in just a moment, and it needs to be constantly fed with the required wave file and number of buffers that might be needed.

If the file loads correctly, then a call to playOverlappingSounds is made. The reason that the number of sounds is one less than the number of dice is simply that the shaking sound represents two dice colliding together, so that two dice only create one shaking sound. For three dice, we play two shaking sounds, and so on.

Following the initiation of any shaking sounds, a flag is set that indicates that the roll has started. The next time OnTimer is called, with rollStatus set to status_starting , we test to see if the shaking sound has stopped, and if it has, starts the rolling sound.

Notice this time that the number of rolling sounds is equal to the number of dice.

Finally, to end the OnTimer function, we check to see if the rolling has stopped, and if it has, it is time to update the dialog box and display the dice.

Throughout the previous procedure, we called or implied the existence of five functions that we have not yet explained, but will now discuss: initDirectSound , loadSound , playOverlappingSounds , testSoundStopped , and closeDirectSound .

DirectSound

The DirectSound utility code, dsutil .cpp, declares two very useful classes: CSoundManager and CSound . You only need one CSoundManager object to manage the audio system, and one CSound object for every sound that you wish to play. For this sample, we just have one CSound object (hence the need to keep loading the shaking and rolling sounds mentioned previously). This keeps the sample simple for now. The declarations are as follows .

CSoundManager* g_pSoundManager = NULL; CSound* g_pSound = NULL;

Note that there is a distinct difference between one sound (identified by one wave file), and one sound buffer . Each buffer is a copy of the sound, and can be mixed as a sound effect in its own right. So if one sound is created with four associated buffers, then four copies of the sound can be mixed (overlayed, panned independently, volume changed, processed for special effects, and so on).

initDirectSound

The initDirectSound function creates a new sound manager object, and initializes it if all goes smoothly.

void initDirectSound( HWND hwndDlg ) { g_pSoundManager = new CSoundManager(); soundWorking = false; if (g_pSoundManager != NULL) { if (SUCCEEDED(g_pSoundManager->Initialize( hwndDlg, DSSCL_PRIORITY))) soundWorking = true; } }

There is a whole range of errors that can occur when you are trying to initialize the audio system; for example, there can be no sound card, no compatible sound card, no DirectX run-time bits, and so on. However, for the High5 sample we ll simply set a flag on whether the sound system is working correctly or not. It is possible that some users of your application will not have the correct sound card up and running, but that should not mean that the application will grind to a halt. It just won t play any sounds.

The first parameter to the Initialize method is the inevitable Windows handle, the second is the cooperative level (how well this application will work with others that are running at the same time). Almost always, this is set to DSSCL_PRIORITY , which largely means that, when this application has focus, only its sounds will be audible.

The other cooperative levels that can be set are DSSCL_NORMAL , which restricts all sound output to 8-bit, and DSSCL_WRITEPRIMARY , which allows the application to write to the primary buffer. The primary buffer contains all the sounds to be played mixed together. Typically, your sounds are loaded into secondary buffers, and then DirectSound handles the mixing into the primary buffer. Use DSSCL_NORMAL only if you are nostalgic about poor-quality PC sound, and DSSCL_WRITEPRIMARY if you are writing your own mixer. Otherwise, stick with DSSCL_PRIORITY .

loadSound

The loadSound function checks to see that the sound system is working, and if so, clears the CSound object and loads in the requested wave file.

bool loadSound(char filename[], int nSounds) { TCHAR fullFilename[MAX_PATH]; if (soundWorking) { // Stop any running sound. if( g_pSound ) { g_pSound->Stop(); g_pSound->Reset(); } // Free any previous sound, and make a new one. SAFE_DELETE( g_pSound ); // Construct the full file name. strcpy(fullFilename,g_soundDir); strcat(fullFilename,filename); // Load the wave file into a DirectSound buffer. if (FAILED(g_pSoundManager->Create( &g_pSound,fullFilename, 0, GUID_NULL, nSounds ))) return false; return true; } else return false; }

The DirectSound SDK Stop method obviously stops a sound that is playing, and the Reset method resets the pointer to the sound buffers back to the beginning. This may be somewhat redundant in this sample, but it is usually good practice to go through this procedure when closing down a sound.

SAFE_DELETE is a useful little macro for deleting an object, and is defined in dxutil.h. After first testing that the object exists, SAFE_DELETE deletes the object and sets the pointer to it to NULL .

The next call does most of the processing required by loadSound.

g_pSoundManager->Create( &g_pSound,filename, 0, GUID_NULL, nSounds )

This call creates a new sound, taking the pointer to the CSound object, a file name for the wave file, some creation flags, a GUID identifying a 3-D-sound algorithm ( GUID_NULL selects the default), and the number of buffers to be created for the sound. The two parameters that require explanation are the creation flags and the GUID for the algorithm.

It is a very common error when programming DirectSound to not set the creation flags to match the sound manipulation that you want. Since we have set the flags to 0 for the High5 sample, nothing can be done to change the sound ” not the volume, not the frequency, and certainly not the panning or anything fancy.

For each sound effect, you should set the creation flags to match the processing that you might want to happen later on in the program. The reasoning behind the creation flags is simply one of performance. If DirectSound is notified in advance that very few, or no, processes (such as volume change) will be required, then all the plumbing required to support this processing can be ignored from the beginning, and greater speed efficiency is the result. The downside, of course, is that it is easy to forget to set the creation flags correctly. The following is a list of commonly used creation flags.

To set a combination of creation flags, simply OR them together, in a statement such as the following.

dwCreationFlags = DSBCAPS_CTRLVOLUME DSBCAPS_CTRLPAN;

For the sake of explanation, we will call sounds that require 3-D processing, 3-D sounds, and those that do not, 2-D sounds.

We will return to this topic for both 2-D and 3-D sound effects, but for our High5 sample, since there is no processing at all, the flags are set to zero.

The second parameter that we need to look at, the GUID for the sound processing algorithm, is easier to explain. For 2-D sound there really aren t any options, so just set it to GUID_NULL . However, for 3-D sound, there are a couple of options that trade performance for quality in some situations (see Chapter 3).

playOverlappingSounds

This function randomly waits up to one fifth of a second before playing the same sound over the top of any sounds that are already running.

void playOverlappingSounds(int nSounds) { int n; DWORD current, delay; DWORD dwFlags = 0L; if (soundWorking) { for (n=0; n<nSounds; n++) { // Wait randomly up to 1/5th of a second (200ms). delay = dieRoll(200); current = getMSTime(); while (getMSTime() < current + delay) {}; // No need to set volume, frequency or panning parameters // as creation flags do not support changing them from the // original recording. g_pSound->Play( 0, dwFlags, 0 , 0 , 0 ); } } }

The DirectSound Play method is an asynchronous call; it starts playing the sound, and then processing immediately returns to continue with the application without waiting for the sound to stop.

Notice that since we have set dwFlags to 0, all the parameters to the Play call are also set to 0. Yet, we still get the sound that we want, so clearly, the zeros do not mean zero sound.

The first parameter is reserved for future use (it will almost certainly never be used), and should be set to 0. The second flag contains the playing flags, not to be confused with the creation flags. However, there is only one playing flag defined, DSBPLAY_LOOPING , so set this flag if you want your sound to loop, or set it to 0 if you do not.

The final three parameters contain the required volume, frequency and panning position, assuming that the appropriate creation flags have been set to enable these. As this is not the case here, these parameters will be ignored, and are set to 0.

The Cacophony sample, described in the next chapter, gives a good example of the use of creation flags, and varying volume, frequency and panning positions .

testSoundStopped

This method simply tests to see if a sound has stopped playing.

bool testSoundStopped() { if (soundWorking AND g_pSound != NULL) { if( !g_pSound->IsSoundPlaying()) { g_pSound->Stop(); g_pSound->Reset(); return true; } else return false; } else return true; }

The IsSoundPlaying method returns a Boolean, and if the sound is not playing, it seems wise to call Stop and then Reset to set the pointer back to the start of the sound buffer.

closeDirectSound

This is a simple function to close things down. Note that you must delete the sound objects before the sound manager object.

void closeDirectSound() { if (soundWorking) { SAFE_DELETE( g_pSound ); SAFE_DELETE( g_pSoundManager ); } }