DirectX 9 Audio Exposed: Interactive Audio Development

"…to Nonlinear Music Format"

This exploration of DirectMusic as an architectural basis for a consumer-friendly nonlinear music format is based on more than just bright ideas and speculation. Over the past few years, Guy Whitmore and I composed and designed many examples of just such a format using DirectMusic. These compositions proved concepts and explored design methods blending various musical sources for demo and instruction purposes. We used familiar artists' material to demonstrate the commercial possibilities of nonlinear playback to music industry A&R people interested in new technology but understandably new to the concept of nonlinear music. Some examples: I used vocal phrase samples of Bjork in an original nonlinear composition to show how a club DJ remix artist might approach nonlinear composition. Guy arranged different and variable instrumental backing tracks with appropriate production value for a commercially released a cappella version of a Sarah McLachlan song to demonstrate a more traditional pop song approach within a nonlinear framework. He also recreated tracks from Moby's Play using DLS instruments and the same public domain vocal samples he used from the Lomax/Smithsonian collection.

We've also been fortunate to work closely with bands keen to explore the potential of nonlinear formats. One of them, ProjeKct X (Pat Mastelotto, Trey Gunn, Adrian Belew, and Robert Fripp) provided multitracks of an improvised session to generate musical material specifically for use in a nonlinear playback format. Not only that, but they have also generously granted permission for me to provide a small portion of that material here for demonstration and educational purposes.

All of the aforementioned demos consisted of discrete, individual files and a small accompanying player application designed to run on most any up-to-date multimedia-capable Windows PC. Though usually larger than the "average" MP3 file, a nonlinear Segment file is by definition not tied to the standard run-time/footprint ratio of a linear file. 10MB equals roughly ten minutes of run time for MP3, whereas a 10MB DirectMusic Segment may be designed to play anywhere for a minute or two up to an hour or even indefinitely until stopped by the user, if that's what the designer wishes.

If you haven't yet listened to the ProjeKct X Segment that accompanies this book, copy the ProjeKct X directory on the companion CD and copy it in its entirety to your C:\ drive. Open the directory that contains both the ProjeKct.exe and ProjeKctX.sgt files, and run ProjeKct.exe (they must be in the same directory for the player to run). The player will automatically load the ProjeKct X Segment on startup (allow some time for the files to decompress, depending on your CPU speed). There is also a WMA file provided as reference of the live linear capture of the piece from the studio. It's worth noting here that the material presented is only a fragment of what was a more than a ten-minute linear piece with three distinct movements.

As playing ProjeKctX.sgt demonstrates, the sound of DirectMusic can be as full and clear as Redbook audio or better if the composer wishes to use higher bandwidth source material. From the user's point of view a stand-alone nonlinear Segment appears to be just another sound file that, with a registered player app installed, only needs to be double-clicked to start. On first listen, it sounds like most any other music file. However, on a second, third, fourth, and even hundredth listen, the real difference between the DirectMusic Segment and a linear file becomes more and more apparent — not in sound quality of course, but in the actual experience of listening. In fact, one could say that there never really is more than "one listen" of a robust nonlinear music file because it can render a unique rendition with every play if the composer so desires. This is the essential difference between linear and nonlinear playback: A robust nonlinear music file continually rewards the engaged and active listener with every listen.

Some people on hearing or reading about nonlinear music for the first time form unrealistic assumptions and expect behavior along the lines of "Mary Had a Little Lamb" somehow magically playing itself as "Twinkle, Twinkle, Little Star." A little listening experience easily dispels such notions as most unlikely (not to mention undesirable). What generally happens with nonlinear playback is more like what happens when listening to different performances of the same piece of music. The nonlinear performance can even sound like different groups of musicians playing the same piece with genreappropriate stylistic variations if the composer provides enough material. Even if the composer designs a huge piece with wildly varying elements, the identity of any real musical composition stays recognizable in the same way that we can recognize Coltrane's rendition of "My Favorite Things" as the "same tune" that Julie Andrews sings in The Sound of Music. Simply put, a composition that has no identifiable elements that can translate across different settings and renditions is likely not a composition in the traditional sense. Conversely, if a composition is recognized by an intelligent listener regardless of arrangement or rendition, then the identity of the piece remains intact as some sort of metadata, whether the format is linear or nonlinear. For instance, it will not take more than one or two listenings of the ProjeKct X file or its original linear performance for the listener to recognize it within a short moment. This is true even though it was a group improvisation to start with and even though the melodic lines, words, and textures play back in a different arrangement every time in the DirectMusic version.