Dynamic audio score/music

Question

I'm interested in developing a game who's background music changes with the mood and scenario of the game's action. Of course many existing games do this (halo for example), but I was interested in any resources/papers/articles talking about the techniques to develop a system like this.

I have some ideas, and I understand that this will be equally challenging to implement at the code level as it will be to come up or acquire music that fits this model. Any links or, answers with ideas in them would he appreciated.

Edit: this is the kind of info I'm looking for :) http://halo.bungie.org/misc/gdc.2002.music/

Answer 1

The earliest system I'm aware of is LucasArts's iMUSE system, used in most of the SCUMM games.

(That is a link to a patent document, but the patent has expired, leaving a very detailed description of how it worked available in the public domain. Hooray for the patent system actually working correctly for once!)

Answer 2

Create Digital Music has covered adaptive game music a couple of times; I think this article about Max/MSP and the Unity game engine is the closest to what you are looking for. It's also worth reading the comments, there are links to a lot more resources on this topic.
There is a more technical and in-depth article about game audio at Gamasutra, which discusses the use of Puredata, the open-source sibling of Max/MSP.

Answer 3

Here is a new interactive music system. psai Unfortunately not much detail as to the engineering design or content authoring pipeline at the time of this posting, however the resulting game-reactive music output is well demonstrated.

You are able to freely try out how things are done from an authoring-perspective in wwise or fmod designer, and these could be useful to assist initial composition your music assets while playing with different interactive authoring approaches before you commit to any particular design in your own engine.

There are many possible combinations of game actions and musical parameters that may be used to shape a composition interactively. Budgets of money time and target platforms are usually the limiting factors within which to scope an interactive music system.

From the audio standpoint you can produce the following types of music assets for different interactive music systems:

• Finished mixes of songs or song sections and transitions for end-to-end or cross-faded playback with optional timing metadata for exit / enter on-beat. This is the least amount of interactivity from a music-standpoint, but still it is sufficient for a great number of games.

• Finished mix stems of songs or song sections and transitions packaged as a multi-channel file or played together in sync to allow dynamic blending among the stems. These can also use timing metadata for on-beat switches and different realtime effects on the stems are now possible.

• Audio clips ranging from mix-stems down to individual instrument parts which are then arranged on an interactive music editor timeline to reconstruct the song sections and transitions. Such an arrangement provides much more granularity for bringing parts in and out of the mix as well as randomization and other dynamic effects within the library of parts.

The above have generally been limited to the base tempo and key as recorded in the audio files unless you allow dynamic bending of playback rate or implement a pitch-shift / time-stretch effect. Alternatively you could author a huge matrix of tempos / keys but that would be pretty wasteful from a data size and work effort standpoint.

• If you require the most granular level of music interactivity you can play back individual note sequences of wave data or synthesis algorithms. Here you can freely change arrangement, tempo, meter, key, instrumentation, synthesis parameters, mix and realtime effects for each part of the composition if you choose to build it all the way down to that level.

Answer 4

Noatikl from Intermorphic is generating some buzz and looks promising.

http://intermorphic.com/tools/noatikl/index.html

some history here:

http://intermorphic.com/tools/noatikl/generative_music.html

Answer 5

Ooh! ooh! I want to do something similar! Once upon a time, I used to make some music with ModPlug Tracker, a windows version of earlier music trackers found in the Amiga scene.

Instead of recording and mixing down the music and compressing it, as you do an MP3, you give the computer a set of samples and a sequence of instructions for when and at what pitch to play the samples. These would be in pattern blocks that would be ordered. Being able to change from one set of patterns into another, and adjust separate channels' volumes on the way, I think would be the way to go.

Originally, the small file sizes resulted in mostly chip tunes, but given the size of things these days, and maybe adding an audio compression routine to the samples may enable some fairly fancy sound.