While the music industry is desperately trying to hardwire music to be a service, enticing consumers with freemium models and having consumers pay per music listened rather than produced, I wonder what would be the next step of music as an art medium.

Advancements

Interactivity

While video games are often considered evolution of movies into interactivity, the same thing can happen to music as well.

And within video games it already does happen.

The simplest form of it is picking different music tracks, probably most prominently music tracks associated with different locations.

A slightly more sophisticated variant is cross-fading between the music tracks of the same structure and motif, but varying intensity (a good example is FTL: Faster Than Light, that uses Explore or Battle variants depending on whether the player is in combat), normally through adding or removing instruments.

More elaborate sound designs incorporate certain flexibility into the music itself to make transitions sound natural at any given moment (e. g. Octopath Traveler, Ori and the Will of the Wisps).

The common theme in all these scenarios is breaking the static nature of the music and making it reflect the surrounding circumstances, enhancing the perception of these circumstances. It’s an evolution of the art of sound design in the movies, where the sequence of events is predefined, interactivity adds a whole new dimension of complexity.

Procedural generation

We seem to have already advanced science far enough to, in large part, generate reasonably pleasant music pieces (e. g. Abundant Music). And there seems to be significant interest in building music programmatically, seeing that the tools for this keep emerging (Sonic Pi, Overtone; okay, Sam Aaron is behind both, but I doubt he’s the only one).

And instrument sound synthesis has advanced far enough that there are successful composers that use synthesizers pretty much exclusively. Tastes aside, this has even resulted in new genres of music emerging.

The demise of MP3 players

In my neighborhood music is rarely played on devices designed exclusively for music playback. It’s usually smartphones, computers or vehicles’ “entertainment systems”. All of them have a lot more information to offer than music and maybe time:

• location information
• step count
• light conditions
• proximity to certain radio beacons, like Wi-Fi networks or Bluetooth trackers
• movement speed

As of right now we just ignore all these functions in music players! And when we don’t, we funnel some of the information from them to service providers against our intent (even if according to an agreement).

Context-aware music

We seem to already be in a good place to build music players that can take note of their surroundings and incorporate them into the music the user is listening to.

In its most primitive form, it can just cross-fade into certain tracks under certain circumstances, e. g. the user approaching specific types of locations (home, office, highway, park, etc.).

More advanced variations can make use of track stems to create a context-aware soundscape that adjusts the volume of individual stems in accordance with certain outside metrics, like the local weather, current speed, etc. This would already require composers to build special tracks, maybe with more stems that actually make up a complete track so that multiple “full” sound variations can be produced.

Further advancement may involve the use of algorithms (maybe AIs) to generate the music tracks with a lot more inputs, allowing for adjustment of not just the instruments, but also the melody, morphing it without interruption. Authoring tracks of this kind is going to be a new kind of craft on the verge between sound design and programming, similar to what video game sound designers currently do, but instead of accompanying adventures in a fictional world it would accompany day-to-day activities.

As is with transition from movies into video games, this opens up more creative freedoms, at the cost of making it harder to come up with a coherent creative work. Say, the correspondence between inputs of the algorithm and the features in the music don’t have to be disclosed, especially if we’re talking about a radio station, which can keep the listeners guessing what certain features of the music mean (and whether there is a meaning to them at all or they’re just based on random noise).

Let your imagination flow 🙂

Trial run

I remember when I got hold of the game soundtracks split into individual instruments, added them into simultaneous multi-track players and played with the volumes of individual tracks to change how they feel. I found that seemingly “full” tracks can provoke many emotional responses depending on how the instrument volumes are set up.

It was a fun experience.

Although I can’t really suggest any software today to give that a try, as Ambio, the Android player I was using is no longer maintained. You’re going to have to find one if you want to give that a try today. Audacity will probably work, and it’s FOSS.

I can suggest a couple soundtracks though.

• Chris Christodoulou is graciously offering his soundtracks for Risk of Rain and Risk of Rain 2 broken up into stems, instrument groups that when put together form a complete track.
• Auditorium by Cipher Prime Studios includes music already broken up into stems, because the gameplay relies on being able to play different stems with different volumes. You can look it up on YouTube to get a feel of what it sounds like (progressive soundtracks, playthrough of my favorite act).

Futurisms