Ideas Towards Musical Interactivity and Replacing Journey's Music- For Game Music Composers
Hi all. So this is part of my end of year portfolio for my master's degree at Wits.
Here is a quick disclaimer :P: I did not choose Journey because I thought I could do a better job, Austin Wintory is 20 million times the composer I am... my idea with this project was to address some fundamental flaws in approach to music design in games that I feel is most prevalent in Journey.
Not taking away anything from the soundtrack but I do feel it follows a more traditional, Western Classical, filmic approach to music composition which involves more developed musical parts that unfold over longer periods of time. The biggest flaw here is that sometimes a long, drawn out melodic line might not be finished before the next part needs to play and this causes some disjointedness either to the melody/music or to the flow of the game with the music... so in an attempt to address this I used a Journey video as a palette for new ideas.
Now I know writing static music to a set video doesn't exactly seem like the way to work on interactive elements in music, however my idea is that it could be quite useful for composers to do this.
Instead of writing music to a set tempo with strong form I decided to play along freely with the video and use a combination of loop based sections with longer melodic lines on top that are affected by what is happening on screen. Obviously since the video is the same each time it is not technically interactive but I feel the ideas for interactivity I feel can be locked down easier by doing this.
For example, in the first area there is one loop that constantly plays as the player moves down the slope. There is also a violin line that also descends with the player. As the player hits the ramps and flies into the air the violins either ascend or sustain to mimic the players actions. As the player stays in the air longer the violins move up in pitch but return to the original melody as the player hits the floor. Small things that are added are chimes and drums every time the player hits a ramp or lands and also a harp sound that sustains as the players remains in the air. That harp sound can also appear randomly in the music to make it less predictable. The idea here is to use smaller loop based sections that can be switched between quicker to make the music feel more interactive even if it is only in smaller pieces. The other idea I had was to prepare two melodies, one for the left and one for the right side of the slope that will be selected as the player progresses down the slope.
So i hope that explains the first slope scene and my ideas for interactivity. I feel like i have talked too much already so i'll leave the vid for people to watch and give feedback if they like :D
Here is a quick disclaimer :P: I did not choose Journey because I thought I could do a better job, Austin Wintory is 20 million times the composer I am... my idea with this project was to address some fundamental flaws in approach to music design in games that I feel is most prevalent in Journey.
Not taking away anything from the soundtrack but I do feel it follows a more traditional, Western Classical, filmic approach to music composition which involves more developed musical parts that unfold over longer periods of time. The biggest flaw here is that sometimes a long, drawn out melodic line might not be finished before the next part needs to play and this causes some disjointedness either to the melody/music or to the flow of the game with the music... so in an attempt to address this I used a Journey video as a palette for new ideas.
Now I know writing static music to a set video doesn't exactly seem like the way to work on interactive elements in music, however my idea is that it could be quite useful for composers to do this.
Instead of writing music to a set tempo with strong form I decided to play along freely with the video and use a combination of loop based sections with longer melodic lines on top that are affected by what is happening on screen. Obviously since the video is the same each time it is not technically interactive but I feel the ideas for interactivity I feel can be locked down easier by doing this.
For example, in the first area there is one loop that constantly plays as the player moves down the slope. There is also a violin line that also descends with the player. As the player hits the ramps and flies into the air the violins either ascend or sustain to mimic the players actions. As the player stays in the air longer the violins move up in pitch but return to the original melody as the player hits the floor. Small things that are added are chimes and drums every time the player hits a ramp or lands and also a harp sound that sustains as the players remains in the air. That harp sound can also appear randomly in the music to make it less predictable. The idea here is to use smaller loop based sections that can be switched between quicker to make the music feel more interactive even if it is only in smaller pieces. The other idea I had was to prepare two melodies, one for the left and one for the right side of the slope that will be selected as the player progresses down the slope.
So i hope that explains the first slope scene and my ideas for interactivity. I feel like i have talked too much already so i'll leave the vid for people to watch and give feedback if they like :D
Comments
Really enjoyed some of the motifs you had there. Two thoughts that came to mind: Having music directly related to the game's actions is the holy grail, but with some of the drums and chimes I almost felt it was leaning towards cheesiness. Also the bit to towards the end where the camera is side on and you can see the golden sunlight and then you get thrust into the chasm - that is one of the most poignant moments in the game (it's a key point in the monomyth narrative of journey). I imagine it's mostly because you ran out of steam, but I felt like the music wasn't supporting the epicness of that centre piece. I say this not as a criticism of your music, but because I think it's important to not lose sight of music's emotion for the sake of technical accuracy ;)
To respond to the running out of steam thing... I intentionally made the middle sparse. The reason for that was firstly to make it different from the original and secondly change texture from dense to less dense. I used subtleties like timing the sun passing through the pillars with the music and all sections are free timed so to fit with the tempo/flow of the game.
I personally tried to make it more diverse over smaller sectionss by using multiple keys and tonalities, where Wintory generally keeps most pieces in the soundtrack based around one chords or few sets of chords and the phrases and musical parts a generally much longer. I attempted to use this idea and work with melodic lines and not worry so much about chords at all.
To refer to the monomyth, sure that is part of the game Journey but my aim is use a video to brainstorm ideas for interactivity in linearish sequences... the monomyth has no relation to what I am doing here and if I were to copy Journey's original music or do something similar it would defeat the point of the exercise.
I guess also prepared hits of percussion could be considered cheesy but for me it is more akin to beat music games which to me aren't that cheesy. I feel the aural/tactile sense of audio responding to you is desirable and can be interesting if players choose to use different paths or play the game twice. These ideas lean more towards modernist music where music is not prepared but random or "found". I want to try work towards those concepts more than just writing a pretty music piece, which many people already do.
I'm not replacing or criticising Journey's music. I am suggesting ideas to compose music with interactive and narrative senses in mind and to evolve game music away from traditional formal structures.
I like the mickey-mousing you are doing and you have some moments that are very pleasing and remind me a lot of the FEZ soundtrack! The idea of having the musical elements interact more with the action on screen is definitely something that falls short in a lot of compositions today and I find that although it isn't always necessary, it can really enhance the experience. You have also carefully chosen your sound pallette with some very lush pads/synths that really bring out the element of it being a journey and they work well together!
Here are some quick suggestions:
To me, some chord changes might sound a bit odd, like at 3:00. The change seems a bit unsettling and I know that dissonants creates tension and suspense but I would rather have it stay lush and beautiful throughout and maintain the illusion of this beautiful world we are sliding through.
I have a few ideas regarding the sound design too:
Some of the sound fx feel a bit "out of place" and need to either be mixed in better or replaced with more subtle sounds in order to maintain this illusion/journey that you are creating with the music.
I like the running sand sound, it really helps create continuity throughout and is very calming...it sounds slightly mid rangey and could work well if it were a bit lighter (more in the high frequencies -type of sound).
Some of the bell sounds are overpowering the mix (probably because you have very little high frequency content) so I would EQ or mix them in a bit more.
Lastly, I would love to hear you recording the cello/violin lines with actual players, I can guarantee that it will make the piece sound even more emotional because you have that extra expression/human touch.
Hope I am making sense to you :)
Good luck with the project, you are definitely on the right way.
I get what you mean about the one change being unsettling, I did try to make it that way due to the dark look of the area that comes afterwards. I could maybe try and make the transition even smoother though. I'll see what I can do.
I definitely haven't spent as much time as I should on the SFX....I put in quite a few as just a place holder and haven't done much to them yet. I think I can spend a bit more time fitting them into the sound scape. There definitely isn't a lot of high range in the piece so maybe that is what is making certain SFX stick out. A bit of a low pass may help things out there.
It obviously would be idea to record the strings live however at this point I don't know many people who could do that for me... I'm keeping my ears out at Wits for people to help me for my end year recital and portfolio though :D If anyone knows anyone let me know!
Appreciate the feedback...it helps :D
Have you heard the interactive/dynamic music in Killer Instinct? It's the best example I can think of for music evolving directly in relation to what's happening on screen, and works really well! Here's a run-through by the composer Mick Gordon, of how he plans and what triggers each dynamic change:
I will watch this video now now and check it out! :D
You could use a certain loop as a spot sound effect, i.e. instead of having a crackling fire loop fade in/out as the player approaches/distances the flames of a camp fire you have a specific piece of music.
I think you could get around the jumping drums becoming too annoying by having multiple variations of samples, and perhaps even different types of drum depending on the terrain/surface the player is on - Wwise and FMOD has randomisation factors that can be applied to a single audio sample so that it sounds different each time.
I really like the idea of changing the pitch of a violin or other instrument according to the players jump height. Not sure how that could be implemented aside from a pitch RTPC, that might sound artificial. I guess spot effects layered by height might trigger different note as you rise/descend?
And things like EQ filters and effects as demonstrated in the video @Gibbo posted, are always cool, and can be pretty simple, like applying a high-pass filter when you're inside a structure, or extra reverb. Game states are also pretty straight forward - changing from one musical piece to another or adding additional parts depending on the players health or other parameter.
Check out the 'vertical' and 'horizontal' music systems that can be created in Wwise/FMOD.
I like the idea of replacing sfx with music and as I said this method of doing this is essentially a mock up and ideally I would want to evolve these ideas into FMod or the like.
I have read a more recent book that discusses vertical and horizontal music systems and I agree with the ideas whole heartedly. Before programs like Fmod existed horizontal and vertical music systems existed in modernist type music so that is where I am approaching it from. I am looking at older composers who use random factors in their music and compose pieces to never be heard the same way twice. and trying to turn these systems into something usable. Of course Fmod is the next step but I feel that would be enough for two whole master's project if I was to discuss Fmod along with compositional techniques and I am considering making an Fmod research project the next phase of my studies (PHD...if I get there)
I love all the ideas you have given me though and don't get me wrong, I believe programs like Fmod are infinitely useful and something I want to research more and use. I just think actual music composition in video games is neglected as a topic and still requires some thought apart from the technical side of things. Like for example a good musical idea can be turned into code by someone much like in Fez, where the programmer (Bedaud I think) added things to his proprietary music system {Fezzer} when Rich Vreeland gave him suggestions or ideas.
In terms of the violins ascending, the loops used for the string lines are so short that almost every 2 seconds you could end the one loop and trigger another loop to play, so my idea would be to have mutiple string samples, one that loops and then when certain factors are met in the game (character is in the air, character is in the air for 2 seconds) the next sample is triggered. In that regards I don't think you would need to do anything other than write multiple loops and assign them to parameters, rather than have the program change the pitch for you. I also did apply eqs to the music when the character enters tunnels and the like, you can see it in the first minute and half way through as well.
Thanks for the comments and apologies for the essay of a reply :P Appreciate the feedback from everyone! :D
Just on a mini side-line, I find the interactive vs. filmic/emotional debate very interesting, especially since early films, particularly animations went through a similar trial period ("Mickey mousing", as mentioned above), finally resulting in the the more emotional sided music, which we now come to expect in anything we watch, and also anything we play.
The interactive vs narrative debate is very interesting and if you want to read about it further my chapter from my Master's thesis about Journey deals with that debate. I can upload to Google Drive or something similar if anyone is interested in that discussion.
:D
I haven't finished the conclussion yet as I need to finish my practical component as it ties into my conclussion.
The chapters and introduction are there but need some editing still so excuse any mistakes (or point them out ;) )
Tell me if that works!