Ideas Towards Musical Interactivity and Replacing Journey's Music- For Game Music Composers

Hi all. So this is part of my end of year portfolio for my master's degree at Wits.

Here is a quick disclaimer :P: I did not choose Journey because I thought I could do a better job, Austin Wintory is 20 million times the composer I am... my idea with this project was to address some fundamental flaws in approach to music design in games that I feel is most prevalent in Journey.

Not taking away anything from the soundtrack but I do feel it follows a more traditional, Western Classical, filmic approach to music composition which involves more developed musical parts that unfold over longer periods of time. The biggest flaw here is that sometimes a long, drawn out melodic line might not be finished before the next part needs to play and this causes some disjointedness either to the melody/music or to the flow of the game with the music... so in an attempt to address this I used a Journey video as a palette for new ideas.

Now I know writing static music to a set video doesn't exactly seem like the way to work on interactive elements in music, however my idea is that it could be quite useful for composers to do this.

Instead of writing music to a set tempo with strong form I decided to play along freely with the video and use a combination of loop based sections with longer melodic lines on top that are affected by what is happening on screen. Obviously since the video is the same each time it is not technically interactive but I feel the ideas for interactivity I feel can be locked down easier by doing this.

For example, in the first area there is one loop that constantly plays as the player moves down the slope. There is also a violin line that also descends with the player. As the player hits the ramps and flies into the air the violins either ascend or sustain to mimic the players actions. As the player stays in the air longer the violins move up in pitch but return to the original melody as the player hits the floor. Small things that are added are chimes and drums every time the player hits a ramp or lands and also a harp sound that sustains as the players remains in the air. That harp sound can also appear randomly in the music to make it less predictable. The idea here is to use smaller loop based sections that can be switched between quicker to make the music feel more interactive even if it is only in smaller pieces. The other idea I had was to prepare two melodies, one for the left and one for the right side of the slope that will be selected as the player progresses down the slope.

So i hope that explains the first slope scene and my ideas for interactivity. I feel like i have talked too much already so i'll leave the vid for people to watch and give feedback if they like :D


Thanked by 2MCA hermantulleken

Comments

  • Disclaimer, It's been a million years since I had any fancies of being a musician and it's really hard to give feedback on something that isn't interactive. However I was curious you found the music in Journey had flaws. That missed me entirely, and I wonder now how many other people noticed?

    Really enjoyed some of the motifs you had there. Two thoughts that came to mind: Having music directly related to the game's actions is the holy grail, but with some of the drums and chimes I almost felt it was leaning towards cheesiness. Also the bit to towards the end where the camera is side on and you can see the golden sunlight and then you get thrust into the chasm - that is one of the most poignant moments in the game (it's a key point in the monomyth narrative of journey). I imagine it's mostly because you ran out of steam, but I felt like the music wasn't supporting the epicness of that centre piece. I say this not as a criticism of your music, but because I think it's important to not lose sight of music's emotion for the sake of technical accuracy ;)
  • I don't think the music of Journey has any flaws as music but from my extensive research of the game and it's music I do believe that interactivity is sacrificed for narrative and vice versa and that is something Wintory even discusses himself. He explains that you can never quite get both to the same level in the same score, however I am trying to expand on his ideas and try to move away from Western Classical Filmic appoarches to game composition, which I believe to be stagnating or even taking a step backwards. I think music can evolve to be both interactive and narrative without being filmic and find new approaches to composition.

    To respond to the running out of steam thing... I intentionally made the middle sparse. The reason for that was firstly to make it different from the original and secondly change texture from dense to less dense. I used subtleties like timing the sun passing through the pillars with the music and all sections are free timed so to fit with the tempo/flow of the game.

    I personally tried to make it more diverse over smaller sectionss by using multiple keys and tonalities, where Wintory generally keeps most pieces in the soundtrack based around one chords or few sets of chords and the phrases and musical parts a generally much longer. I attempted to use this idea and work with melodic lines and not worry so much about chords at all.

    To refer to the monomyth, sure that is part of the game Journey but my aim is use a video to brainstorm ideas for interactivity in linearish sequences... the monomyth has no relation to what I am doing here and if I were to copy Journey's original music or do something similar it would defeat the point of the exercise.

    I guess also prepared hits of percussion could be considered cheesy but for me it is more akin to beat music games which to me aren't that cheesy. I feel the aural/tactile sense of audio responding to you is desirable and can be interesting if players choose to use different paths or play the game twice. These ideas lean more towards modernist music where music is not prepared but random or "found". I want to try work towards those concepts more than just writing a pretty music piece, which many people already do.

    I'm not replacing or criticising Journey's music. I am suggesting ideas to compose music with interactive and narrative senses in mind and to evolve game music away from traditional formal structures.

  • Hey man,

    I like the mickey-mousing you are doing and you have some moments that are very pleasing and remind me a lot of the FEZ soundtrack! The idea of having the musical elements interact more with the action on screen is definitely something that falls short in a lot of compositions today and I find that although it isn't always necessary, it can really enhance the experience. You have also carefully chosen your sound pallette with some very lush pads/synths that really bring out the element of it being a journey and they work well together!

    Here are some quick suggestions:

    To me, some chord changes might sound a bit odd, like at 3:00. The change seems a bit unsettling and I know that dissonants creates tension and suspense but I would rather have it stay lush and beautiful throughout and maintain the illusion of this beautiful world we are sliding through.

    I have a few ideas regarding the sound design too:

    Some of the sound fx feel a bit "out of place" and need to either be mixed in better or replaced with more subtle sounds in order to maintain this illusion/journey that you are creating with the music.
    I like the running sand sound, it really helps create continuity throughout and is very calming...it sounds slightly mid rangey and could work well if it were a bit lighter (more in the high frequencies -type of sound).
    Some of the bell sounds are overpowering the mix (probably because you have very little high frequency content) so I would EQ or mix them in a bit more.

    Lastly, I would love to hear you recording the cello/violin lines with actual players, I can guarantee that it will make the piece sound even more emotional because you have that extra expression/human touch.

    Hope I am making sense to you :)

    Good luck with the project, you are definitely on the right way.
    Thanked by 1Tim_Harbour
  • Thanks @Mexicanopiumdog

    I get what you mean about the one change being unsettling, I did try to make it that way due to the dark look of the area that comes afterwards. I could maybe try and make the transition even smoother though. I'll see what I can do.

    I definitely haven't spent as much time as I should on the SFX....I put in quite a few as just a place holder and haven't done much to them yet. I think I can spend a bit more time fitting them into the sound scape. There definitely isn't a lot of high range in the piece so maybe that is what is making certain SFX stick out. A bit of a low pass may help things out there.

    It obviously would be idea to record the strings live however at this point I don't know many people who could do that for me... I'm keeping my ears out at Wits for people to help me for my end year recital and portfolio though :D If anyone knows anyone let me know!

    Appreciate the feedback...it helps :D
  • Cool project man, an interesting concept and the example you created over the Journey video works well. Would be quite tough to implement such a deeply interactive music approach though in this kind of game without it getting annoying (having a drum beat every time you land for example could be too repetitive for people who like to bunny hop everywhere) - would imagine it works better for more scripted experiences or on a less detailed level.

    Have you heard the interactive/dynamic music in Killer Instinct? It's the best example I can think of for music evolving directly in relation to what's happening on screen, and works really well! Here's a run-through by the composer Mick Gordon, of how he plans and what triggers each dynamic change:
    Thanked by 2Tim_Harbour MCA
  • Yeah Gibbo I agree every time would be annoying. I think that is the purpose of the mock up to see what it feels like. It could work better perhaps if the sounds were played every 2-10 times and that number was selected randomly. Similar to what Fez does. But in general I guess that idea is towards how to craft music for an interactive setting..I'm just brainstorming and not proposing one outcome to the problem.

    I will watch this video now now and check it out! :D
  • I've not played around in Wwise/FMOD for a while and if you've not already I really recommend trying one or the other out to get a better understanding of game audio - theory is one thing but practice is another - and there's nothing stopping you from replacing traditional sound effects with musical motifs.

    You could use a certain loop as a spot sound effect, i.e. instead of having a crackling fire loop fade in/out as the player approaches/distances the flames of a camp fire you have a specific piece of music.

    I think you could get around the jumping drums becoming too annoying by having multiple variations of samples, and perhaps even different types of drum depending on the terrain/surface the player is on - Wwise and FMOD has randomisation factors that can be applied to a single audio sample so that it sounds different each time.

    I really like the idea of changing the pitch of a violin or other instrument according to the players jump height. Not sure how that could be implemented aside from a pitch RTPC, that might sound artificial. I guess spot effects layered by height might trigger different note as you rise/descend?

    And things like EQ filters and effects as demonstrated in the video @Gibbo posted, are always cool, and can be pretty simple, like applying a high-pass filter when you're inside a structure, or extra reverb. Game states are also pretty straight forward - changing from one musical piece to another or adding additional parts depending on the players health or other parameter.

    Check out the 'vertical' and 'horizontal' music systems that can be created in Wwise/FMOD.
  • edited
    I agree with you @MCA FmodWwise is the way to go in the future, and I am not dismissing it as the end goal for putting together truly interactive music. However due to the research I have done I feel there is a severe void in the aspect of music composition in games, 90% of writing if not more looks at the technical aspects such as ADSR, mixing, effects, basic concepts but none discuss ideas of pitch, form, harmony and phrasing which i feel is fundamental and never addressed. Instead people write Film-esque type music and try and cram it in Wwise/Fmod. I'm not saying it isn't useful and I'm not saying it's something I am not going to use but my research is about actual music composition and methods that can be evolved or created to help with the narrative/interactive divide.

    I like the idea of replacing sfx with music and as I said this method of doing this is essentially a mock up and ideally I would want to evolve these ideas into FMod or the like.

    I have read a more recent book that discusses vertical and horizontal music systems and I agree with the ideas whole heartedly. Before programs like Fmod existed horizontal and vertical music systems existed in modernist type music so that is where I am approaching it from. I am looking at older composers who use random factors in their music and compose pieces to never be heard the same way twice. and trying to turn these systems into something usable. Of course Fmod is the next step but I feel that would be enough for two whole master's project if I was to discuss Fmod along with compositional techniques and I am considering making an Fmod research project the next phase of my studies (PHD...if I get there)

    I love all the ideas you have given me though and don't get me wrong, I believe programs like Fmod are infinitely useful and something I want to research more and use. I just think actual music composition in video games is neglected as a topic and still requires some thought apart from the technical side of things. Like for example a good musical idea can be turned into code by someone much like in Fez, where the programmer (Bedaud I think) added things to his proprietary music system {Fezzer} when Rich Vreeland gave him suggestions or ideas.

    In terms of the violins ascending, the loops used for the string lines are so short that almost every 2 seconds you could end the one loop and trigger another loop to play, so my idea would be to have mutiple string samples, one that loops and then when certain factors are met in the game (character is in the air, character is in the air for 2 seconds) the next sample is triggered. In that regards I don't think you would need to do anything other than write multiple loops and assign them to parameters, rather than have the program change the pitch for you. I also did apply eqs to the music when the character enters tunnels and the like, you can see it in the first minute and half way through as well.

    Thanks for the comments and apologies for the essay of a reply :P Appreciate the feedback from everyone! :D
    Thanked by 1MCA
  • @Gibbo ... love that video. And for me it kind cements that the idea for me that the underlying musical concept needs to be good. In this case the game just has multiple loop attached to different game states (adaptive music) and it has music that happens in response to direct player action (interactive) and it's implemented as simply as different loops for different game states/stages. Really cool video, I love the music and the ideas :D
    Thanked by 1Gibbo
  • Yeah and on top of that Mick also works in the concept of music for SFX in the form of music 'hits' for every character's finishing ultra combo (unique to every stage, which is a lot of variations), works really well!
    Thanked by 1Tim_Harbour
  • @Tim, great to see some serious study going into the process of composition. Having done composition at varsity and playing around with FMOD and Wwise (mostly Wwise) I completely agree with your views that composition has been left behind by the technology. Have you had a look at Winifred Phillips' blog or her book A Composers Guide to Game Music? I haven't got hold of it myself, but it seems to be winning lots of awards, and looks like exactly what you're looking for info on. She covers both technical and compositional aspects.

    Just on a mini side-line, I find the interactive vs. filmic/emotional debate very interesting, especially since early films, particularly animations went through a similar trial period ("Mickey mousing", as mentioned above), finally resulting in the the more emotional sided music, which we now come to expect in anything we watch, and also anything we play.
    Thanked by 1Tim_Harbour
  • I'm pretty sure I've read most of that book @McCoySoundD but then again so many of them have similar names haha but I've read a bunch. I found there is some new literature such as this book which deals more with the compositional process but they mostly came out this year and last so I haven't really had time to look at them. I really want to.

    The interactive vs narrative debate is very interesting and if you want to read about it further my chapter from my Master's thesis about Journey deals with that debate. I can upload to Google Drive or something similar if anyone is interested in that discussion.

    :D
    Thanked by 1MCA
  • YES Please! Would love to see your thesis. Not just about that debate either.
  • I'll upload the chapters now and link them.

    I haven't finished the conclussion yet as I need to finish my practical component as it ties into my conclussion.

    The chapters and introduction are there but need some editing still so excuse any mistakes (or point them out ;) )
Sign In or Register to comment.