22nd October '14 by Gary Robertson
Hey there guys and gals, it's been a while hasn't it? You remember me right? This time I get to tell you all about the other things I do here for Monstrum. Not only do I program the beasts that hunt you down, I also put Jaime's sounds into the game to creep you all out. So Hi, I might be the guy you love to hate as you play through it.
So how do the sounds of Monstrum get from our workspaces into your ears and get you involved in our game? By our own Audio System of course!
When we first started work on Monstrum we realised that we needed to do more than just play sounds in the right places to give you scares every now and then. We decided we would need to do everything we can with the sounds and use sound to its full potential. This includes cross-fading between various sound sources, audio being occluded by walls and also sounds that affect the game itself! So how does it all work?
After I import the raw sound files into the assets of our game we import those files straight into our audio system. Basically, the audio assets are all broken down into folders in Unity. These folders are then used to create Audio Libraries, with each folder being its own separate library. These allow us to edit all of the monster's roars all at the same time in one easy to use system.
Possibly the least scary screenshot of a roar to ever exist
Once this Audio Library has been created we can start to mold the sound into what we want it to be.
So do we:
(Audio Type) Want it to be a sound effect or a piece of music?
(Loop Type) Want the sound to loop or to only play once?
(Randomness Type) Want to randomize and pick a sound from the library at random or just list through them?
(Occlusion Type) Want the sound to be occluded by walls?
(Start Granular) Want it to play right away or play it after some time has passed?
(Start Volume) Want it to be loud or quiet?
(Start Pitch) Want to increase or decrease the pitch?
(Fade This) Want this sound to fade in or just play it right away?
The most eagle eyed of our readers will also notice that there are some randomness variables in there as well. These are to give us a bit more variation from a fewer sources, as pitch shifting a group of 10 samples is lighter than having a group of 50 hanging around!
These variables let us finely tune each sound's playback properties. When we call for a sound to be played within the game the audio source refers back to the Audio Library of the sound needing to be played and follows what the library tells it to do. Therefore our lovely monster roar behaves in a completely different way from your character's footsteps, falling over traps... being killed by the monsters. You know, all those things that you might do in the game.
Occlusion? What's That?
Monstrum is a procedurally generated horror game where you encounter a monster that occupies the ship you are stranded on. So, you are not the only thing on the ship making sounds, and these sounds may not be coming from the room you are currently in. It may be the monster walking about in the rooms close by, up the stairs, anywhere. By changing how you hear the sound you are given an idea of where that monster is. We do this by occluding the sound to make it sound quieter and more muffled depending on how many things are between you and the monster.
Every wall that is between you and the sound's current location will decrease the sound's volume and lower the cutoff point of a Low Pass Filter. The filter modifies the sound by stopping frequencies above a cutoff point from being played, only frequencies below that to pass through. This gives the impression of the sound being muffled by the walls. The more walls between you the sound means you will get lower volume and a more muffled sound. So when you are hiding terrified in your metal coffin (sometimes referred to as lockers) you will be able to say something like this:
“I HEAR IT! I HEAR IT! IT'S THERE! IT'S RIGHT IN FRONT OF ME!”
“It's to the right, but it sounds quiet and a bit muffled. I think it may be safe now”
I got scared a used a radio instead of a monster to demonstrate
Sounds That Affect The Gameplay?
All of the above just tells the game how to play the sounds, how does all of that affect how you play Monstrum? What's missing in the post is an extra little script not only allows you to hear the sound, but it allows the monsters to hear it as well! Some sounds have greater effect than others but all have the ability to alert them towards that area. During your time on the ship this feature could be a blessing or a curse. You could use it to your advantage and try to distract the monster to another location or you could just bring him right to you as you were making too much noise. No matter how you cause these sounds, expect to have company soon.
That will be all for now.
7th October '14 by Adam Dart
Phew! We've been polishing up the game for EGX London and now that it's over, it is time to update you all with another art blog!
We have been going back and polishing up many parts of the ship, making things look better where we rushed them, and adding in new transition parts to blend one area to the next.
Our old main stair-well went straight into the lower deck of the ship whilst retaining the upper deck theme. It was very immersion breaking when you'd walk straight from the crew living quarter themed hallways, straight to the rusty metal lower deck corridors so we've now gone back and added a proper lower deck stair well with an appropriate transition piece so that it blends the two sections together well.
Here are some screengrabs taken straight from Autodesk Maya...
I've also started thinking about how the exterior of the ship will fit all the sections together properly. Here are some mock-up shots of the exterior fit to a smaller size ship. This will most likely change a lot as there are still many considerations that I have not taken into account but it's a start to what might happen outside.
I've also considered changing the corridors around the container holds to make it more open when I have the time. These corridors leave a clear view into the container room to help the player identify where they are in the ship, however it means that more needs to be rendered in an area where framerate is already to poorest.
This issue will most likely apply for the exterior as well.
Here are some other shots of some rooms and assets that have been polished up to look a lot better than before.
New bathroom theme with new asset textures,
Polished up engine room textures and human sized piping additions,
Strange wall growths..... wait. What?
Polished up.... door .....fra.....
...do you hear something in the vents?
Adam and the Arts
1st October '14 by Jaime Cross
We're on to the second proper part of my little series of blogs covering the use of audio in storytelling for games. Previously on Team Junkfish I spoke about the use of music in games and how we're going about those ideas in Monstrum. This time I'm going to talk a bit more about the use of sound effects and how they build up the world and everything in it.
Before I start, here's a Gamasutra article by Andrew High on game music that is very much worth a read.
The IEZA Framework
Let's begin with some theory first and explain some terms that I'll be using later. I'll try keep it short! The IEZA Framework can be used to describe the sonic environment in a game, and is a good starting point when conceptualising the sound of a game. It looks like this:
Basically, it breaks down game audio along two axes: “Diegetic” and “Non-Diegetic” which we've covered before (primer: diegetic when something originates within the game world, such as footsteps, non-diegetic: something outside the world, like music with no in-game source) and “Setting” and “Activity” which relate to, in basic terms, the overall environment and mood of the game and the actions that can happen in it respectively. From there it gets broken down into four quadrants:
“Interface” - Cues that are used to provide information to the player, such as health pickups or menu selections. It lies on “Non-Diegetic”, as your character won't be able to hear you changing menu settings as it's disconnected from the in-game world, and “Activity” as the player is performing an action in order for the sound to happen.
“Effect” - Relates to things like footsteps, gunshots and other sonic responses to player actions. As such it lies on the “Activity” axis, with the sounds falling on “Diegetic” axis due to their in-game world origins.
“Zone” - Relates to the sonic space that the player currently inhabits and should be designed to replicate that environment. This lies on the “Diegetic” and “Setting” axes, as it has in-game origins, but it's not being driven by player actions. Think environmental sound effects, traffic, wind, that sort of thing.
“Affect” - This one is a bit nebulous, as it relates to audio that somehow “affects” the player, such as setting a mood via music. This could be something like an orchestral sting when a monster jumps out at you, or addressing an area you happen to be in (like bagpipes in Scotland, for one stereotype). This sits on the “Non-Diegetic” and “Setting” axes.
By following this framework we can define different types of sounds, and from there we can discuss how they assist in world building and story telling. For a more in-depth read check this article by Richard van Tol and Sander Huiberts.
So now that that's out the road, let's get down to business. I'm going to try break down the audio from a few games into these categories and explain how they help with world building and storying. So without much further ado...
Spoiler: It's not that scary.
Animal Crossing: New Leaf
Probably not a game that people would expect, but "Animal Crossing: New Leaf" (I'll use Animal Crossing in reference to this title from here on it) has some rather interesting and detailed sound design choices that help build up the world and the characters in it. In the game you play as the new mayor of a town, and it's up to you to bring in new residents and drive its development forward. The game rotates through a day night cycle in real time, and also has a weather system in place, both of which have an effect on the sonic background of the game. There are a couple of points that I quite like about the game, so I'll break them down into the IEZA sub groups where applicable in this.
Starting with some of the more obvious ones: footsteps! The easiest way to give somewhere a sense place is through footsteps (and reverb). Changing the sound for each surface is an obvious thing, but it it's still rather effective. Going from town's dirt ground, down a stone ramp to the beach and then across the shoreline with the sound changing beneath your feet just feels right. Even the carpets in your house sound different depending on their texture. Similarly, when the rain drifts into town the footsteps naturally change to reflect this, but they aren't alone in this regard. If you go digging in the rain the mud takes on a denser, wet sound as you throw it about and pack in holes. It's small, it's subtle, and yet it makes the game world seem a bit more plausable, a bit more believeable, for it.
Another interesting thing in Animal Crossing is the speech. When you talk to characters in the game they start “speaking” in a seemingly garbled language which, if you listen closely, seems to replicate what's in the text box! While any sort of chatter could have done this vocalisation, in addition to giving the townsfolk some degree of variation with their “voices” (although they aren't completely unique) adds to their own character. When Timmy or Tommy Nook are trying to sell you something and dip into their little quiet sales pitch, which is reflected in the text by a smaller font as well, you can believe that they're talking to your player. This also ties into...
A further nice little touch in the game is the text entry. As you enter each letter your character seemingly reads it out for you, giving them their own voice among the crowd. It's particularly fitting when another character asks you for an answer or you send out a message and you get to vocalise what is said (to some degree).
The town itself has some obvious zone related sounds effects, namely the beach area and when you stand near the river as you hear their respective flowing waters. But it also makes use of ambient sound to alert you to collectable insects, giving certain families or bugs their own calls so that you can not only tell where an insect is but what type it may be. This is a nice little touch, as the developers could have went with one stock insect noise but instead populates the world with a variety of life. It also makes it easier to try and find insects you haven't caught for your collection too. There are also items that float into the town on balloons, which tend to duck the music to allow a wind sound to play as the balloon comes into view. It is a small way to attract the players' attention to a focal point by utilising environmental audio.
On a side note, the 3DS actually has a surprisingly decent HRTF implementation with its speakers that doesn't get enough love or attention. Nintendo really went all in on the 3D! But implementation is another key part of game audio and is often overlooked by developers. This is, however, a blog post unto itself and has a fair amount of tech considerations too! So perhaps further down the line!
It's not the Old Grey Whistle Test, but still.
There are also a few instances of diegetic music in the game. Most obviously is Club LOL, the nightclub. It occasionally features the guitar playing canine troubadour K.K. Slider, on top of general club nights which he DJs under the psuedonym DJ K.K.. On Saturdays K.K. Slider will perform various songs, which you have to actively become the audience for by taking a seat, take requests and pass copies of tracks on to you to play in your house. Similarly, when travelling to and from the island you'll also be greeted with a little sea shanty from the boat's helmsman, Kapp'n. While the music remains the same, he tends to change the lyrics quite a bit, singing along via his synthesised voice.
Lastly, the use of music in Animal Crossing in terms of world and character building is pretty intriguing too. There is a “town theme” that each potential villiager has their own variation of. This little jingle plays whenever you talk to someone, and manages to put across some aspects of their personality. If you don't like the original you can actually change the tune that's played, allowing you to impart some of your own personality back into the town too! This is limited to 16 notes and the scale of C Major, but is still a nice feature that plays on the game's customisation and town building motif. If you play the game different times of the day you'll also get different background music, reflecting how active the town is.
There's also a little throwback/Easter egg in the Nook brother's T&T Mart, which riffs off of the instrumentation and themes from the Wii Shop Channel's music. Of course, no Animal Crossing game would be complete without “Tokata's Song”, a notable musical 19-note Easter egg by Nintendo sound designer Kazumi Totaka that appears in a number of different titles which he has worked on. He's also the source of K.K. Slider's name and the voice of Yoshi. The more you know.
These are only a few examples of Animal Crossing's well thought out and crafted sound design, and only from one game too. Another excellent example of using sound effects to build up the game world and story is ThatGameCompany's “Journey”. Sound designer Steve Johnson wrote a detailed breakdown of his process which you can read here. Visually the world of “Journey” is stunning, but the sound from both it and its inhabitants make really bring the game to life. The article also has sound examples, so give it a read! Similarly, the sound design in "Abe's Oddysey" and "Abe's Exoddus" allows players with little or no vision to navigate through and complete the game. I'll talk about that in the next major part of this series, which will have look at accessability and Audio games, and might even touch on the implementation side of things too.
Listed and Further Reading
Caleb Bridge - Creating Audio That Matters (Gamasutra) - Touches on audio immersion
Andrew High - Is Game Music All It Can Be? (Gamasutra) - Excellent, lengthy article on game music
Steve Johnson - The Sound Design of Journey (Gamasutra)
Darren Korb - Build That Wall: Creating the Audio for Bastion (GDC Vault)
Kirk Hamilton - How To Compose A Great Town Theme Song In Animal Crossing: New Leaf
Richard van Tol and Sander Huiberts - IEZA: A Framework For Game Audio (Gamasutra) - Breakdown and examples of the IEZA framework
IZEA Framework (Wikipedia)
Kristine Jørgensen - Left in the dark: Playing Computer Games with the Sound Turned off - Talks about how audio provides information and how a lack of sound changes players' perceptions. It's part of "From Pac-Man to Pop Music", a collection of studies covering a variety of different game audio topics.
Karen Collins - Playing With Sound - Another of Collins' books, this time on how players interact with game audio as opposed to listening to it, and how it can alter players' experiences.
Andy Farnell - Designing Sound - A technical book on sound design, and practically a bible for anyone interested in PureData and DSP.
David Sonnenschein - Sound Design - A more general sound design book, but covers a lot of the theory behind it very well.
Use Distinct Sound/Music Design (Game Accessibility Guidelines) - More of a segue into the next possible topic, it has examples of a blind player playing through "Abe's Oddysey" and beating Ed Boon at "Mortal Kombat" through sound alone.
9th September '14 by Andrew Bean
Today I thought I'd explain how the journal system in Monstrum is implemented. The journal is our equivalent of a quest/mission log in many other games, and has a few noteworthy features:
It is diegetic, which means it exists in the game world itself, instead of, for example, being rendered on a 2D plane in front of everything else. We chose a diegetic interface as it is generally more immersive, and also works better for the Oculus Rift.
The information in the journal is rendered on the page itself, following the curve of the paper. This makes the journal feel more natural than, say, having text float above the page.
In order to create the effect of having the UI rendered on to the page, the journal model is first split in to two - the cover and the page. This is so the two meshes can be UV mapped separately. The cover is just a standard model, but the page is UV mapped with (0,0) in one corner to (1,1) in the opposite. A separate camera is pointed at a custom UI system and draws to a RenderTexture that is displayed on the page. At this point, we can see the journal info on the page. The trouble is, we can't interact with it!
To solve this second problem, we have to start with the player. The player starts an interaction with any given object by looking at it. Therefore, we raycast along the camera forward, looking for the collision mesh of the journal page. If we miss, the player isn't looking at it. If we hit, how do we know where on the page they are looking? Well, Unity provides the texture coordinates of a hit from a raycast. These will be mapped between (0,0) and (1,1), as defined earlier by the UV map. All we need to do now is use the Camera.ViewportPointToRay function, and cast the returned ray, masking for our journal UI elements. At this point we can do whatever we like with the returned hit data (button clicks, mouseovers, positioning the journal cursor). Hooray!
I hope this technique may be of use to someone!
5th September '14 by Adam Dart
Once again, we've been working on some super cool stuff that we would rather keep secret for now. However, I thought I would share some interesting videos that may, or may not be related in some way or form. (Maybe even set the mood by watching the first 3 whilst listening to this playlist of the Alien soundtrack... )
1 – Macropinna Microstoma. A cool semi-transparent fish.
2 – Squid versus Fish
3 – Life in the Deep Sea
And one last one...
Take from this what you will.....
Adam and the Arts.