Sound Design as World-Building: How to Create a Believable Universe With Sound

Sound is half the presentation of a movie, but it is always in service to the whole.

Sound design working at its best seamlessly describes, enhances and elevates the image, giving weight, character and detail to all we see. It creates the universe in which the story is taking place, further extending the reality on screen.

Sound must feel right. The audience should believe that the sound originates with what they see on screen, as if the things they’re seeing—even the most unbelievable—are actually real. That’s what we mean when we say a sound “glues onto” an image.

I have had the great honor of working on three films for Sony Pictures Animation: Cloudy with a Chance of Meatballs, Hotel Transylvania and Cloudy with a Chance of Meatballs 2—films in which sound played a huge role in establishing world and character. I work in exactly the same way for a live action movie as I would an animated one—in animation, the physics and scale are distorted and exaggerated, but it’s the same basic process. What is so wonderful and terrifying about an animated project, though, is how much sound is not present when you start. The picture editorial team has been working to create the basic track of the film, sometimes for years, but there is always just so much to do. Where do you start?

The Script Comes First

How directly a script relates to the current state of a picture can vary enormously. But no matter how outdated the script is, disregard disclaimers about “it’s changed so much” or “they have been improvising every scene,” and read it. That is the thing that was green-lit. That is what started it all.

So often I get small clues from the writer with regard to tone and intent, setting and mood—hidden gems of information that you might otherwise miss. As you read it, let your mind wander and jot down what pops into your head. Sometimes I end up writing visual words like “dark” or “red” in the margin. Or I will reference a real-world item (e.g. “hippo”) or an abstract feeling (e.g. “elegant speed”) that popped into my head. I create a document of questions or ideas, and these notes are easy to hand out to team members to give direction or ask for input.

The words on the page, and your “mind’s eyes and ears,” can be a very powerful combination. Key ideas and directions can emerge without the prejudice of images driving your creativity.

What is its Essence? What Does it do for the Story?

Decide at the beginning what the overarching theme or tone for the film is. Futuristic? Organic? Period? Metallic? Stone? Loud? Confusing?

When designing for specific character, or an item, use the same analytical process. Is it large or small, cute or terrifying, mechanical or organic, realistic or fantasical?

Consider the scale. Maybe an object is small but has enormous mass, or it is huge with little mass. Also, does the scale shift over the course of the story? What is the sonic center of the item? Can it be distilled down to one basic idea or overall essence?

With complex, moving things, there may be multiple centers of sound. Sometimes the various states of the thing have nothing to do with each other. The sound a vehicle makes in its optimal motion state really has nothing in common with the sound that vehicle makes when it comes to a sudden, crashing halt on the side of a building. Looking for and identifying the sonic center can be very valuable, because sometimes it becomes the thing to avoid.

Go Organic

I am a firm believer in organic, both in food and sonics. Find a real-world analog for your problem. Thinking about how a similar item works or sounds can lead you to an answer. A familiar sound played out of context against an unfamiliar image may make the thing more acceptable to the audience.

Cody Cameron and Kris Pearn, the directors of Cloudy 2, wanted the “foodimals” (food animals) in the movie to sound distinctly natural and organic, in contrast to the mechanical villains. From this starting point, one of the first orders of business was to record a new library of organic food sounds. We spent a day in a controlled recording environment with a huge array of fruit and vegetable items. These were recorded to Pro Tools at 96 Khz, 24 bit. We manipulated the raw materials in every way we could imagine to see what sonic treats we could uncover, using newly acquired microphone preamps from Pueblo Audio (puebloaudio.com ).

My favorite moment of the day came when sound editor David Werntz slowly peeled a banana extremely close to a Sennheiser MKH-30 and a Neumann KMR-81. We sat in the control room, listening to what sounded like the skin of a giant space creature being peeled off of its skeleton. After the first peel, David, not hearing what we were hearing, exhaled and said, “Oh well, that was nothing.” The look on his face when we played it back was precious.

The Marx Brother-like pickle characters in Cloudy 2 have small eyes that blink and mouths that open via a perpendicular slice. Their heads tilt when they speak, like they are hinged at the back. I imagined that the sound of their mouths opening and closing would, in reality, sound something like two halves of a watermelon opening and closing. It was one of those occasions when your first impression actually plays out on screen. The very first attempt at using the watermelon recordings for those movements glued right onto the image.

Sounds of the pickle characters’ movements were almost all created during the team’s vegetable recording session

Sounds of the pickle characters’ movements were almost all created during the team’s vegetable recording session

Pitch, Speed and Reverse

Playing sounds at a different speed, pitch or direction—all three at times in varying proportions—can transform them into what you’re looking for. Slowing a gunshot down can make a smaller gun sound like a cannon. A recording of a book being slammed onto a wood surface played at half- or quarter-speed may sound like a gun. A gun played two or three times normal speed may sound like a book hitting a wooden table.

There are numerous free or near-free digital audio workstations (DAW) programs or apps available on the Internet. They all allow you to vari-speed and reverse a sound. Find a sound clip of a movie that contains some amazing soundscapes and effects. Load it into your DAW. Listen to it carefully. Pitch it up in increasing intervals, starting with 20 percent. As you pitch it up, you may hear some of the ingredients begin to reveal themselves. Go the other way and pitch it down. Keep checking in with the track as you try different settings. Now flip the tracks in reverse. If you analyze sounds that you think are impossible to create with this method, they may suddenly become very possible.

I use an application called soundminer (soundminer.com) to search my sound libraries. It allows me to easily listen to things at wildly varying pitches, speeds and directions, and really kicks the creative juices into gear. The pitch and volume settings can reset as you move through a list of sounds, or it can keep the setting. There’s a “show sounds in same folder” option, so when I hear something interesting I can immediately see what its neighbors in the folder are. They also have a roulette wheel button, so I poke that button and start listening to random sounds when I feel stuck. Things can come into focus pretty quick. Even if the only sounds you start hearing are things that will “never work,” now you have that information, too.

Don’t Go it Alone

Trying to create sounds for picture without guide tracks may be dangerous, and a waste of time. I always listen to the dialogue tracks and music temp tracks from the picture editor. Creating a sound that doesn’t work with dialogue or music is pretty pointless. The voice is always going to be there, so why do any work without it? If your amazing sounds work with the dialogue and music as well, you’re closer to that magical blend that adds up to more than the sum of the parts.

Incorporating Foley

A well-performed and recorded Foley track (i.e. studio reproduction of real-life sounds) can accomplish very quickly what would take much more time to build out of sound effects recordings. For key props and footsteps, though, I don’t rely on Foley alone. Combining location recordings with a Foley performance adds an additional layer of interest and variation.

I worked on a film that had a few key scenes in an articulated city bus. Location sound specialists John Fassal, Eric Potter and I arranged to hire and record an articulated bus for on-board and exterior motor and movement sounds. I had figured we would grab some movement and specific interior sounds without the motor running, but on our first run on board the bus, it blew a tire. We had to wait hours for a repair truck to come and change the tire, so we ended up recording any sound we could think of—footsteps, movement, sitting and standing, hand grabs on metal poles, etc. We then layered in location recordings on top of the parallel Foley tracks, creating a sound that neither Foley or sound effects alone could achieve.

“Worldizing” a Foley track can also add a very organic ambience to what may feel like a sterile track. Don’t be afraid of recording the ambience from more than one location—using multiple tracks and microphones or by recording multiple passes—and then varying the balances later. The proliferation of high-quality portable recording devices and smartphones or tablet devices makes “worldizing” so much easier than before.

Moving the microphone can add a layer of dynamism or movement that might not otherwise be achieved. Try using a shotgun mic in a hard surfaced room, playing a footstep track that is meant for a character racing past the camera and away in the space. Make a bold movement with the mic at the peak of the on-screen movement. Alone it may sound odd, but layered with Foley or effects, it can be stunning.

Ambient sounds

Approaching background sound—ambience, atmosphere, etc.—for an animated film is very similar to a traditional live-action film. The same questions apply: How realistic is the location? What time period and geographic setting is it? What scale are you trying to describe, enhance or extend? How dense? What story can background sound tell?

In Cloudy 2, the island of Swallow Falls is an overgrown jungle teaming with food wildlife. To create the jungle atmosphere, we utilized many standard components, often pitched or twisted in some way to shift the scale. Some of the sounds we created for the large animals were modified to create distant versions—the roars and plaintive wails of a “Cheese Spider” creating a spooky, dangerous atmosphere. Repurposing large close-up sounds (used later in a film) to create atmospheric elements helps lend an overall thematic tone that might not otherwise come through for the viewer.

The Cheese Spider sizzles like a patty off the griddle, and has footsteps like a 4x4 being slammed into the earth just above a tiny microphone

The Cheese Spider sizzles like a patty off the griddle, and has footsteps like a 4×4 being slammed into the earth just above a tiny microphone

Background often performs a similar role as music. Accordingly, when music is in, it may be a good time for background sound to either reduce in level, density or tone, or step aside altogether. Thinning, reducing or simplifying the backgrounds in an inverse proportion to the content of the musical score creates opportunities for greater dynamics, clarity and definition in the overall mix.

Making the perfect version of something can prove the opposite of what is actually required. For a recent film, I went to great pains to record a very specific vehicle. We shut down an area with a three-person crew armed with the best gear and a certificate of insurance. We spent hours getting the quintessential sounds of the vehicle. We edited and mastered the recordings; edited the elements to perfection and made a multi-layered premix of the sounds. That’s when the directors decided it would be better to have the vehicle be electric-powered instead of gasoline.

What’s “right” is whatever works. “Sounds good—is good” is a common starting point, but arriving at what sounds good can be difficult. Breaking something down by asking the essential questions—animal, mineral or vegetable?—can be all it takes, a game of Twenty Questions. What does the script tell you about the sound? How do the other characters or items in the world react to the thing? How does it react to the rest of the world? This is what guides me to the end of the process. Well, at least to the next round of revisions! MM

Geoffrey Rubay has worked in the motion picture sound industry for the past 21 years, with credits as editor, designer, supervisor or re-recording mixer on over 100 projects, including Reservoir Dogs, Amores Perros and Star Trek (2009). This article appeared in MovieMaker‘s Complete Guide to Making Movies 2015.

Leave a Reply

Your email address will not be published. Required fields are marked *