The role of sound effects and musical elements in the moving image

Sound and music, or the absence of it has a profound impact on film and TV. It is used to enhance drama and to help illustrate the emotional content in the story. It is also used to manipulate the feelings and sentiments of viewers. This is universal. Whether you watch domestic or foreign content, music is used to this effect. Rarely do you see a film or TV program that doesn't employ music and sound.

Sound effects and sound libraries are used all over films and it is pretty much guaranteed that there will be at least one instance of this in every film and TV show you watch.

Jack Foley is widely known as the initial founder of adding post-production sound effects to film. The reason this process takes place is because some sound effects could be hard to record on-set and need to be more distinguished, need to be taken from the sound of something else or need to be edited to achieve the desired sound. People who record sound effects separately from the film are known as "Foley artists"

You can see that they are experimenting with many different objects to achieve a fitting sound for the movie, whilst watching the film to keep it in time.

An example of a sound being taken from somewhere else is the process used to create the laser blast sound from George Lucas' "Star Wars" movies. Sound designer Ben Burtt climbed a radio tower and used a hammer to strike one of the guide wires whilst recording it to produce the “pew, pew” sounds.

Another example of this lies in Peter Jackson's "The Lord of the Rings: The Fellowship of the Ring". David Farmer (sound designer on the LotR project) came up with the original template for the Balrog, he wanted it to sound like it was something that would live in the very bowels of the world, sort of like a big flaming turd with a sword and a whip. Or a giant horned tapeworm, if you will.

To that end, the Balrog's voice, and some of its movement, wound up being something ingenious in its simplicity: a cinder block scraping along a wooden floor at different speeds. That delightfully cracky, grinding sound that accompanies the demon is made of a mixture of rocks grinding together and the cinder block tearing over someone's parquet.

Ben Burtt, who is the sound designer for the Star Wars movies, used a very interesting method for creating the famous sounds of the lightsabers in the films, as explained in the following video.

The sound of the lightsaber moving when a microphone is waved past the sound source is created by the scientific principal known as the "Doppler Effect". This is the apparent change in the frequency of a wave caused by motion between the sound source and the observer.

For example, if you were to stand at the side of a road and a car drove past you, the sound waves given off by the car as it approaches are compressed against the front of the car making it appear to be higher pitched. Then when the car passes, the sound waves at the back of the car are flowing off of it and are spaced further apart, making the sound appear to be lower in pitch. here is a video of a fire truck siren demonstrating the Doppler effect.

Music in Movies:

Since the dawn of motion pictures, music has played an integral part of the cinematic experience. Before the advent of "talkies," music quickly became a necessary tool to aid the narrative. These conventions have become movie-making standards and are still used today.

Music can help express character emotion. In the days of silent film, the only methods to express how a character felt were the dialog cards, the actor's face, and the music score, all of which worked together to convey the necessary emotion. In a silent film, if a character delivers what appears to be a tense or dramatic speech, tense and dramatic music is sure to accompany it. In modern film-making, the same can be said to be true.

Music score is a basic and effective way to heighten the drama of a given scene in a film. Regardless of the genre (drama, comedy, romance), a film score can add to nearly any scene. In an action set-piece, the music will match the action in terms of power and intensity. In a comedy scene, the score can be expected to be as light or silly as the action taking place within the scene.

Music can establish a mood. At the beginning of a movie or scene, music is often used (along with establishing shots) to help set a tone, before any dramatic action takes place. In the beginning of a horror film, shots of a desolate, empty street may have ominous, foreboding music accompanying them, to establish that the action about to transpire is very scary.

Music can establish a time or setting. In many period pieces, film score or music from the era is used to help establish and reinforce the specific time period which the move is set in. Example: A film set in medieval times, will often use source or score music from the very era that it is trying to recreate, thus further embedding the viewer within a particular time and place.

Music can help advance the story. The music montage is a popular way to condense a large amount of information into a short amount of time. The use of a pop song or score selection, accompanied by thematically related shots (the lead character is sad, people falling in love, hero in training) advances the story without spending the days, months, or years it would take in real life for the actual events to transpire.

Music also can mislead the viewer. Musical misdirection is most often employed in, but not limited to, thriller and horror movies; usually to lull the viewer into a sense of complacency before a big scare. Example: The heroine babysitter walks into the upstairs bedroom as calm music plays, only to discover a killer behind the door, immediately cued by pierced, shocking music. In an opposite example, the babysitter walks upstairs, with tense and scary music underneath the scene, expecting to find a killer, when it turns out only to be the house cat.

Music guides your emotions in a film. Think of movies like Psycho, The Ten Commandments, Super Fly or the Indiana Jones films. Think of Seinfeld, Lost or Numb3rs. The music in those films and shows serve as a powerful guide as to how you should feel about what is happening on the screen. Music brings a film to life like nothing else can. It has its own language that can signal a certain theme, a mood, a quality in a particular character or even give landscapes in the film a character like quality. Certain kinds of instruments can be used to create this effect. The instruments can evoke a certain era or time period, culture, country or a fantasy land. The Portrait of a Lady, Batman, Amadeus and The Lord of the Rings all have scores that employ these techniques. It can range from ancient, ethnic sounds or something evocative of antiquity like the score for Conan the Barbarian or Spartacus, with romantic orchestral style underpinnings or something very modern and atonal like The Matrix or Alien. Lest you think I've forgotten about soundtracks, soundtracks are every bit as effective as film scores.

The music for Raiders of the Lost Ark played a key role in this film, and John Williams provided an excellent score for this film, also earning a nomination for an Oscar for best score. The spirit of Williams' style for Raiders of the Lost Ark is finely tuned to the adventuresome tone of the film's story, matching the exuberance of each of its scenes with the same precision of theme and emotion. The title march attracts the most obvious attention when the masses recall Raiders of the Lost Ark, but in reality the extremely effective and even catchy subthemes for the score are equally vital to the score's success. Still, it's the title march you hear in stadiums and in trailers for the following entries in the franchise; just as Monty Norman's theme for James Bond and Williams' theme for Darth Vader are engrained in pop culture as the most obvious musical representations of one serial movie character, the march for Indy Jones is worthy of the same distinction. The score is a rare occasion in which the entire package, with only a few small detriments in lesser cues, is better than the brightest moments of almost any other score. Williams so thoroughly nails the pulse of this picture, from the melodramatic awe of the Ark to the gritty rhythms of Jones resilience as he battles a convey of trucks, that Raiders of the Lost Ark is a cinematic experience much greater in both intensity and entertainment value because of Williams' contribution. Three major themes exist in the score, and the purpose of each is so clear that the composer would work all of them into sequels in the franchise. A variety of lesser motifs, including secondary phrases of these major themes, occupy significant roles in the work. There has long been speculation about additional motifs in Raiders of the Lost Ark, though while Williams does definitely conjure auxiliary ideas throughout the score, their direct application remains open for debate. What isn't contested is the harmonic beauty of the score. Even the film's major action sequences  offer exhilarating tonal structures and readily enjoyable rhythms, producing a consistently fluid experience.

Soundtracks of the Lord of the Rings film trilogy music was composed and conducted by Howard Shore. The scores use a technique called leitmotif, which is a musical phrase associated with a character, a feeling, an event, etc. Below is one of the theme music for Lord of the Rings called "Concerning Hobbits", this is the theme for the Shire. On the soundtrack, it starts by a solo flute playing the theme, followed by a solo violin and then the complete orchestra repeating this theme. The track then quiets down and the solo flute plays a second melody, followed by the orchestra. The track ends with the violin and then the flute repeating the first melody.

1. First we have an introduction given by the cellos and bassoons to set the "hobbit" mood. Notice that in most of the other tracks, the low strings, when played by itself, gives a kind of foreboding. Here, by playing them staccato (detached) and playing 1-5-8-5-1-5-8 which is tonal it gives it a more playful sound. (The numbers refer to the notes of the scale. If you are in C then C is 1, D is 2, E is 3 and so forth.)
2. At: 05 the tin whistle, I believe, gives us Frodo's theme when he says, "It's wonderful to see you Gandalf!” Notice that it is played light and airy, in other words, there is little bass and completely tonal. This is definitely our happy Frodo! The theme also has much ornamentation such as grace notes and slides which is a folk derivative. The high strings take over the cello part, which helps maintain a steady, but very subtle beat. Almost all of the tracks have a lot of dissonance in them and the fact that this is so tonal makes the music seem very folk like. The harmony is also quite simple in its form.
3. This time at: 23 the bassoons play the 1-5-8-5-1-5-8 intro by themselves. This is where Frodo asks about the outside world.
4. At: 28 a solo violin plays the folk like melody. Each note is much attached to make it sound like a fiddle which is a folk instrument. A harpsichord is added playing the notes of each chord one at a time to give a very simple accompaniment. The fact that HS used harpsichord instead of piano makes it seem even more folk like and detached. (A piano hits the strings and a harpsichord plucks them.) The strings also do what I call a boom-chuck-chuck, boom-chuck-chuck which is also very simple in its form and gives a kind of humour to the music. This is where Gandalf says, "Well, what can I tell you?" and you see Hobbiton for the first time.
5. At: 43 we hear a version of Frodo's theme again while we see the market place at Hobbiton. The music is very legato (opposite of detached) and the contrast makes this section seem very warm and inviting. However, you can still hear the harpsichord still playing its detached notes in the background to continue the happy/playful side of the hobbits.
6. At: 54 the solo violin enters again as we see the "Happy Birthday Bilbo Baggins" sign raised. The music also rises in pitch to help the expectation of the party. This doesn't last very long since Gandalf changes the subject to Bilbo.
7. 1:04. here the music changes to a more serious tone, a premonition perhaps? There is no tune until 1:13, just held chords by the strings. It's like the playfulness of the hobbits paused for a moment of foreboding. However, there is still no dissonance here.
8. Of course it doesn't last very long because at 1:18 Frodo changes the subject and tells Gandalf that he can keep his secrets and the boom-chuck-chuck and harpsichord returns. Then he says, "Whatever you did, you've been officially labelled as a disturber of the peace." This is right on 1:36. Here the violins play even more staccato, any more and I don't think the note would exist, with the addition of the flute to give it as much of a humorous tone quality as possible as the grumpy old hobbit gives Gandalf the grim look.
9. Then we see the countryside again with the young hobbits running toward Gandalf's cart at 1:44. Frodo's theme comes back played very lyrically, again to give us the warm, fuzzy hobbit feeling.
10. At 1:57 the hobbit children cheer for the fireworks and we get the interaction with the grumpy hobbit and his wife. The music returns to the light and detached theme to say that this is funny. I think the wife's glare happens at the silence at 2:07.
11. At 2:15 as Frodo says, "Gandalf, I’m glad you're back." His theme returns so we can end with the warm fuzziness as Gandalf pulls up to Bag End.

Music in film is also about the imagery. Intimate drawing rooms, beer parties, a coronation, sweeping landscape vistas, a foggy road all of these scenes will require the music to adapt and fit what the audience sees on screen and to help tell the story in these scenes.

Music can be broadly appealing and direct in its storytelling like in Shrek or Spiderman or subtle and minimalist in films like Spider or Memento. Music is not all that is used. Ambient sound in film is just as important. Animal sounds, flushing toilets, throwing objects together, crushing paper, dropping things on surfaces or rubbing things together all these techniques will get interesting sounds that can fit the scenes in a film or show. These sounds can be used to surprising and unexpected ways. The only limit is the creativity of the team doing the work. Sound, whether from ADR, film music, modern music or soundscapes is not accidental. Some of the sounds you hear in films come from making accidental noises but they are kept and used because of some certain quality that can be used to enhance the film. Sounds are recorded from nature or from urban settings. The point is that sound is designed, not simply recorded and thrown into the mix.

Like white space in graphic design, silence is also used to dramatic effect. Too much music can grate on the nerves. Beats of silence can be put in to a scene or silence can be used as an element to underscore something or someone and it can be just as dramatic and effective as music. Sometimes sound has to be pulled out in order to be heard properly. It, like music, can overwhelm viewers and may have to be pulled back, which can make it more impactful, more visceral. Less can be more and this technique can heighten the intensity in a scene without overwhelming it.

Overall, music and sound are an integral part of film and TV production. It helps tell the story, guide the audience and evoke mood, character and themes that can drive home whatever the story is trying to convey. Music has a profound effect on our emotions and if the performances don't affect you, often the film score or the soundtrack will.

Sounds for Intrigue

Create intrigue by "asking questions" with the weird twanging of high strings, the plinking of a child's piano, or the buzzing of a synthesizer. War of the Worlds is possibly the best example of using weird sounds to fascinate and capture the audience's imagination.

Sounds for Tension

One of the best ways to create suspense is just to raise the volume or tempo of ambient sounds. It's amazing how irritating ambient sounds can be, and this can be used to your advantage. You can really grate on the audience, subtly building tension in them with annoying spaceship beeps or jungle insects.

Theme Music

Bring atmosphere to individual characters by giving them their own theme music. Of course ensure that it matches their style!

Sounds for Heroism

After all these years, horns and brass still provide the crescendo of sounds to represent heroism. Not just accompanying sounds for heroes, but acts of heroism are often represented with a powerful orchestra as well.

Sounds for Humour

Of course, using sounds for humour means not having to worry too much about making the sounds "invisible". In fact, making the sound overly obvious can provide a bit of humour in itself. Use techniques such as timing, exaggeration, and surprise.

Non-Pitched Atmospheric Sounds

Non-pitched atmospheric sounds are used in film making to create a similar feeling to what the score tries to establish, but in a more subtle fashion. They aren't usually very noticeable unless you're looking for them, but will still have the effect regardless of if you're aware of a sound being played.

A brilliant example of the effectiveness of atmospheric sounds is Orin Peli's "Paranormal Activity". Often throughout the film, when there is a silence, an extremely low rumble is used to create subtle suspense and a feeling of uneasiness in the viewer.

Note how the extremely low frequency sound stops as soon as the door slams shut. The sound is used to build suspense, to make the viewer aware that something is going to happen. This is very effective on high end sound systems such as those in cinemas where the sub-woofers can produce extremely low sounds. Found footage movies like this generally have no score or non-diegetic sounds, so these techniques are usually relied on to keep it subtle and make it feel more realistic than high-budget blockbuster.

The thing with Non-pitched sounds is that they are the little sounds you see in movies that you know are real, non pitched sounds could be a door slamming or birds chirping at the park while sound effects are usually done on synthesizers or created by recording various different sounds to create unrealistic sounds like the sound of a lazer gun or hover car.

Although non-pitched sounds are not always their true sounds, for example if you recorded a plain taking off it is unlikely that the sound of the real plane will be recorded on the footage usually sound designers will come up with  a different way to re-create the sound and to make it sound like a plane and then edit it in, this is where Foley comes in.

 Foley effects are sound effects added to the film during post production (after the shooting stops). They include sounds such as footsteps, clothes rustling, crockery clinking, paper folding, doors opening and slamming, punches hitting, glass breaking, etc. In other words, many of the sounds that the sound recordists on set did their best to avoid recording during the shoot.

The boom operator's job is to clearly record the dialogue, and only the dialogue. At first glance it may seem odd that we add back to the soundtrack the very sounds the sound recordists tried to exclude. But the key word here is control. By excluding these sounds during filming and adding them in post, they have complete control over the timing, quality, and relative volume of the sound effects.

For example, an introductory shot of a biker wearing a leather jacket might be enhanced if we hear his jacket creak as he enters the shot, but do we really want to hear it every time he moves? By adding the Foley sound fx in post, they can control its intensity, and fade it down once the dialogue begins. Even something as simple as boots on gravel can interfere with the comprehension of the dialogue if it is recorded too loudly. Far better for the actor to wear sneakers or socks (assuming their feet are off screen!) and for the boot-crunching to be added during Foley.

Foley is usually performed by Foley artists. Ideally they stand on a Foley stage (an area with a variety of possible surfaces and props) in a Foley studio (a specialized sound studio), though any post production sound studio will do with a little modification. The Foley artists can clearly see a screen which displays the footage they are to add sound fx to, and they perform their sound effects while watching this screen for timing. The actions they perform can include walking, running, jostling each other, rubbing their clothing, handling props, and breaking objects, all while closely observing the screen to ensure their sound fx are appropriate to the vision.

Increasingly, many simple Foley sound fx are done without Foley artists the sound effects are stored electronically and performed by the post production sound engineer on a keyboard while watching the visual. Done poorly this type of "Foley" sounds bland and repetitive, and it is nowhere near as flexible as the real thing, but it is much cheaper than renting a Foley stage and paying Foley artists to create the Foley sound effects.

Without Foley, a film sounds empty and hollow - the actors seem to be talking in a vacuum. The sound recordist, if they did a good job, has given us the dialogue and excluded everything else, but our films needs more than this for the picture to come alive. We need to hear the little sounds of clothes, furniture, etc . But we need to control those sound effects so they don't obscure any of the dialogue.

Another common use for Foley sound replacement is adding it to documentary footage. Old historical film seems lifeless when it is screened without sound, and adding Foley to it helps bring those long dead images to life. Next time you watch a history documentary that uses silent archival footage, listen closely and you should hear at least minimal Foley sound fx, mostly footsteps, behind the narration.

Foley can also be used to enhance comedy or action scenes. Watch most comedy films and you'll notice that many of the sounds are enhanced for comic effect, and sometimes the Foley sound is the joke. As for action, most fist fights do not involve the actors really hitting each other, and even if they did we would not be able to record a satisfying punch sound. By punching and variously molesting such objects as cabbages, celery and sides of beef, Foley artists can record unique and much more 'realistic' action sounds.

The three important types of sound we use in films and television are referred to as hard effects, Foley, and backgrounds. The difference between the types is primarily their length. Hard effects will ordinarily short sounds like gunshots, light switches, a big slap on face, glass smashing, etc. Backgrounds normally begin in the beginning of the scene and carry on all the way to the finish of the scene. As an example, if the scene we editing on is staged at a city park, we'll rarely not contain effects like birds chirping and singing from the beginning of the scene to the end of it even though birds usually are not seen all the time and in a lot of cases, not seen in any way.

Why do we have to have the background sound effects?

If you ever look at a footage of the film as it had been recorded on location you are going to understand that it's somewhat dead as much as ambiance sounds. For example, in case you see a scene in a bar you will see a lot people speaking but you may only hear the voices of the important characters. The way in which this really is done is usually that the people who you see speaking are basically pretending to be talking or speaking in very low voices. Clearly, this is very unnatural for us humans to watch considering within real bar there really are a lot of people speaking. If the people in the bar were talking normally, that would have made it hard for the mixer to mix it in a way that the audience can easily understand the dialogue of the main actors (which is always the most important).

In order to make it more natural, the sound editor can add in the sound effect of men and women speaking (sometimes called "Walla"). The editor will also add other sound to help the scenes seem more realistic such as glass cups clinking, drinks mixing, room tones, perhaps an off stage billiard game and much more.

What are room tones? Inside of pretty much any room you enter you'll notice some kind of sound. It could be from your air condition, it could be from city traffic outside, it can be from a refrigerator or something else. You don't at all times notice it but our brain is quite use to hearing those sounds all the time. Then to make the film as natural to us humans as real-life, we always add room-tones on internal scenes. Once scenes are outdoors, we have the choice of including traffic or wind. Additionally to making the movie appear much more natural they are also very helpful for the dialogue mixer because the dialogue coming in to the mix stage might have times when the background noise in the dialogue recordings have noticeable changes as well as holes. A background sounds are helpful to mask those holes.

But, the most important reason we add background sound effects is to make a movie or Television show richer. The backgrounds are excellent for setting the atmosphere that your director would like the audience to experience. As an example, you can have two different films which are shot at the precise same location in a city, let's say one romantic movie and one suspenseful. In a romantic film we will put light city traffic elements, nearly no horns, wind in the trees, and birds. For the suspenseful film you can add very heavy city traffic, people honking , police sirens, nervous crowd voices, helicopters, etc. Together with music, the background sounds is a fantastic instrument of setting the mood for a film.

An example of how non-pitched sounds will be used in a romantic movie is the final scene to "Pretty Woman", and as well as the music helping make the scene feel more romantic, pay attention to the little sounds that help make the film look more realistic.

The start of the clip you hear a slight wind breeze out on the street to fill the background noise, when it cuts to  Julia Roberts in her room you can hear her footsteps walking around the room. As you hear the car horn, she walks towards the window and you hear her footsteps again. When the music kicks in, unless you pay close attention you won't be able to notice the next sounds but, the sound of the limo driving is there as well as the wings of the birds flapping away. There is also the screeching noise of the ladder getting pulled down as he climbs up. All these sounds are important even though you don't always notice them they are there to make the film seem more realistic and its because its realistic you don't notice them as that's what you can expect to hear if it was in a real life situation.

My last example is a scene from the "Matrix", listen to how the non-pitched sounds help create a more dramatic and suspenseful scene as Neo (main character) attempts a daring escape from the agents.

At the start of the clip when the delivery man comes in you can hear the slight noises of Neo's chair turning round, then all the little sounds of Neo signing for a package, like the writing of the pen and passing of the package. Then comes the sound of the ripping opening of the package, the ringing of the phone and the sound of the phone flicking open, hardly you will ever see a phone make that sound when flicked open but it is added for dramatic effect.

Morpheus voice on the phone would of been added in the post production as it won't be possible to record the scene and the phone call at the same time, the sound designer would of also altered the frequencies in Morpheus voice using an Equalizer to make it sound like the receiving end of a phone call, making the conversation on the phone more realistic.

As the agents walk around the room you can hear their footsteps, as well as the music being played, this also adds suspense as they walk around in search for Neo. The footsteps of Neo is also heard as he runs to the room at the end after being told to, but his footsteps are more fast paced creating the feeling of haste and panic. As Neo enters the room you hear the sound of the door open and close which is another non-pitched sound. Then comes the sound of the window opening followed straight after with the sound of the wind from outside, all these sounds come together to make it feel that he has actually opened the window, the wind also helps build suspense as it makes the audience feel that he is high up.

After the phone call with Morpheus, Neo climbs out of the window and yet you can hear all the little sound of his steps to make it sound real, and the wind gets more intense. You then hear Neo's feet shuffling along the edge of the window making his movements feel like he's in a panic but is also moving fast, at the same time the wind starts to blow hard making almost a whistle sound creating more suspense.

Neo then comes to an obstacle, at this point you can hear the metal clanging and rattling as he grabs hold of it and tries to make his way around. Neo takes one final look down before a massive wind blows, making a whistle, sound knocking the phone out of his hand. Neo quickly grabs onto the metal barrier and the sounds of the metal clang and bang are heard as his hands lock on for his life. The window continues to blow and Neo finally says "I can't do this" and forfeits his efforts of escaping. As Neo is captured and being taking into custody you hear light traffic noises in the background to ease the tension and the doors of the car opening and shutting as  Neo is put inside.

Without these little sounds films will not be brought to life as much, as these are the sounds you hear in everyday life so as you watch these films you expect to hear certain sound that you would normally hear in a real life situation, this when used properly help make an impact on a scene as I have just demonstrated to you by using these two examples.