All about... film sound and how we restore it

A typical dictionary offers many definitions of acoustic ‘sound’ – covering energy transfer, neurological perception, media content, collectively, such as ‘the sound of The Beatles,’ and environmentally, as in ‘within the sound of the church bells’. Sound invites cross-sensory and emotional response. Just like all media, to record and preserve sound is to share our experiences and tell our stories.

The widespread recording, transmission and reproduction of sound predates that for the moving image. Yet, after the birth of silent cinema at the end of the 19th century, it was another 30 years before recorded sound and pictures were combined in a single medium.

The silent era was not necessarily so silent, however. An early way to bring sound to the movies was by synchronised live performance. Actors spoke, musicians played, and performers created sound effects while the image played on the cinema screen.

Recorded sound boosted the creative powers of filmmakers by giving them the opportunity to retake, replace and edit the sound prior to its performance. The young film industry sought ways to combine their aural and visual artistry into one potent product.

The first known attempt to synchronise sound with picture happened in 1894. Thomas Edison had successfully invented and marketed the wax cylinder for recorded sound. Inspired by Eadweard Muybridge’s Zoetrope photography, Edison and inventor W. K-L. Dickson, developed the Kinetophone, a proto-sound-film system. A reproduction of their attempts to synchronise wax cylinder discs with a Kinetoscope can be seen here:

Meanwhile, the BFI National Archive resynchronised the 1898 wax cylinder sound of the music hall star Lil Hawthorne singing ‘Kitty Mahone’ to a 1900 film of her performing the song:

These two examples are recreated by modern technology. At the time these original recordings were made, the technology to synchronise sound to the moving image was still experimental. Before adequate amplification equipment became available, these systems would never become commercially viable.

On the audio retail market, the wax cylinder was succeeded by the shellac disc, designed to play on ‘phonograph’ players, the progenitors of the vinyl record turntable. On the wave of the rapid growth of early cinema, inventors and engineers designed a disc phonograph that could be synchronised to a film projector.

The first marketable version of this was introduced by Warner Brothers in the 1920s: a single-sided 16-inch shellac disc format called Vitaphone, which contained about 11 minutes of sound per disc, about the same as an average film reel. The earliest cinema releases using Vitaphone sound were the swashbuckler Don Juan in 1926 and the showbusiness story The Jazz Singer in 1927, in which the sound of a Hollywood movie, though not in its entirety, was heard in sync with picture for the first time. 

The fragility of Vitaphone discs meant they wore out quickly and were prone to breakages in transit. Along with their cumbersome size and weight, this meant that their commercial days in cinema distribution were numbered. A soundtrack that could somehow be encoded onto the film itself was needed. On the east coast of the United States in the early 20th century, exactly that process was being developed.

Military physicists had been developing methods of recording sound waves on to negative film and using modulated light beams for ship-to-ship communications since the 1900s. Converting sound to light would in time enable filmmakers to record sound onto the same piece of film as the image.

To achieve this, lamps were fitted to modified movie projectors to expose movie film to a light source which, when fed with an electric sound signal, varied in brightness. The varying intensity light passed through a slit and onto the film. The brighter the light, the darker the exposure. The resulting photographic ‘soundtrack’ therefore appeared as horizontal and parallel lines of varying brightness, analogous to the original sound. These became known as ‘variable density’ soundtracks. The BFI holds a great many of these in its collections.

After film processing to fix the image, a light could be shone through the variable density soundtrack in a projector, picked up by a photosensitive receptor, converted back into electrical sound waves, amplified, and sent to a theatre loudspeaker. The resulting product using this method, ‘Phonofilm’, was a success in independent cinemas on the east coast of America, but without similar success in Hollywood.

Aided by RCA and Western Electric companies, optical sound recording technologies were progressively refined, improving frequency response and dynamic range. The resulting ‘variable area’ soundtracks worked by exposing to film to a varying width of steady light passed through a slit, reflected off a rotating mirror controlled by the amplified sound signal, and onto the film.

The resulting exposure resembles the silhouetted profile of a decorative candlestick. Printed between the sprocket perforations and the picture, the sound can be replayed to theatre amplifiers and loudspeakers, and its synchronism ensured by being on the same strip of film.

The sound quality of the soundtrack was still not perfect though. More and more refinements were developed to improve noise, distortion and tone.

Early sound recordings were ‘monophonic’, or mono for short; a single channel of sound containing only one spatial dimension. Throughout the late 20th and 21st centuries various systems have emerged and played in theatres to widen the spatial sound experience, to better replicate sound in the outside world. Two-channel variable area stereo tracks appeared, sometimes with electronic noise reduction to further improve sound quality. In parallel, various multi-channel surround sound systems were developed, offering three, four and more channels to create a thrilling three-dimensional sound spaces, as well as vibrations that could be felt through the cinema seat.

Other synchronous cinema sound systems also emerged in the digital era of the late 20th and early 21st centuries. These included electronically synchronised digital audio stored and delivered on separate media such as compact disc, magneto-optical disc and hard disk. There were refinements to digital sound on film which could reproduce multiple channels and spectacular ‘surround’ sound productions. These involved the careful production of each synchronised sound channel to create the illusion of a 3D sound space. Experiments followed in the clever manipulation of ‘phase’ – one of the ways we locate sounds naturally – in binaural recordings using microphones set into a model of a human head. Today the perception of the sound’s height above or below the listener as well as around is available to sound artists using the Dolby Atmos theatre and home replay systems.

Electromagnetic sound transmission systems such as the telegraph, telephone and radio began as simultaneous transmission lines of sound from one place to another. With the advent of sound recording technologies beginning in the late 1800s, the transmission line could in time be fed to a recording and playback device. That meant that a recording made at one time could be played back at another.

After decades of playing recorded discs on air, development in the 1940s of magnetic recording tape by the Ampex Corporation of America led to the recording and time-shifting of entire radio shows. Legendary singer Bing Crosby famously invested $50,000 to help this development, allowing him to prerecord his shows and harmonise transmission schedules with his travel. The magnetic audio recording tape that emerged – a plastic ribbon coated with a ferric oxide emulsion – became a popular recording format around the world.

The BFI National Archive holds many recordings on ¼ inch tape, including interviews with actors and directors recorded at BFI Southbank. Of the many important technical and cultural consequences of magnetic tape recording was the advent of video tape, digital audio recording and cassette recording, putting the power to record into the hands of the public.

How is this relevant to film sound? The same technology as magnetic audio tape was used in the development of magnetic sound film, or ‘mag’ for short. Mag has the same dimensions and perforations as picture film, runs on differently equipped machines, or ‘sound followers’, which are synchronised with the film projectors and telecine machines (devices for converting film to video). In the days of analogue film soundtracks, mag represented another improvement in sound quality, allowing the easier manipulation of additional channels and the management of complex productions. It became a favoured production and delivery medium for film and TV producers.

Film and television archives, including the BFI, hold many hundreds of thousands of films and tapes on physical media, which are ultimately vulnerable to decay due to the natural decomposition of the organic materials on which they reside. Holding these precious items in temperature and humidity-controlled vaults and caring for their condition dramatically extends the life expectancy of these media. Meanwhile, digitising them – turning the recordings into data files – has become a vital part of making historic films accessible to audiences in today’s digital media culture. At the BFI, as in other archives, it also provides another way to help preserve our cultural treasures in a ‘digital vault’.

How do we do that? After selections, inspections, repairs and cleaning, the film soundtracks are sent to the BFI Conservation Centre’s Audio Transfer Suite for digitisation. Whether magnetic or optical, the soundtracks could be recorded in one of many different film gauges and track configurations requiring different heads to be selected to fit to different compatible replay machines.

Optical soundtracks could be on cellulose triacetate, polyester or the exceedingly combustible pre-1951 nitrate cellulose base. With exceptional safety and care, optical soundtracks are digitised using white-light LED and photocell replay machines, red-light LED scanners with Sondor Resonances which offers digital image processing technology to digitise directly from a positive copy or an original negative, and a multitude of magnetic replay options.

These ‘raw scans’ are preserved digitally, and a copy may then be sent to a digital restoration and remastering session.

Synchronising sound to picture either by the BFI National Archive or by a partner laboratory is assured by synchronising the sound replay machines’ motor to a precise electronic pulse generator. Films speeds are typically 24 frames per second since the end of the silent era and 25 frames per second for those specifically made for or recorded from television. With separate image digitisation workflows following a similar synchronisation policy, we can ensure sound and picture sync when they are finally combined into one carrier medium and/or photographed back onto film.

A separate room with gas extraction and equipment are set aside for digitising films that have advanced acetic decomposition or ‘vinegar syndrome’. There is a time pressure to digitise these as while the passive preservation staves off further decay it cannot reverse it, and it is desirable to capture these recordings as soon as possible.

As agreed through its relationship with the International Association of Sound and Audiovisual Archives (iASA), the BFI follows agreed standards to preserve the digital sound files. We preserve digital audio at a sample rate of 96kHz, 24-bit depth, and with the number of channels the same as the source medium. For a vintage feature film comprising several reels of film, there will be a corresponding digital sound file for each reel. The same applies to every tape spool and every side of audio cassettes.

Once digitised, the sound files can be used to digitally restore and remaster the soundtrack for theatres and home viewing using a variety of computer applications.

Critical listening is central to the process, but different people hear sounds differently. Younger people hear higher frequencies better; older people have a larger echoic memory. There are infinitely varying subjective and objective responses from person to person. The qualitative responses of as many people as possible attending our reviews inform our remastering decisions.

Digital remastering aims to create a new soundtrack to play on current media using the original as the source. Noise resulting from printed dirt, splices, damage, mould and grain is usually reduced or removed. Tonal adjustments may be made to control harsh overtones in the sound as heard on current theatre sound systems. The process is aided by a guide picture reference to help inform the extent to which adjustments are made and ensure that no sound is misinterpreted as noise. The aim is to minimize distractions so that the story has an unimpeded path to the viewer.

Digital restoration aims to ‘undo’ the wear and tear of successive copying and use. Minimal tonal changes may be applied. The aim in this case is to reproduce how the soundtrack would have been heard when first played in the cinema, so that audiences may enjoy both the story and the character of film itself.

A 90-minute feature film soundtrack can typically take up to 100 hours or more to completely restore or remaster. Once finished, the soundtrack is then recombined with the restored or remastered digital picture, reviewed again for approval by the BFI and its co-funding partners to ensure everything is present and correct, and if so, delivered to the BFI National Archive, theatres, publishers, BFI Blu-ray, BFI Player, Mediatheque, our library streaming service BFI Replay, or to the laboratory for printing back to film.

Technologies continue to develop rapidly, and sound remastering faces an exciting future. With machine learning becoming ever more powerful, we will be able to ‘undo’ and ‘do’ so much more than we can today. With increased power comes increased responsibility and we must ensure we use future technologies well. With care we may be able to create ever more refined restorations and remasters to delight, entertain and move us, and which celebrate more than ever our priceless collections of the moving image and its other half, the sound.

The following books on film sound are available in the reading room of the BFI Reuben Library at BFI Southbank, with many more available for ordering from the closed stacks:

• Sound Theory, Sound Practice by Rick Altman
• Audio-vision: Sound on Screen by Michel Chion
• Sound Design for Moving Image: From Concept to Realization by Kahra Scott-James
• Understanding Sound Tracks Through Film Theory by Elsie Walker
• Theories of the Soundtrack by James Buhler
• Silent Film Sound by Rick Altman 
• Hearing the Movies: Music and Sound in Film History by James Buhler, David Neumeyer and Rob Deemer
• Sound: Dialogue, Music, and Effects by Kathryn Kalinak
• Hollywood Soundscapes: Film Sound Style, Craft and Production in the Classical Era by Helen Hanson
• Film, a Sound Art by Michel Chion
• The Routledge Companion to Screen Music and Sound, edited by Miguel Mera, Ronald Sadoff and Ben Winters
• The Oxford Handbook of Sound and Image in Digital Media, edited by Carol Vernallis, Amy Herzog and John Richardson
• The Oxford Handbook of Cinematic Listening by Carlo Cenciarelli
• A Filmmaker’s Guide to Sound Design: Bridging the Gap Between Filmmakers and Technicians to Realize the Storytelling Power of Sound by Matthew Polis and Peter Rea