Unless you have been living on Mars, you must have read or heard about ‘Now and Then’, the new (and possibly last) song by The Beatles. The song is based on a rough piano and voice demo recorded on cassette by John Lennon in 1978, and later reworked and developed by the rest of the band.
Initially the other Beatles members worked on Lennon’s demo in the mid ’90s, but audio restoration technology at that time wasn’t advanced enough to achieve proper track separation. Fast forward almost 30 years and… ‘Now and Then’ becomes an actual Beatles song, officially released on digital platforms and even as a single on 7″ vynil!
Audio Restoration Makes Miracles
This tech miracle has been made possible by the use of a proprietary audio restoration technology, called Mal. As reported, among others, by the BBC, “the software was able to lift Lennon’s voice from the original cassette recording, removing the background hiss and the hum of the mains electricity that had hampered previous attempts to complete the song”.
In the realm of music technology, the boundaries are continuously being shifted, and one of the latest groundbreaking developments is Mal — the machine learning (ML) software already utilized by acclaimed director Peter Jackson for the restoration of The Beatles’ archive footage.
The Genesis of Mal
The software’s name, Mal, mirrors the name of The Beatles’ road manager, Mal Evans, who was instrumental in the band’s success. Just as Evans played a key role in The Beatles’ journey, Mal, the software, has been pivotal in breathing new life into the band’s extensive sound and video archives.
Mal was developed out of necessity. The Beatles’ archives comprise a wealth of material, but much of it was poorly preserved, difficult to understand, or in formats that presented significant challenges for modern equipment. The task of restoring this treasure trove seemed insurmountable until Jackson employed machine learning technology.
Mal’s Working Mechanism
Machine learning, a subset of artificial intelligence, involves training computers to learn from data and make intelligent decisions or predictions. In the case of Mal, the software was trained on countless hours of interviews, concerts, and other Beatles-related audiovisual material.
The intelligent software was able to identify, analyze, and clean up audio tracks, as well as restore, color-correct, and upsample video footage. By learning The Beatles’ speech patterns, Mal could even reconstruct dialogue from segments where the audio was missing or unintelligible, significantly enhancing the clarity and comprehensibility of the archive material.
More specifically, for ‘Now and Then’ the software was trained on John Lennon’s voice and piano recordings, to isolate, extract and rebuild (where necessary) the missing frequencies, and remove unwanted noises (including the TV in the background, part of Lennon’s demo!).
As Giles Martin (George Martin’s son, producer of the song) put it “Essentially, what the machine learning does is it recognizes someone’s voice. So if you and I have a conversation and we’re in a crowded room and there’s a piano playing in the background, we can teach the AI what the sound of your voice, the sound of my voice, and it can extract those voices”.
Here you can hear the final version of the song:
Mal’s Impact on Music Restoration
Mal’s work on The Beatles’ archives and for the latest single, ‘Now and Then’, is a game-changer in the field of music restoration. Achieving such a high level of restoration manually would have taken an enormous amount of time and resources. Mal, on the other hand, was able to accomplish this in a fraction of the time, showcasing the potential of machine learning technology in this field.
Moreover, Mal’s ability to understand and emulate The Beatles’ vocal/speech/musical patterns could open up new possibilities for other artists’ archives. This could lead to the restoration and even the creation of new material from old or damaged recordings, opening up a world of possibilities for music historians, researchers, and fans.
In conclusion, Mal represents a seismic shift in the way we approach music restoration and archival work. As we look to the future, it’s exciting to imagine how machine learning and AI will continue to shape the music industry, resurrecting the past and creating the new in ways we can only begin to imagine.
Other Tools For Audio Restoration
Unfortunately, Mal is not a commercially available product (yet?), but there are several tools and technologies used in the field of audio restoration. While they may not be exactly the same as Mal, these tools utilize various techniques to improve audio quality, remove noise, and sometimes even reconstruct damaged or missing parts of audio files. Here are a few examples:
- Izotope RX: Widely regarded as the industry standard for audio repair and enhancement, Izotope RX offers a comprehensive suite of tools for cleaning up and restoring audio. It uses advanced algorithms to remove noise, distortion, and other unwanted sounds, and it includes spectral analysis tools that allow users to visualize and manually edit their audio. You can check our iZotope RX review (for an older version).
- Adobe Audition: This professional audio workstation from Adobe allows users to mix, edit, and create audio content with a comprehensive toolset that includes waveform, spectral display, and multitrack. It also has features specifically aimed at restoring and repairing audio.
- Acon Digital Restoration Suite: This software suite includes four audio restoration and noise reduction plugins that are the result of several years of research within the field of noise reduction and audio restoration algorithms.
- Cedar Audio’s CEDAR Studio, CEDAR Cambridge: These are professional audio restoration systems used in post-production for film, TV, and radio. They offer a suite of restoration tools for cleaning up dialogue, music, and other audio.
- Audacity: Although not as advanced as some of the other options, Audacity is a free, open-source audio software that includes basic noise reduction and audio enhancement tools.
These tools, while perhaps not as specialized or as targeted as Mal, demonstrate the breadth of options available for audio restoration.
Remember though, the success of audio restoration also heavily depends on the skill and expertise of the person using these tools. A seasoned audio engineer can often achieve remarkable results, even with less advanced equipment.