How Peter Jackson's Beatles Documentary Used Custom AI To Remove Background Noise (msn.com) 44
Peter Jackson's seven-hour documentary "Get Back" (now streaming on Disney+) edits footage from the Beatles' ambitious recording sessions for their 1970 album Let It Be. But long-time Slashdot reader MattSparkes writes that the whole documentary "would have been impossible without custom-made artificial intelligence, say sound engineers."
Sixty hours of footage were recorded but most of the audio was captured by a single microphone that picked up the musicians' instruments in a noisy jumble rather than a carefully crafted mix. It also recorded background noise and chatter, which made much of the footage unusable.
The team scoured academic papers on using AI to separate audio sources but realised that none of the previous research would work for a music documentary. They consulted with Paris Smaragdis at the University of Chicago and started to create a neural network called MAL (machine assisted learning) and a set of training data that was higher quality than datasets used in academic experiments.
The Washington Post describes it as "a sort of sonic forensics," adding that the name MAL was a deliberate homage to the HAL computer in 2001: a Space Odyssey — and to the Beatles' beloved road manager and principal assistant, Mal Evans. Using MAL, Jackson and his colleagues were able to painstakingly and precisely isolate each and every audio track — be it musical instrumentation, singing or studio chatter — from the original mono recordings made for most of "Let It Be." "What we've managed to do is split it all apart in a way that is utterly clean and sounds much better," Jackson said.
Other interesting observations from the Post:
The team scoured academic papers on using AI to separate audio sources but realised that none of the previous research would work for a music documentary. They consulted with Paris Smaragdis at the University of Chicago and started to create a neural network called MAL (machine assisted learning) and a set of training data that was higher quality than datasets used in academic experiments.
The Washington Post describes it as "a sort of sonic forensics," adding that the name MAL was a deliberate homage to the HAL computer in 2001: a Space Odyssey — and to the Beatles' beloved road manager and principal assistant, Mal Evans. Using MAL, Jackson and his colleagues were able to painstakingly and precisely isolate each and every audio track — be it musical instrumentation, singing or studio chatter — from the original mono recordings made for most of "Let It Be." "What we've managed to do is split it all apart in a way that is utterly clean and sounds much better," Jackson said.
Other interesting observations from the Post:
- "Get Back" tapped nearly 120 hours of previously unheard audio recordings. Jackson and his team started work in 2017.
- Jackson's team also "carefully restored, upgraded and enlarged the grainy original 16-millimeter" footage from the 1969 documentary Let It Be "so that it now pops with vibrant color."
- Jackson's documentary "was originally set to open in theaters last year as a two-and-a-half hour feature film, but was pushed back by the pandemic. With more time unexpectedly on his hands, Jackson transformed his feature film into the six-hour epic...."
- Jackson would also like to release an expanded director's cut sometime in the future, "but there are no current plans to do so."
- "At one point, Jackson's favorite version of his Get Back film clocked in at 18 hours..."
Rigged!, fake!, I'm onto them! (Score:2, Funny)
...Even the Beatles are not immune to fakery! The Deep State probably sent Yoko to break the Beatles up. She looks like a commie, Japan is not sending their best, I know these things because I have the best mind and eyes, everyone says so! #MBGA!"
Re: (Score:2)
Looks great (Score:3)
I was watching this in 4k and it looks like it was recorded yesterday. It’s really for the fans who want to see the creative process behind the songs.
Re: (Score:2)
It's a pretty amazing film. I'm on my second run through, and it's an amazing snapshot in time.
Re: Looks great (Score:1)
Re: (Score:2)
I was watching this in 4k and it looks like it was recorded yesterday. It’s really for the fans who want to see the creative process behind the songs.
Yes! I actually started out thinking they had hired re-enactors to portray whatever grainy 1969-quality audio and video was captured at the time, and figured I'd see credits at the end "Paul McCartney body double portrayed by so-and so..."
For all I've complained about Jackson's penchant for extending the money milking machine as far as it can go (I'm talkin' to YOU, The Hobbit!) he did an amazing job restoring archival data.
I've been posting occasional reviews on Facebook as I progress through the series, f
Re: (Score:2)
Yes! I actually started out thinking they had hired re-enactors to portray whatever grainy 1969-quality audio and video was captured at the time, and figured I'd see credits at the end "Paul McCartney body double portrayed by so-and so..."
But it already was a body double by this time. Abbey Road cover - could it be more clear? :).
Re: Looks great (Score:2)
What could have been. (Score:5, Funny)
Re: (Score:2)
Why? If anything, the doc show's how much of a non-factor she was in their breakup. Paul's obsessive micromanagement was the main driver in splintering the group.
Re: (Score:3)
Huh, what I saw was a petulant George Harrison acting like a prima donna, and actually leaving the band. McCartney was trying to keep George and a smack addicted Lennon on track.
Re: (Score:2)
Maybe. That's probably how he saw it. But like most control freaks, he doesn't really consider the feelings and emotions part until after the damage has been done.
Re: (Score:2)
Of course thats how he saw it, thats how it was. Macca was easily the most talented writer, musician and singer in the band, and by far the most successful afterwards, giving great live performance for the next 45 years.
Re: (Score:2)
Re: (Score:1)
If he wanted to do things "right" and the others just wanted to goof off, then maybe it was time to break-up. They had enough money that they didn't need to be slave to a 9-to-5 clock anymore.
Re: What could have been. (Score:2)
Re: (Score:1)
> casts a light on just how much of a no-talent-assclown she was though. Her 'Art' was a joke
I'm confident that with clever use of reverb she could make some interesting tunes. A lot of voices that are scratchy or shriek when raw make the best reverb and effects inputs. One of these days I may remix some of her work. (I'm an amateur musician, very amateur.)
Brian Epstein (Score:1)
More than one Beatle said Brian Epstein's death was a major cause, as he knew how to keep the peace.
Re: (Score:2)
I think they even mention this in the doc.
Why? (Score:2)
What did she do?
Yoko a scapegoat (Score:2)
The breakup was NOT her fault. All 4 were drifting apart. Both Ringo and George had walked off the set, for example, and it was over creative differences, not Yoko. They also had growing families and wanted to spend time at home. Ringo said he even liked Yoko hanging around, she had a quirky sense of humor that helped pass long days in the studio.
Re: (Score:2)
Well, for starters, she "sang." And that's enough. Guessing you post-date The Beatles.
Re: (Score:2)
One thing that the show reveals is that Yoko was
basically a non-entity in the making of Let It Be, just sitting
there listening to the music. She didn't do any good or any
harm.
A light touch needed (Score:4, Interesting)
I've heard current remastered versions of albums like Abbey Road. They sound good. Too good, like a 21st century recording of an excellent tribute band. I know the Beatles had the money to shoot lots of 35mm, but a light touch is still needed to preserve the history.
...laura
Re: (Score:3)
There is a remastered version called Let It Be Naked which is supposed to sound much better than Phil Spector’s wall of sound nonsense.
Re: (Score:2)
Let It Be Naked which is supposed to sound much better than Phil Spector's wall of sound nonsense.
Supposed to? Believe me, it does.
Re: (Score:2)
There is a remastered version called Let It Be Naked
I searched for this on a well-known video site [pornhub.com] but all the results I got didn't seem to feature the Beatles in them. Is there a particular search term I need to use?
What about Mal Evans? (Score:3)
Very interesting technology for the cops and TLAs (Score:2)
Scary
MAL HAL IBM (Score:1)
OK I'm 20 years slow but two days ago was informed IBM shr = HAL
2.5 hours transformed into 6 hour epic (Score:1)
> Jackson's documentary "was originally set to open in theaters last year as a two-and-a-half hour feature film, but was pushed back by the pandemic. With more time unexpectedly on his hands, Jackson transformed his feature film into the six-hour epic...."
I'm shocked! Shocked to find that Peter Jackson would do such a thing.
Re: (Score:2)
I'm shocked! Shocked to find that Peter Jackson would do such a thing.
Yeah, and perhaps just as unexpected:
Jackson would also like to release an expanded director's cut sometime in the future
Re: 2.5 hours transformed into 6 hour epic (Score:2)
I don't buy it (Score:2)
Re:I don't buy it (Score:4, Interesting)
For the most part, the original audio recordings should be high-quality. They knew they were filming a documentary, and mention it repeatedly in the footage. You can see, in places, the film crew holding boom mics above people to capture their conversation clearly. They also knew they were recording an album. Most of the footage is from inside the recording studio. Any time those tapes were rolling, Peter Jackson would have multiple tracks from different mics all around the room to choose from.
So it's not exactly home video or amateur footage that he was working with. Although there are a few scenes (like with the microphone hidden in the flower pot) where you can tell they had to run a noisy recording through your normal NR process. Or where you can tell they had to crop and zoom from a distant shot to show someone's facial expression, so that the shot looks blurrier than the rest of the footage. (I guess they were really concerned with removing any trace of film grain.)
So, basically the same kind of "custom AI" you've been able to get as freeware for many years.
Re: (Score:2)
I'm just not buying the whole âoeAIâ thing
Whatever it is, good call, Jarjar. Save your money.
Re: I don't buy it (Score:1)
Many Thanks (Score:2)
to Jackson and his Team for their groundbreaking work on restoration techniques. I'm looking forward to this being applied to many other relics to bring them back to life!
Please no... (Score:2)
Jackson would also like to release an expanded director's cut sometime in the future...
I hope not. And I speak as someone who has the utmost musical respect for the Beatles and their contribution to the world of rock. Even though the movie exposed the methodology behind their creativity, most of the time it was like sitting through a group's most aimless and boring band practice. Expanding the experience would not add to it.
Re:Please no... (Score:4, Informative)
It isn't stated enough that this is fans-only material. If you didn't like "Get Back" before, you definitely won't like hearing a half-finished version of the chorus, played 10 times in a row, while John and Paul argue about how to play it. Especially not for 8 hours. I had to split it up between many days of viewing.
I'm not even confident that this should be considered as the definitive cut, either. Peter Jackson said that he viewed his version as a companion piece to the original 1970 cut, that he was trying to tell "the rest of" the story that was left out. That he was trying to avoid using footage that made it into the 1970 cut. If you have seen the 1970 version, that's maybe a good idea. But did he pass over some important footage in an attempt to present only "new" material? My guess is yes.
I feel like I need to see the 1970 cut on top of Peter Jackson's, to get the full picture. And at that point, you might as well just sit in the mancave with PJ and watch all 80 hours of footage he has. Non Beatles-fans... stay away!
Re: (Score:1)
Jackson would also like to release an expanded director's cut sometime in the future...
I hope not.
Maybe he wants to add some skateboard scenes.
Re: (Score:2)
AI code to fix Yellow Submarine revealed (Score:2)
rm -fr /*