The Eerie AI World of Deepfake Music (theguardian.com) 44
Artificial intelligence is being used to create new songs seemingly performed by Frank Sinatra and other dead stars. 'Deepfakes' are cute tricks -- but they could change pop for ever. From a report: "It's Christmas time! It's hot tub time!" sings Frank Sinatra. At least, it sounds like him. With an easy swing, cheery bonhomie, and understated brass and string flourishes, this could just about pass as some long lost Sinatra demo. Even the voice -- that rich tone once described as "all legato and regrets" -- is eerily familiar, even if it does lurch between keys and, at times, sounds as if it was recorded at the bottom of a swimming pool. The song in question not a genuine track, but a convincing fake created by "research and deployment company" OpenAI, whose Jukebox project uses artificial intelligence to generate music, complete with lyrics, in a variety of genres and artist styles. Along with Sinatra, they've done what are known as "deepfakes" of Katy Perry, Elvis, Simon and Garfunkel, 2Pac, Celine Dion and more. Having trained the model using 1.2m songs scraped from the web, complete with the corresponding lyrics and metadata, it can output raw audio several minutes long based on whatever you feed it. Input, say, Queen or Dolly Parton or Mozart, and you'll get an approximation out the other end.
"As a piece of engineering, it's really impressive," says Dr Matthew Yee-King, an electronic musician, researcher and academic at Goldsmiths. (OpenAI declined to be interviewed.) "They break down an audio signal into a set of lexemes of music -- a dictionary if you like -- at three different layers of time, giving you a set of core fragments that is sufficient to reconstruct the music that was fed in. The algorithm can then rearrange these fragments, based on the stimulus you input. So, give it some Ella Fitzgerald for example, and it will find and piece together the relevant bits of the 'dictionary' to create something in her musical space." Admirable as the technical achievement is, there's something horrifying about some of the samples, particularly those of artists who have long since died -- sad ghosts lost in the machine, mumbling banal cliches. "The screams of the damned" reads one comment below that Sinatra sample; "SOUNDS FUCKING DEMONIC" reads another. We're down in the Uncanny Valley. Deepfake music is set to have wide-ranging ramifications for the music industry as more companies apply algorithms to music. Google's Magenta Project -- billed as "exploring machine learning as a tool in the creative process" -- has developed several open source APIs that allow composition using entirely new, machine-generated sounds, or human-AI co-creations. Numerous startups, such as Amper Music, produce custom, AI-generated music for media content, complete with global copyright. Even Spotify is dabbling; its AI research group is led by Francois Pachet, former head of Sony Music's computer science lab.
"As a piece of engineering, it's really impressive," says Dr Matthew Yee-King, an electronic musician, researcher and academic at Goldsmiths. (OpenAI declined to be interviewed.) "They break down an audio signal into a set of lexemes of music -- a dictionary if you like -- at three different layers of time, giving you a set of core fragments that is sufficient to reconstruct the music that was fed in. The algorithm can then rearrange these fragments, based on the stimulus you input. So, give it some Ella Fitzgerald for example, and it will find and piece together the relevant bits of the 'dictionary' to create something in her musical space." Admirable as the technical achievement is, there's something horrifying about some of the samples, particularly those of artists who have long since died -- sad ghosts lost in the machine, mumbling banal cliches. "The screams of the damned" reads one comment below that Sinatra sample; "SOUNDS FUCKING DEMONIC" reads another. We're down in the Uncanny Valley. Deepfake music is set to have wide-ranging ramifications for the music industry as more companies apply algorithms to music. Google's Magenta Project -- billed as "exploring machine learning as a tool in the creative process" -- has developed several open source APIs that allow composition using entirely new, machine-generated sounds, or human-AI co-creations. Numerous startups, such as Amper Music, produce custom, AI-generated music for media content, complete with global copyright. Even Spotify is dabbling; its AI research group is led by Francois Pachet, former head of Sony Music's computer science lab.
Perfect (Score:2)
Old music now sells more than new music, whether because people consuming new music don't pay (possible) or because 99% of new music is indistinguishable from rehashed versions of cheaper old music (equally possible IMO.) So if you're going to cater to people who buy music, it's better to rehash the oldies than to create something "new" (which isn't really new.)
Re: (Score:2)
Take "Despacito" for example. The whole song is practically 3 or 4 chords played repeatedly for an entire song.
Simply not true: https://www.youtube.com/watch?... [youtube.com]
Re: (Score:2)
So? At four chords, it's got one more chord than, say, Jimi Hendrix's "All Along The Watchtower", two more than either Aretha Franklin's "Chain of Fools" or Miles Davis' "So What", and a whole three chords more than Howling Wolf's "Moaning at Midnight", Led Zepplin's "Whole Lotta Love", George Thorogood's "Bad to the Bone" and Henry Mancini's "Peter Gunn Theme".
If chord counts are your measure of what makes a song "crap" or not, you have a hell of a lot to learn about how to use your ears.
Re: (Score:2)
sorry, I replied to the wrong comment
I have a use for it (Score:1)
Easy and just create random job performance review via the AI model and feed it into the next performance revidw for the employees on the big data fake AI projects.
Re: (Score:2)
So? At four chords, it's got;
One more chord than Jimi Hendrix's "All Along The Watchtower"
Two more than either Aretha Franklin's "Chain of Fools" or Miles Davis' "So What"
And a whole three chords more than Howling Wolf's "Moaning at Midnight", Led Zepplin's "Whole Lotta Love", and Henry Mancini's "Peter Gunn Theme".
If chord counts are your measure of what makes a song "crap" or not, you have a hell of a lot to learn about how to use your ears.
Re: (Score:2)
This is nonsensical.
Rock and roll has always been accused of being 3-4 chords. Theres a famous quote , Keith Richards was asked "Do your songs only have 3 chords?", to which he replied "Oh no! Some even have 4!"
And before that Tin Pan Alley, and Folk where also accused of this sort of thing. Hell if you removed the ii-IV-I progression you'd kill off 3/4 of jazz, and the Blues is an entire genre defined around the "12 bar" 3 chord progression.
Literally nothing has changed in that respect.
Re: (Score:2)
Old music now sells more than new music
I didn't even know this was true, and looking at the data [vice.com] shows that old music isn't catching up as much as new music is falling off a cliff. Old music sales have been dropping too, so it isn't like people are moving from new music to old music. It looks more like the people who buy new music are not buying music all together at a rate much higher than those who purchase old music.
So it is probably less that music today is derivative (my guess is this has always been true for almost all forms of entertainme
Lots of story for just one sample (Score:3)
Re: (Score:2)
Re: (Score:1)
Re:Lots of story for just one sample (Score:5, Informative)
Go to the source: https://jukebox.openai.com/ [openai.com]
Take a listen to the source (Score:2)
Re: (Score:1)
Agreed, and much of the voices sound like the singer had his head in a bucket or something like that.
Composers of elevator music and shallow pop music might soon have a problem though. These fake "compositions" are good enough as muzak, even if they lack things like recognizable themes and dramatic highlights.
Now all that is needed is cleaning up If the sound quality.
Re: (Score:2)
Input, say, Queen or Dolly Parton or Mozart, and you'll get an approximation out the other end.
Input, say, a recording of an above-ground nuclear weapons test recorded live in hell and you've got Grindcore out the other end. Magic.
Does this mean??? (Score:2)
That Nickelback or Rebecca Black are also options? *shudder*
Just what I always wanted! (Score:3)
Deepfake elevator music thanks to AI. That may actually be an improvement ...
The 9th Concentric Circle of AI Hell (Score:5, Funny)
Re: (Score:3)
Or of Rick Astley based on his song "Never Gonna Give You Up". It won't be real though until they deep fake the video as well.
nat king cole/natalie cole (Score:2)
Didn't they already sorta do this a few years back?
https://www.youtube.com/watch?... [youtube.com]
Re: (Score:1)
Rehash from May (Score:5, Interesting)
This is is rehash from May when these songs where released:
https://entertainment.slashdot... [slashdot.org]
The Frank Sinatra song was just one of many released at the time. Here is a much better article (from the original Slashdot story) with technical details:
https://venturebeat.com/2020/0... [venturebeat.com]
Apparently Guardian picked up on this half a year later and ran a story specific to the one Frank Sinatra song, and so here it is on Slashdot again.
I had listened to many of the samples in different genres. Some are more passable than others, while a number of them are downright awful. To me, these kinds of Deep Fake AI products (same as images, text, etc) could find good use in video games, to create a much deeper world but without having to rely on existing works, both for copyright issues, and for greater customization of the content to suit the game. Imagine Fallout with its big-band and 40s era music, that was custom generated thematically for the game. Same with the tomes of books found everywhere in games like Skyrim - they could be produced by AI. In fact, the books could be generated dynamically, affected by the choices the player made in the game.
More "A.I.-generated" examples (Score:2)
https://make.girls.moe/ [girls.moe]
Doesn't seem to be working even on the latest Chrome.
https://www.thiswaifudoesnotex... [thiswaifud...texist.net]
Some are downright awful and creepy, others are up to the level of a beginner-but-skilled artist.
https://waifulabs.com/ [waifulabs.com]
Quite amazing results for the most part.
Fuck those projects (Score:2)
You want to create new songs by long-deceased artists?
Use your fancy algorithms to generate the lyrics if you want, then use Vocaloid to really make new songs. Surely you can fine-tune existing voices to match the voice you want.
Re: (Score:2)
But I don't want to tell my simulated Jim Morrison how to sing (nor do I want to write his lyrics). Let it find its own way, the same as its simulated Jimi Hendrix, Chris Squire and Ginger Baker accompaniment.
1.2m songs scraped from the web (Score:1)
> 1.2m songs scraped from the web
They couldn't even be bothered to buy the albums of the artists they're faking.
I assume OpenAI will be OK with people pirating their commercial software.
Deepfake Music? (Score:2)
I thought it was called "autotune".
Re: (Score:2)
Oh sure... (Score:3)
We've eliminated musicians' ability to make a living selling CDs.
Covid-19 has eliminated their ability to make a living through touring, and will soon shutter most small venues for good anyway.
Let's make sure we stamp out publishing royalties as an opportunity for revenue, too.
Re: (Score:2)
Those venues will come back. Even in places where there are these quasi-lockdowns, you are seeing the green shoots. Restaurants, bars, music clubs will all be back.
Music isn't going to die because of COVID.
Re: (Score:2)
Agreed. It's what humans do.
Why shouldn't we? (Score:2)
Is there something special about being a musician that should give them special employment protection when the advancement of technology puts everyone ELSE out of work?
Re: (Score:2)
Let's make sure we stamp out publishing royalties as an opportunity for revenue, too.
No, the Big Boys will not allow it. "In The Future(tm)", Disney and other corps will pay big money for the best MAIs (Musical AI), and will want payment for each bastardized tune.. They will flood their channels with it, get influencers, get fans. These fans will study everything they can about the MAIs, and will illuminate or imaginate regional differences. Advertisers will pay Big Bucks to insert their preferred brand words into the lyrics. Any time, any style.
--
But, I also predict that hordes of pre-teen
Pretty horrible (Score:1)
Re: (Score:3)
The quality difference between the songs is huge. Ella Fitzgerald was pretty convincing https://jukebox.openai.com/?so... [openai.com] and swapping the text is a great trick. Shakira on the other hand was unrecognizable. The text was hilarious though. It's as if Trump wrote a love song:
I'm an absolute beginner
And I'm absolutely sane
As long as we're together
The rest can go to hell
Is this new? (Score:2)
I always thought they began doing this since the day Jim Morrison died.
And now get off my lawn.
This AI needs Autotune (Score:2)
Badly!
Sorry, guys, but for once I really HAD to RTFA.
Won't happen again.
Simon and Garfunkel lyrics about safe OpenAI ! (Score:2)
This link
https://jukebox.openai.com/?so... [openai.com]
Shows Simon and Garfunkel lyrics about safe OpenAI.
Oh safe A. I.
Our goal to make sure
Everyone can benefit
From A. G. I.
(Everyone, everyone)
Might sound silly,
But we're very serious,
All of us here at Open A. I.
Trying to build A. I.
To benefit humanity
(Everyone, everyone)
This is why
We work together.
Everyone at Open A. I.,
Is giving everything they have,
So that you,
And everyone else,
Can benefit
Maybe the training corpus is being nudged to be benign? ;-)
What's the difference (Score:2)
Pop music today sounds so much like corporate ordered souless artificial crap, AI generated music can't possibly be worse.
Yes, I sound like one of those old geezers who chant "Well back in my day..." but the truth is is that it's target audience is noticing the same thing and complaining about it. This is what happens when money and corporate culture take over everything, and music is expected to be pumped out like McDonalds burgers, as if "writer's block" never existed.
This isn't new, as shit filler tracks
Lyrics? (Score:2)
Some of the words are unintelligible, and as TFA says, sounds downright demonic. I can't really describe it well, but 'Fake Sinatra' sings normally and sounds like the real deal, then devolves into mumbling for a moment followed with a very frightning, maniac-sounding vocal 'sting' (not to be confused with the eccentric but normal stings of the real singer). Also, the song ends in a rather awkward, incomplete way
All in all, it's very interesting to listen to, if nothing else.