Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
AI Music

The Eerie AI World of Deepfake Music (theguardian.com) 44

Posted by msmash from the closer-look dept.
Artificial intelligence is being used to create new songs seemingly performed by Frank Sinatra and other dead stars. 'Deepfakes' are cute tricks -- but they could change pop for ever. From a report: "It's Christmas time! It's hot tub time!" sings Frank Sinatra. At least, it sounds like him. With an easy swing, cheery bonhomie, and understated brass and string flourishes, this could just about pass as some long lost Sinatra demo. Even the voice -- that rich tone once described as "all legato and regrets" -- is eerily familiar, even if it does lurch between keys and, at times, sounds as if it was recorded at the bottom of a swimming pool. The song in question not a genuine track, but a convincing fake created by "research and deployment company" OpenAI, whose Jukebox project uses artificial intelligence to generate music, complete with lyrics, in a variety of genres and artist styles. Along with Sinatra, they've done what are known as "deepfakes" of Katy Perry, Elvis, Simon and Garfunkel, 2Pac, Celine Dion and more. Having trained the model using 1.2m songs scraped from the web, complete with the corresponding lyrics and metadata, it can output raw audio several minutes long based on whatever you feed it. Input, say, Queen or Dolly Parton or Mozart, and you'll get an approximation out the other end.

"As a piece of engineering, it's really impressive," says Dr Matthew Yee-King, an electronic musician, researcher and academic at Goldsmiths. (OpenAI declined to be interviewed.) "They break down an audio signal into a set of lexemes of music -- a dictionary if you like -- at three different layers of time, giving you a set of core fragments that is sufficient to reconstruct the music that was fed in. The algorithm can then rearrange these fragments, based on the stimulus you input. So, give it some Ella Fitzgerald for example, and it will find and piece together the relevant bits of the 'dictionary' to create something in her musical space." Admirable as the technical achievement is, there's something horrifying about some of the samples, particularly those of artists who have long since died -- sad ghosts lost in the machine, mumbling banal cliches. "The screams of the damned" reads one comment below that Sinatra sample; "SOUNDS FUCKING DEMONIC" reads another. We're down in the Uncanny Valley. Deepfake music is set to have wide-ranging ramifications for the music industry as more companies apply algorithms to music. Google's Magenta Project -- billed as "exploring machine learning as a tool in the creative process" -- has developed several open source APIs that allow composition using entirely new, machine-generated sounds, or human-AI co-creations. Numerous startups, such as Amper Music, produce custom, AI-generated music for media content, complete with global copyright. Even Spotify is dabbling; its AI research group is led by Francois Pachet, former head of Sony Music's computer science lab.
This discussion has been archived. No new comments can be posted.

The Eerie AI World of Deepfake Music More

The Eerie AI World of Deepfake Music

Comments Filter:

  • Old music now sells more than new music, whether because people consuming new music don't pay (possible) or because 99% of new music is indistinguishable from rehashed versions of cheaper old music (equally possible IMO.) So if you're going to cater to people who buy music, it's better to rehash the oldies than to create something "new" (which isn't really new.)

    • Re: (Score:2)

      by ranton ( 36917 )

      Old music now sells more than new music

      I didn't even know this was true, and looking at the data [vice.com] shows that old music isn't catching up as much as new music is falling off a cliff. Old music sales have been dropping too, so it isn't like people are moving from new music to old music. It looks more like the people who buy new music are not buying music all together at a rate much higher than those who purchase old music.

      So it is probably less that music today is derivative (my guess is this has always been true for almost all forms of entertainme

  • Lots of story for just one sample (Score:3)

    by Visarga ( 1071662 ) on Tuesday November 10, 2020 @11:03AM (#60707490)
    I was expecting more samples.

  • That Nickelback or Rebecca Black are also options? *shudder*

  • Just what I always wanted! (Score:3)

    by Kiliani ( 816330 ) on Tuesday November 10, 2020 @11:16AM (#60707530)

    Deepfake elevator music thanks to AI. That may actually be an improvement ...

  • The 9th Concentric Circle of AI Hell (Score:5, Funny)

    by PeeAitchPee ( 712652 ) on Tuesday November 10, 2020 @11:19AM (#60707538)
    Deep fakes of Celine Dion songs (shudders).

    • Re: (Score:3)

      by gtall ( 79522 )

      Or of Rick Astley based on his song "Never Gonna Give You Up". It won't be real though until they deep fake the video as well.

  • Didn't they already sorta do this a few years back?

    https://www.youtube.com/watch?... [youtube.com]

    • YouTube is blocked where I am right now, but if it's the Nat King Cole/Natalie Cole duet on "Unforgettable" you are referring to, they took the original Nat King Cole song and spliced it with a recording of her. It was closer to a remaster in the technology involved.

  • Rehash from May (Score:5, Interesting)

    by Dan East ( 318230 ) on Tuesday November 10, 2020 @11:24AM (#60707544) Journal

    This is is rehash from May when these songs where released:
    https://entertainment.slashdot... [slashdot.org]

    The Frank Sinatra song was just one of many released at the time. Here is a much better article (from the original Slashdot story) with technical details:
    https://venturebeat.com/2020/0... [venturebeat.com]

    Apparently Guardian picked up on this half a year later and ran a story specific to the one Frank Sinatra song, and so here it is on Slashdot again.

    I had listened to many of the samples in different genres. Some are more passable than others, while a number of them are downright awful. To me, these kinds of Deep Fake AI products (same as images, text, etc) could find good use in video games, to create a much deeper world but without having to rely on existing works, both for copyright issues, and for greater customization of the content to suit the game. Imagine Fallout with its big-band and 40s era music, that was custom generated thematically for the game. Same with the tomes of books found everywhere in games like Skyrim - they could be produced by AI. In fact, the books could be generated dynamically, affected by the choices the player made in the game.

  • You want to create new songs by long-deceased artists?

    Use your fancy algorithms to generate the lyrics if you want, then use Vocaloid to really make new songs. Surely you can fine-tune existing voices to match the voice you want.

    • Re: (Score:2)

      by Sloppy ( 14984 )

      But I don't want to tell my simulated Jim Morrison how to sing (nor do I want to write his lyrics). Let it find its own way, the same as its simulated Jimi Hendrix, Chris Squire and Ginger Baker accompaniment.

  • > 1.2m songs scraped from the web

    They couldn't even be bothered to buy the albums of the artists they're faking.

    I assume OpenAI will be OK with people pirating their commercial software.

  • I thought it was called "autotune".

    • not even autotune anymore. I never watch the musical "actor" anymore as the musical "actor" lipsyncs. In a true twist of irony, the comedic actors do their own real time singing, and in last weeks episode, Baldwin actually played the piano and sung (badly) a YMCA hit.

  • Oh sure... (Score:3)

    by jddj ( 1085169 ) on Tuesday November 10, 2020 @12:30PM (#60707740) Journal

    We've eliminated musicians' ability to make a living selling CDs.

    Covid-19 has eliminated their ability to make a living through touring, and will soon shutter most small venues for good anyway.

    Let's make sure we stamp out publishing royalties as an opportunity for revenue, too.

    • Covid-19 has eliminated their ability to make a living through touring, and will soon shutter most small venues for good anyway.

      Those venues will come back. Even in places where there are these quasi-lockdowns, you are seeing the green shoots. Restaurants, bars, music clubs will all be back.

      Music isn't going to die because of COVID.

    • Is there something special about being a musician that should give them special employment protection when the advancement of technology puts everyone ELSE out of work?

    • Let's make sure we stamp out publishing royalties as an opportunity for revenue, too.

      No, the Big Boys will not allow it. "In The Future(tm)", Disney and other corps will pay big money for the best MAIs (Musical AI), and will want payment for each bastardized tune.. They will flood their channels with it, get influencers, get fans. These fans will study everything they can about the MAIs, and will illuminate or imaginate regional differences. Advertisers will pay Big Bucks to insert their preferred brand words into the lyrics. Any time, any style.

      --
      But, I also predict that hordes of pre-teen

  • I guess it's progress although the sound is actually pretty horrible and the song pretty blah. I listened to the Simon and Garfunkel and it definitely didn't generate another Bridge over Troubled Water. As potential sure, but it's far far from ready. Probably a better first step would be elevator music.

    • The quality difference between the songs is huge. Ella Fitzgerald was pretty convincing https://jukebox.openai.com/?so... [openai.com] and swapping the text is a great trick. Shakira on the other hand was unrecognizable. The text was hilarious though. It's as if Trump wrote a love song:

      I'm an absolute beginner
      And I'm absolutely sane
      As long as we're together
      The rest can go to hell

  • I always thought they began doing this since the day Jim Morrison died.

    And now get off my lawn.

  • Badly!

    Sorry, guys, but for once I really HAD to RTFA.
    Won't happen again.

  • This link
    https://jukebox.openai.com/?so... [openai.com]

    Shows Simon and Garfunkel lyrics about safe OpenAI.

    Oh safe A. I.
    Our goal to make sure
    Everyone can benefit
    From A. G. I.
    (Everyone, everyone)
    Might sound silly,
    But we're very serious,
    All of us here at Open A. I.
    Trying to build A. I.
    To benefit humanity
    (Everyone, everyone)
    This is why
    We work together.
    Everyone at Open A. I.,
    Is giving everything they have,
    So that you,
    And everyone else,
    Can benefit

    Maybe the training corpus is being nudged to be benign? ;-)

  • Pop music today sounds so much like corporate ordered souless artificial crap, AI generated music can't possibly be worse.

    Yes, I sound like one of those old geezers who chant "Well back in my day..." but the truth is is that it's target audience is noticing the same thing and complaining about it. This is what happens when money and corporate culture take over everything, and music is expected to be pumped out like McDonalds burgers, as if "writer's block" never existed.

    This isn't new, as shit filler tracks

  • Some of the words are unintelligible, and as TFA says, sounds downright demonic. I can't really describe it well, but 'Fake Sinatra' sings normally and sounds like the real deal, then devolves into mumbling for a moment followed with a very frightning, maniac-sounding vocal 'sting' (not to be confused with the eccentric but normal stings of the real singer). Also, the song ends in a rather awkward, incomplete way

      All in all, it's very interesting to listen to, if nothing else.

Slashdot Top Deals

He has not acquired a fortune; the fortune has acquired him. -- Bion

Close