Follow Slashdot stories on Twitter

 


Forgot your password?
Close
typodupeerror
×
AI Music

Data Scientist Tries AI/Human Collaboration For Audio-Visual Art (lionbridge.ai) 26

Posted by EditorDavid from the new-frontiers dept.
"Swirls of color and images blend together as faces, scenery, objects, and architecture transform to music." That's how AI training company Lionbridge is describing Neural Synesthesia.

Slashdot reader shirappu explains: Neural Synesthesia is an AI art project that creator Xander Steenbrugge calls a collaboration between man and machine. To create each piece, he feeds a generative network with curated image datasets and combines the ever-transforming results with music that is programmed to control the shifting visuals.
Steenbrugge describes how the music controls the visuals in an interview with Lionbridge: I think coding for the first rendered video took over six months because I was doing it in my spare time. The biggest challenge was how to manipulate the generative adversarial network (GAN)'s latent input space using features extracted from the audio track. I wanted to create a satisfying match between visual and auditory perception for viewers.

I apply a Fourier Transform to extract time varying frequency components from the audio. I also perform harmonic/percussive decomposition, which basically separates the melody from the rhythmic components of the track. These three signals (instantaneous frequency content, melodic energy, and beats) are then combined to manipulate the GANs latent space, resulting in visuals that are directly controlled by the audio...

[Y]ou are not limited by your own imagination. There's an entirely alien system that is also influencing the same space of ideas, often in unexpected and interesting ways. This leads you as a creator into areas you never would have wandered by yourself.
This discussion has been archived. No new comments can be posted.

Data Scientist Tries AI/Human Collaboration For Audio-Visual Art More

Data Scientist Tries AI/Human Collaboration For Audio-Visual Art

Comments Filter:
  • Comment removed based on user account deletion

    • Maybe they could work "blockchain" in it and get all the buzzword brownie points.

      And, of course, be sure to mention the art is proven good by "settled science".

    • "Tonight we pose the question: What Is Art?"
      "You mean like, paintings and stuff?"
      "Yes. Goodnight."

    • That is the type of description of "art" that appeals to some of the weirdest people in society.

      Disclosure: I am a scientist, with a sympathy towards art. (I'm also an amateur musician.) I have met many artists devoted to various media. Some artists are weird, but in no way would I say that artists or their supporters are "some of the weirdest people in society." There may be a few poseurs, but generally the artistic community is serious and dedicated towards creating things that are worthy of consideration.

      I wonder if digital artists refer to their craft as "human/GIMP collaboration" and refer to the exact technological processed used in this way?

      "I apply an alpha layer to the background, and use Bézier curves to simulate smooth gradients across an 32 bit colour pallet to create the effect of real imagery on a liquid crystal display device."

      Well, if the techniques in question are innovative or revolutionary, then I suppose they might

      • Comment removed based on user account deletion

        • Thank you very much for your response. It deserves a well-considered reply.

          I'll freely concede that the issue of "artistic collaboration" is a philosophical one when it involves interaction of a human with a non-human. But I would say that the issue is hardly new. The most striking examples might be the 20th century minimalist projects where the initial states of the media (canvas, wall, etc.) are considered a major component in the work. Or the musical equivalent in John Cage's 4'33" where the ambient soun

    • That is the type of description of "art" that appeals to some of the weirdest people in society.

      It appeals to me. I think they are interesting pictures.

      I just have one question: What's the human for?

    • Re: (Score:2)

      by fazig ( 2909523 )
      You answered your own question more or less.
      The former half of that pseudo quote makes sense, so I can assume that you have at least some idea of the terms that digital artists use. The latter half with the "effect of real imagery on a liquid crystal display device" sounds a nonsensical to me and to you probably as well. And to someone who has no idea of digital art the entire thing might sound as nonsensical as Star Trek technobabble, which to them might be indistinguishable from a genuine scientific pape

  • AI image fiddling can be creepy enough without weird art projects. Let it be.

  • Observing AI and robots and being aware that they are there where microcomputers where in the mid-80ies feels weird. AI is probably going to replace large parts of my job as a developer in my lifetime which sometimes has me question wether learning new technology X or Y even makes sense in the mid to long term.

    Very weird indeed.

    I definitely have to ramp up my yoga and exercise skills, that's an investment a robot won't be able to replace on my end.

  • This is basically a fancy winamp visualization plugin. Although we haven’t used MP3 players in a while so it feels new again.

  • Artist discovers morphing, dresses it in AI word salad.

Slashdot Top Deals

You know you've landed gear-up when it takes full power to taxi.

Close