'Biologically Plausible' Deep Learning Neurons Predict the Chords of Bach (ibm.com) 24
IBM's research blog shares an article about "polyphonic music prediction using the Johann Sebastian Bach chorales dataset" achieved by using "biologically plausible neurons," a new approach to deep learning "that incorporates biologically-inspired neural dynamics and enables in-memory acceleration, bringing it closer to the way in which the human brain works."
At IBM Research Europe we have been investigating both Spiking Neural Networks (SNNs) and Artificial Neural Networks (ANNs) for more than a decade, and one day we were struck with the thought: "Could we combine the characteristics of the neural dynamics of a spiking neuron and an ANN?" The answer is yes, we could. More specifically, we have modelled a spiking neuron using a construct comprising two recurrently-connected artificial neurons — we call it a spiking neural unit (SNU)... It enables a reuse of architectures, frameworks, training algorithms and infrastructure. From a theoretical perspective, the unique biologically-realistic dynamics of SNNs become available for the deep learning community...
Furthermore, a spiking neural unit lends itself to efficient implementation in artificial neural network accelerators and is particularly well-suited for applications using in-memory computing. In-memory computing is a promising new approach for AI hardware that takes inspiration from the architecture of the brain, in which memory and computations are combined in the neurons. In-memory computing avoids the energy cost of shuffling data back and forth between separate memory and processors by performing computations in memory — phase change memory technology is a promising candidate for such implementation, which is well understood and is on its way to commercialization in the coming years. Our work involves experimental demonstration of in-memory spiking neural unit implementation that exhibits a robustness to hardware imperfections that is superior to that of other state-of-the-art artificial neural network units...
The task of polyphonic music prediction on the Johann Sebastian Bach dataset was to predict at each time step the set of notes, i.e. a chord, to be played in the consecutive time step. We used an SNU-based architecture with an output layer of sigmoidal neurons that allows a direct comparison of the obtained loss values to these from ANNs. The SNU-based network achieved an average loss of 8.72 and set the SNN state-of-the-art performance for the Bach chorales dataset. An sSNU-based network further reduced the average loss to 8.39 and surpassed corresponding architectures using state-of-the-art ANN units.
Slashdot reader IBMResearch notes that besides being energy-efficient, the results "point towards the broad adoption of more biologically-realistic deep learning for applications in artificial intelligence."
Furthermore, a spiking neural unit lends itself to efficient implementation in artificial neural network accelerators and is particularly well-suited for applications using in-memory computing. In-memory computing is a promising new approach for AI hardware that takes inspiration from the architecture of the brain, in which memory and computations are combined in the neurons. In-memory computing avoids the energy cost of shuffling data back and forth between separate memory and processors by performing computations in memory — phase change memory technology is a promising candidate for such implementation, which is well understood and is on its way to commercialization in the coming years. Our work involves experimental demonstration of in-memory spiking neural unit implementation that exhibits a robustness to hardware imperfections that is superior to that of other state-of-the-art artificial neural network units...
The task of polyphonic music prediction on the Johann Sebastian Bach dataset was to predict at each time step the set of notes, i.e. a chord, to be played in the consecutive time step. We used an SNU-based architecture with an output layer of sigmoidal neurons that allows a direct comparison of the obtained loss values to these from ANNs. The SNU-based network achieved an average loss of 8.72 and set the SNN state-of-the-art performance for the Bach chorales dataset. An sSNU-based network further reduced the average loss to 8.39 and surpassed corresponding architectures using state-of-the-art ANN units.
Slashdot reader IBMResearch notes that besides being energy-efficient, the results "point towards the broad adoption of more biologically-realistic deep learning for applications in artificial intelligence."
I like Bach (Score:2)
When I see Teens playing it on an electric guitar, my neurons fire.
https://www.youtube.com/watch?... [youtube.com]
Re:I like Bach (Score:5, Insightful)
Re: (Score:3)
Re: (Score:1)
Re: (Score:2)
Nicely calibrated Elect whistle, there.
Well played, sir.
Predicting Bach (Score:2)
I had read somewhere, maybe back in the 80s, that some people find his music too predictable and/or mathematical. Like IBM's efforts at diagnosing cancer, I wonder if they've picked an easy data set and are patting themselves on the back for it. Write me when they predict Erik Satie or John Cage.
Re: (Score:2)
Re: (Score:2)
Write me when they predict Erik Satie or John Cage.
Meh. I've had Windows predict one of Cage's tunes many times.
Bach? (Score:2)
Well, really only one thing to say, then.
Soli Deo Gloria.
Re: (Score:1)
Well, really only one thing to say, then.
Soli Deo Gloria.
Grape soda and Sherman Oaks real-estate valuation are evidence of God? I guess...
Wayne Rogers' precedent for walking off that set might be divine intervention.
But Burghoff is alive, and Doppler 3000 is a mystery only Joel Hodgson could have appreciated given an affinity for IBM...or Jim Mallon.
Re: (Score:2)
Lost looking for the First Circle, (bura)Virgil?
Re: (Score:2)
Also Bach Zarathustra.
"biologically plausible" (Score:1)
Re: (Score:1)
Damn, IBM Is really getting desperate. NN are nothing like the human brain.
It depends on the NNs. Spiking NNs are much closer.
Re: (Score:2)
As the understanding of neural structures grows... (Score:3, Insightful)
Re: (Score:3)
Always fun to see militantly atheist professors of philosophy grind to a halt on this [owl232.net].
Re: (Score:2)
Short answer: no. There is more to us than electrical signals. We have thoughts, but it isn't our thoughts that drive us. We are driven not only by our senses alone, but by emotions, impulses and hormones, too. Our senses make our conscious, but below that is our subconscious and with it all that we don't experience directly. So when for example you're hungry then your body sends you an impulse and your thoughts change and you begin to search for food. Your brain starts to explore all options, from how much
IBM Circling the drain (Score:1)
It's got to the point that when I read that IBM have done X, Y, or Z I just assume they're lying. IBM hasn't done squat for 25 years.
Re: (Score:1)
Biological-like AI (Score:2)
Why the hell would we want that? No. Biological organisms, Humans, are Fâ(TM)d up. No thanks.
Thank you for the comments (Score:2)
#ConsciousnessAttracts #Abstractions (Score:1)
#TheHardProblem
Sir Roger Penrose and Dr Stuart Hameroff explain in this video the possibility of intra-neuronal quantum resonant computing as physical representiveness of medium for consciousness; Clearly rooted in a practical application of Anaesthesiology.
https://youtu.be/xGbgDf4HCHU?t... [youtu.be]
(watch whole video for full effect)
I wonder how stephen wolfram's hyperobjects would fit in the calculation scheme of 10E27 calculations per second when their microtubule paradigm holds to be useful.
The idea that neural-n