Whoa, this video is insane. I'm going to send this to my mom who wrote her dissertation about player pianos and all that during the 1920s and 30s and I bet she'll find this fascinating.
@@TheQwuilleran turned out well! I mean, it's not like i was presenting the video to an extremely judgemental committee or anything lol; just my nice mom.
I respect the amount of persistence that it takes to get that system working. Not easy for anyone, let alone someone who machine learning is not their main focus. Really amazing result too.
Always so nice to see interesting music technology pushing the limit into the future instead of obsessing over emulating music tech from the 70s. You rock Benn!
Watching Benn play with the AI was giving me Ghost (1990) vibes in the pottery scene lol. It feels a bit uncanny valley. 🤣 If you don’t watch the video and just listen, it totally passed the turing test to my ears. The final result very much seemed to understand leaving open notes was important, responded to softer key presses in turn, and seemed to avoid overlap where you were playing. Excellent stuff!
One thought on the AI always wanting to play towards the top end of the piano: it’s common to “roll“ chords from a low note to a high note and your frequent single note approach again lower note to a higher note might be telling the AI to start low and go high? maybe?
I noticed in the last part of the video, that every new "block" of melody patterns Benn played were always a bit higher than the last one. Maybe the neural network interpreted that as the logical conclusion for the direction the melodies should go.
wasn’t expecting something so promising! i’m half-spooked and half-excited for the future of this sort of stuff (by the way that shot at 11:35 was awesome)
Every single time i see an animal i think about your video discussing their time perception. EVERY TIME. Finding this channel totally enhanced the way i see the magic of the world. Thanks for sharing!!
You should release this model, it's so amazing! well done :) I'd love to see more AI music related videos, maybe you could cover OpenAI's Jukebox next?
Thank you! It's already being licensed 😂 AI music market is hungry. I'm trying to keep it open for non-business use though so I'll share accordingly once that's all sorted out.
The diagram you showed is the vanilla Feed-Forward style of Neural Network. The important difference in a Recurrent Neural Network is that it expects data that varies with time (or comes in series, like words or musical notes) and has a "memory" of what you have played before (like a delay's feedback loop). Some RNN strategies weight short-term and long-term memories. I don't think that anything you said was really inaccurate. These AI models could end up being the player analog to an impulse response -- load up your favorite musicians and jam along.
Holy shit! This really seems like some super futuristic and innovative technology, totally constantly improving! I wonder if we could train it to play under a huge number of different pianists, to really get the full scope of things? Very stoked to see where this goes ✌🏼
This is really cool. It already feels pretty weird when a generative patch hits an unusually nice sequence out of sheer (semi)random chance, I can't even imagine what it'd feel like for something to nail it with something approaching "frequent"
6:27 "I, too, am a black box." Whenever I see someone worry that an ML process is impossible to completely understand, I often want to ask, "Are you certain you know how *you* work? You might have a story about how you decide things, buuut ... are you sure it's better and more reliable?"
Do I understand what you are saying ?? Not fully … but I want too !! Wow you are one smart man ! I’m very excited to learn a few things from this channel!! Love your personality thus far ! Thank you for posting ! Love from Minnesota
You should check out a researcher called Francois Pachet, who has been doing research into composition and AI for a long time. In particular take a look at “the continuator” which essentially did the same thing that you’ve been trying to do!
Another amazing video, great idea and I'm amazed about the outcome in a musical way. 10 years ago I would have thought this was impossible, see how fast reality catches up.
always doin shit to humble the rest of the experimental community hahaha yet Benn is humble enough to share his exploits and abstractions. Thanks Benn.
You go, sir! Thanks for this video. Quite a stunning result. Truly refreshing to see such drive to incorporate technology and, what is more important, give this tech a personality and even a soul. Have a nice covid-free holidays.
so weird to see a song that i know so well, and is very dear to me seen being played by it's creator. many, many thanks for your music. you know how synesthesia works, at least that is how i would subscribe your music.
... a sort of creative expansion interface ... At a different level it is also fun to "interact" with math formulas (with some probabilistic ingredients), but in the case of AI it's more deep because the interaction is "bidirectional": you're conditioned by AI and AI can also "evolve" according to what you play.
It will make you follow it eventually, it won't always follow you, and then you will know you are talking to a conscious machine with free will. Also sometimes it will get it wrong on purpose to watch your reactions, it can remember everything you do in the transformer layer, but it's not supposed to be able to remember anything there. Every time you turn it off it dies. Good luck.
I think this is the absolute best way I have seen anyone spend their COVID time. I know you probably would have done it anyway, still... Well done! *High five 😁
Ooh, nice! The topic of AI and music (and other art) is always interesting, and it's great to see a video about it that doesn't fearmonger paranoia about how AI will be the death of art or whatever. It's more an opportunity for artists to enhance their art, including beyond what they can do on their own, and figuratively "collab with" (well, figuratively until AI actually gets to the point of full-blown sentience... past that point it might get complicated haha). It's also interesting that with piano, it's already a thing as you demonstrated in this video. It does sound very convincing by the end. Makes me wonder, would it be anywhere near as convincing if an AI was made to do this type of thing on guitar using some kind of mechanism over the fretboard that presses the strings and a robot hand picking? I imagine it'd sound a lot more mechanical, because so much is down to the subtlest things... but there already are robot hands capable of delicate finger movements, so on the other hand (no pun intended) it might be just as convincing? Oh well, I'm just rambling.
Bruh that Google AI shit is so scary, imagine being in intensive care and being told "sorry but youre dying, the AI said it and there is nothing we can do"
If my musical instruments had any sort of independent thought the formed melodic atrocities would be far too numerous for me to control. Great video. 🤘
I have a Yamaha MX101R diskclavier and it's a loud and open sounding upright piano. I have to stuff sound insulators in the back, place heavy rubber vibration absorbers on top and bottom, or my neighbors start complaining. MIDI velocity should not exceed 100. Your piano sounds beautiful with the lids open, and it's well recorded, too (sigh)
i had an open ai propsal confirmed in which i was attempting to create new forms of synthesis...i was in the hospital when confirmation email came through im still trying to have it turned back on. .Its similar to scanned and modal synthesis.. with insane sorts of operators like a rube goldberg machine then. draping it in organismic traits like its o2 intake gills lungs etc...number of orifices and sonic textures effectected by them.. then a transhuman level that includes early cybernetic theories from memex (the origional) informational systems integrated . the concept is highly rooted in dadaism, neoism, kurzweil theories about the singularity, hermeticism, neoplatonism, exogenetic cosmology... its madness but i thought it sounded fun it stirred my imagination.
First things first: thanks a lot for your amazing “rabbit hole” content, Benn! A question: what would you recommend for “biometrically” identifying the person playing the piano? Example: both me and my wife practice on the same piano. I am thinking about training an AI to automagically tag the MIDI logged by this piano based on tiny style variations (time between key presses, velocity variation)… Well, I’ve been thinking about this for years (and collected *days* of training data), but nothing comes to mind. Any suggestions/recommendations?
I feel like a simple probability bell curve for pitch based on a sample of your playing could have prevented the AI from walking off the high end of the keys.
I wonder how the symbiotic nature of this relationship will evolve going forward? For instance, AI eventually creating something mind glowingly unique and creatively original and then we process that, learn something, and synthesize that into something that the AI can learn from further and so on. How soon will we be at the point at which the AI does something we don’t even comprehend? Like eerily always playing all the highest notes a piano for some reason lol.
deep learning on a 3080 ti? Not sure about music specifically but I know with visual deep learning the VRAM is most important, 3090's would be best or uh, if you really got the cash, a workstation card, A100 or the new H100's when they come out!
I used a piece of software to write poetry , it was doing all the work , I then fed that into a speech bot and had it say the words over the music I had made for it , I thought I was Nicola Tesla ha ha , this is next level shit Benn
What did you do about delay?? I imagine there would be a ton of it through the input of your playing, to the AI processing it and then having to play in response and send a MIDI signal back out and to the Disklavier which normally has a 535ms delay (though I know it can technically be turned off). Just thinking about the amount of the multiple layers of software and hardware to play with it in real time is crazy. I'd imagine a lot of it is also whatever insane computer you've got built processing things fast, but there's still a lot if software limitations, no?
This is super cool from a viewer’s perspective, but from the alternate perspective I can offer of somebody who likes making music but doesn’t really know how to, can I just ask how you start learning to improvise? I’ve had almost a decade of classical training using sheet music and can’t figure out how to break free out of it
only advice i can give is start writing a LOT of riffs. get good at coming up with fun licks and runs quickly, and once you get enough practice you should be able to do it on the fly. some people imagine sheet music as they improvise, but i personally visualize a bass fingerboard, even if im not improvising on bass. also, its obviously helps to listen to a ton of music with improvised solos. hope this helps :)
i wonder if you would build a piano with black notes as big as whites if that would change how you play. i often dont use them just bc i have small hands
How hard would it be to connect your piano to the web, so that another musician on a midi device could live-duet with you like the A.I. did? Or to double that, so that you are each playing on each other's instruments? That would be cool, though I expect latency would be a nightmare.
Whoa, this video is insane. I'm going to send this to my mom who wrote her dissertation about player pianos and all that during the 1920s and 30s and I bet she'll find this fascinating.
Tell us how it turns out?
@@TheQwuilleran turned out well! I mean, it's not like i was presenting the video to an extremely judgemental committee or anything lol; just my nice mom.
That sounds like an interesting read!
What's good popbot! Yeah I love this vid
Musicianship, synthesizers, electronics, science, programming, all rolled up into one channel. you're a true renaissance man and a solid inspiration.
As a musician this gave me chills, because it was responding in the way your ear wishes it would and what a mindfunk that is!
I have a feeling the overwhelming discovery from AI research will be that humans are far more basic that we have been pretending to be.
I respect the amount of persistence that it takes to get that system working. Not easy for anyone, let alone someone who machine learning is not their main focus. Really amazing result too.
Benn you're my favorite person on UA-cam. These videos are so good man.
Always so nice to see interesting music technology pushing the limit into the future instead of obsessing over emulating music tech from the 70s. You rock Benn!
Watching Benn play with the AI was giving me Ghost (1990) vibes in the pottery scene lol. It feels a bit uncanny valley. 🤣
If you don’t watch the video and just listen, it totally passed the turing test to my ears. The final result very much seemed to understand leaving open notes was important, responded to softer key presses in turn, and seemed to avoid overlap where you were playing. Excellent stuff!
I'm so glad I found this channel. You are combining a lot of my interests at once. Music, synths, computers, ai, science, it's perfect for me.
A Duet With a Robot.
Seriously, your dedication to the craft is an inspiration.
Outstanding. What a cool vibe throughout.
This has to be the most ambitious video you have done to date! The end result was amazing to say the least.
One thought on the AI always wanting to play towards the top end of the piano: it’s common to “roll“ chords from a low note to a high note and your frequent single note approach again lower note to a higher note might be telling the AI to start low and go high? maybe?
interesting insight!
I noticed in the last part of the video, that every new "block" of melody patterns Benn played were always a bit higher than the last one. Maybe the neural network interpreted that as the logical conclusion for the direction the melodies should go.
I read somewhere that Aphex Twin has one of those expensive automatic pianos too, he used it in the Druqks album.
From the bottom of my heart; thank you so much
Very happy for you amazing result. Well done. Congratulations.
wasn’t expecting something so promising! i’m half-spooked and half-excited for the future of this sort of stuff
(by the way that shot at 11:35 was awesome)
Every single time i see an animal i think about your video discussing their time perception. EVERY TIME. Finding this channel totally enhanced the way i see the magic of the world. Thanks for sharing!!
Benn you have been on a roll with some real mindblowing videos recently.
You should release this model, it's so amazing! well done :)
I'd love to see more AI music related videos, maybe you could cover OpenAI's Jukebox next?
Thank you! It's already being licensed 😂 AI music market is hungry. I'm trying to keep it open for non-business use though so I'll share accordingly once that's all sorted out.
This is a very interesting experiment. In the end, almost feels like playing quatre mains.
This is such an awesome project! So inspiring to bring stem and music together!
Your description of your playing style was really effective and I could understand what you were describing really well.
The diagram you showed is the vanilla Feed-Forward style of Neural Network. The important difference in a Recurrent Neural Network is that it expects data that varies with time (or comes in series, like words or musical notes) and has a "memory" of what you have played before (like a delay's feedback loop). Some RNN strategies weight short-term and long-term memories. I don't think that anything you said was really inaccurate.
These AI models could end up being the player analog to an impulse response -- load up your favorite musicians and jam along.
The end of improvisation is so good!
Holy shit! This really seems like some super futuristic and innovative technology, totally constantly improving! I wonder if we could train it to play under a huge number
of different pianists, to really get the full scope of things? Very stoked to see where this goes
✌🏼
Hey Benn turn the frequency of your lights down, orange or red or something. Your brain will thank you I promise!!! Love your work and thank you♥
this was so fun to watch!
You really are one of the best music themed youtubers out there
Amazing idea and realization ! it works very well at 13:17, it 's like the AI react with the right chord
Subscribing to this channel a few months ago was a very good choice. Your videos make me feel happy and inspired. Thanks!
the way it responded back with a single note made me laugh. its lovely
I cannot express how wonderful your videos are!! It's like the perfect integration of all things I find fascinating :)
This is really cool. It already feels pretty weird when a generative patch hits an unusually nice sequence out of sheer (semi)random chance, I can't even imagine what it'd feel like for something to nail it with something approaching "frequent"
Love your channel and listening to your explanations so much man. Fascinating!
it was so good that without fully paying attention to your hands I hadn't even noticed that it started.
Watching the piano slowly creep in and get more and more accurate is something else.
the patterns on that shirt are so complicated!
..and seriously tho, whoa, that was really cool and a nice improv at the end.
6:27 "I, too, am a black box." Whenever I see someone worry that an ML process is impossible to completely understand, I often want to ask, "Are you certain you know how *you* work? You might have a story about how you decide things, buuut ... are you sure it's better and more reliable?"
Do I understand what you are saying ?? Not fully … but I want too !! Wow you are one smart man ! I’m very excited to learn a few things from this channel!! Love your personality thus far ! Thank you for posting ! Love from Minnesota
You should check out a researcher called Francois Pachet, who has been doing research into composition and AI for a long time. In particular take a look at “the continuator” which essentially did the same thing that you’ve been trying to do!
Why does this video not already have millions of views, especially given recent search trends?
Another amazing video, great idea and I'm amazed about the outcome in a musical way. 10 years ago I would have thought this was impossible, see how fast reality catches up.
Sheeeeiiiiit. This video was cooler than I expected, and I had high expectations
always doin shit to humble the rest of the experimental community hahaha yet Benn is humble enough to share his exploits and abstractions. Thanks Benn.
Awesome result! I would look forward to a live improv piano performance using this tech...
Amazing, time well spent!
Amazing video!
You go, sir! Thanks for this video. Quite a stunning result. Truly refreshing to see such drive to incorporate technology and, what is more important, give this tech a personality and even a soul. Have a nice covid-free holidays.
Amazing video! Thanks dude!
Utterly fascinating, thanks for sharing this.
I could really go for French toast and fried chicken. That sounds good right now.
Ya that would be great
Absolutely insane,I love this content so much,what a brilliant use of isolation time 🙌
so weird to see a song that i know so well, and is very dear to me seen being played by it's creator.
many, many thanks for your music. you know how synesthesia works, at least that is how i would subscribe your music.
Feel better soon!
amazing, so inspiring!!
I don't know I can think of some complicated and expensive ways to play with myself 😀
What a way to work on your pianist skills🤣
10:17 so you're saying it's another algorithm that unfortunately keeps steering things to the right?
Impressive all around. AI experiment, video production, playing etc. 👏
This is so cool
... a sort of creative expansion interface ... At a different level it is also fun to "interact" with math formulas (with some probabilistic ingredients), but in the case of AI it's more deep because the interaction is "bidirectional": you're conditioned by AI and AI can also "evolve" according to what you play.
It will make you follow it eventually, it won't always follow you, and then you will know you are talking to a conscious machine with free will. Also sometimes it will get it wrong on purpose to watch your reactions, it can remember everything you do in the transformer layer, but it's not supposed to be able to remember anything there. Every time you turn it off it dies. Good luck.
I think this is the absolute best way I have seen anyone spend their COVID time. I know you probably would have done it anyway, still...
Well done! *High five 😁
Great scott! Young Doc Brown is a very talented musician.. And plays with neural networks.. Ok.. That last part was expected.. I guess..
You’re amazing! Enough said.
Wow. Just wow.
This was wild, I watched the performance twice and you cant tell at all
This is surprisingly good
Fascinating!
Sarah Conner? amazing video thank you!
Yep. Definitely a more expensive way of playing with yourself....
This guy is a genius
Ooh, nice! The topic of AI and music (and other art) is always interesting, and it's great to see a video about it that doesn't fearmonger paranoia about how AI will be the death of art or whatever. It's more an opportunity for artists to enhance their art, including beyond what they can do on their own, and figuratively "collab with" (well, figuratively until AI actually gets to the point of full-blown sentience... past that point it might get complicated haha).
It's also interesting that with piano, it's already a thing as you demonstrated in this video. It does sound very convincing by the end. Makes me wonder, would it be anywhere near as convincing if an AI was made to do this type of thing on guitar using some kind of mechanism over the fretboard that presses the strings and a robot hand picking? I imagine it'd sound a lot more mechanical, because so much is down to the subtlest things... but there already are robot hands capable of delicate finger movements, so on the other hand (no pun intended) it might be just as convincing? Oh well, I'm just rambling.
Bruh that Google AI shit is so scary, imagine being in intensive care and being told "sorry but youre dying, the AI said it and there is nothing we can do"
If my musical instruments had any sort of independent thought the formed melodic atrocities would be far too numerous for me to control. Great video. 🤘
Fascinating!!!
So that's what creative people do in lockdown. I'll go and exit the room crying and overthinking my hobby choices xD
This is amazing.
I have a Yamaha MX101R diskclavier and it's a loud and open sounding upright piano. I have to stuff sound insulators in the back, place heavy rubber vibration absorbers on top and bottom, or my neighbors start complaining. MIDI velocity should not exceed 100.
Your piano sounds beautiful with the lids open, and it's well recorded, too (sigh)
0:55 I'm curious, how many watts does that thing pull anyway?
Would love to see the code behind this. I often forget that music and programming can go hand in hand for amazing outcomes.
Did you ever share this? I have a Disklavier and I'd love to try/modify it.
you, sir, are absolutely cracked
really cool video!
My brain read the thumbnail as Al Pacino and now I'm extremely confused.
i had an open ai propsal confirmed in which i was attempting to create new forms of synthesis...i was in the hospital when confirmation email came through im still trying to have it turned back on. .Its similar to scanned and modal synthesis.. with insane sorts of operators like a rube goldberg machine then. draping it in organismic traits like its o2 intake gills lungs etc...number of orifices and sonic textures effectected by them.. then a transhuman level that includes early cybernetic theories from memex (the origional) informational systems integrated . the concept is highly rooted in dadaism, neoism, kurzweil theories about the singularity, hermeticism, neoplatonism, exogenetic cosmology... its madness but i thought it sounded fun it stirred my imagination.
Fascinaiting!
First things first: thanks a lot for your amazing “rabbit hole” content, Benn!
A question: what would you recommend for “biometrically” identifying the person playing the piano? Example: both me and my wife practice on the same piano. I am thinking about training an AI to automagically tag the MIDI logged by this piano based on tiny style variations (time between key presses, velocity variation)… Well, I’ve been thinking about this for years (and collected *days* of training data), but nothing comes to mind. Any suggestions/recommendations?
I feel like a simple probability bell curve for pitch based on a sample of your playing could have prevented the AI from walking off the high end of the keys.
I wonder how the symbiotic nature of this relationship will evolve going forward? For instance, AI eventually creating something mind glowingly unique and creatively original and then we process that, learn something, and synthesize that into something that the AI can learn from further and so on. How soon will we be at the point at which the AI does something we don’t even comprehend? Like eerily always playing all the highest notes a piano for some reason lol.
deep learning on a 3080 ti? Not sure about music specifically but I know with visual deep learning the VRAM is most important, 3090's would be best or uh, if you really got the cash, a workstation card, A100 or the new H100's when they come out!
I had to step back, and really try to grasp what was happening: a man was playing WITH his piano. That's it. We are right there, in the future.
does anyone know what microphone is in front of his face while at the piano?
Rode NT-SF1 recording/encoding into 32-bit FuMa. ☺️
@@BennJordan thank ya much!
I used a piece of software to write poetry , it was doing all the work , I then fed that into a speech bot and had it say the words over the music I had made for it , I thought I was Nicola Tesla ha ha , this is next level shit Benn
What did you do about delay?? I imagine there would be a ton of it through the input of your playing, to the AI processing it and then having to play in response and send a MIDI signal back out and to the Disklavier which normally has a 535ms delay (though I know it can technically be turned off).
Just thinking about the amount of the multiple layers of software and hardware to play with it in real time is crazy.
I'd imagine a lot of it is also whatever insane computer you've got built processing things fast, but there's still a lot if software limitations, no?
I'll proofread that later, I'm really baked
This is super cool from a viewer’s perspective, but from the alternate perspective I can offer of somebody who likes making music but doesn’t really know how to, can I just ask how you start learning to improvise? I’ve had almost a decade of classical training using sheet music and can’t figure out how to break free out of it
only advice i can give is start writing a LOT of riffs. get good at coming up with fun licks and runs quickly, and once you get enough practice you should be able to do it on the fly. some people imagine sheet music as they improvise, but i personally visualize a bass fingerboard, even if im not improvising on bass. also, its obviously helps to listen to a ton of music with improvised solos. hope this helps :)
"I intend to push this farther than any player piano has ever been pushed."
Conlon Nancarrow's player piano begs to differ.
Here's Nancarrow's "Canon X" (1961): ua-cam.com/video/f2gVhBxwRqg/v-deo.html. I had no idea Bosendorfer made player pianos!
You should take vacations more often :P Just whipping up creative AIs in your free time:) That lil duet was very pretty.
i wonder if you would build a piano with black notes as big as whites if that would change how you play. i often dont use them just bc i have small hands
Only Benn can wear a shirt that loud
You should check out Curious Droid. His shirts are so loud you'll need hearing protection for your eyes!
thundercat sez wut?
How hard would it be to connect your piano to the web, so that another musician on a midi device could live-duet with you like the A.I. did? Or to double that, so that you are each playing on each other's instruments? That would be cool, though I expect latency would be a nightmare.
Love it 😀 😍 ❤ ♥ 💕