I generated a photorealistic music video with AI (GauGAN2)
Вставка
- Опубліковано 1 бер 2022
- I fell in love with AI generated landscapes recently. Although this tool is only meant for images, I decided to see if making a cohesive video was possible. Choosing which parts of the landscape would represent different parts of the beat was a lot of fun! Maintaining continuity from one frame to another is challenging, but after much trial and error, I've found that minimizing the motion of the horizon maximizes cohesion between frames.
To support these projects and include your name in the next video, here is my Patreon page: / doodlechaos
Discord: / discord (I spoil what the next video will be pretty often here)
Twitter: / doodlechaos
Official tiktok: / doodlechaos
Shorts Channel: / @doodlechaosshorts2905
Instagram Shorts: / doodlechaos_shorts
Email: contact@doodlechaos.com
Here is a link to the web version if you'd like to try it yourself!
Here is a quick explanation of my process for this project:
1. Animate segmentation maps in sync with music in After Effects.
2. Export this animation as a series of images.
3. Convert the image sequence into landscape photos via GauGan2.
I was able to automate the conversion process. My first attempt used AHK to move my mouse and click to upload the correct image, convert, download, and name. This was sketchy and actually crashed my chrome browser somehow (I think due to downloading so many images and they were all being cached?). Attempt two was to run it locally. Unfortunately, the NVIDIA Canvas downloadable software doesn't allow png segmentation maps as input (only .can files). My final solution works by sending images directly to the gaugan.org server via python. If you're interested in trying out something like this, I'll make the script I used available for download on patreon.
Resources:
www.nvidia.com/en-us/research...
blogs.nvidia.com/blog/2022/03...
Music Credit:
Track: Lost Sky - Vision [NCS Release]
Music provided by NoCopyrightSounds.
Watch: • Lost Sky - Vision | Du...
Free Download / Stream: ncs.io/VisionID
Shoutout to Winkey for giving me the idea to make this Bob Ross themed. - Розваги
Now that was different! I loved the part at 3:20!
Thanks! That part was the main reason for the epilepsy warning at the start.
ok
Me too! Best part
looked like an actual visualizer
@@Mereaux oh yeah yeah oh yeah
That was super original. I love seeing experimental ideas like this!!!!!!
Thanks Creezy!
🔥👌
Calfreezy
@@DoodleChaosI noticed that the link doesn't work anymore, can you get a new one (although it may be because I'm on mobile)
This is such an interesting medium. At first I thought it was a little weird but cool. But actually this is super cool.
@Anna ♪ Jeez, you gotta warn us before dropping something like that. I also don't see the relevance.
@@JoveRogers97 It's just a bot. Report and move on
@@oceanman207 I already did. My comment is not meant for Anna.
Its called acapella
Nice
WOAH. That visualizer totally caught me off guard. Now that is amazing.
He's using his actual powers to terraform the Earth now
Who Bob Ross?😂
watch out guys, they'll be coming for you as an instrument next
I wouldn't even mind
@@SwarumtheForum in fact, that might be a good idea
Yea
It's really cool how the picture moves with the music
the mountain moving remind me of my cat crawling under a blanket
i've seen this AI but wow i've never seen it put like this :O
this is so cool, especially those sound waves at 3:14
Sup Groovy
@Jax Hin ai is getting smarter 😂
Npc getting more smarter than hoomans
@Jax Hin not npc, its ai
Plot twist: the npc made it
What an interesting way to synchronise music!! Each frame is truly beautiful, I see how you fell in love with it! Nice job!! :D
I'm glad you like it! Thanks :D
yes
Props to the AI for moving mountains, changing the terra and creating/drying ocean for this sick beat
As someone who has played around with GauGAN a LOT, I commend you for your skills. It is truly incredible and hard to get consistent and looking how you want it. Well done.
Awesome! Personally I love the part at 1:32. Every frame looks like a real landscape.
As someone who’s been following AI image generation for the past several years, it’s such a pleasure to be able to see it utilized to this degree. Very innovative and beautiful!
It's crazy that an AI can be used to generate such realistic looking landscapes
This is such a creative idea. You even managed to seamlessly work your own logo in as an integral part of the video.
And the little behind-the-scenes at the end!
Wow! That was trippy and cool - amazing what can be done with today's software...and even more amazing, YOUR brain for coming up with these ideas and then managing to execute them too! This channel is a deep well of happiness for me and I'm so grateful you keep making things to share with us! Thank you!
won't be long before the AI starts coming up with these ideas as well
What a clever idea! It's so cool that you could pause the video almost anywhere, and there would be a beautiful landscape. And the way the landscape flowed and changed was amazing!
The beauty of nature + the beauty of music *chef's kiss
Visualizer part was definitely the best, but generally the whole video just woah'd me!
As someone who's already seen and used this AI, I know what it's capable of, but was not expecting something this good. Great job, can't imagine how long it took to make this masterpiece!
Okay, I heard the expression "rolling hills" before, but can we also use it for mountains and/or islands?
Great video, as always! Very atmospheric. A little harsh on the eyes, at times, but a real treat nonetheless =)
That's absolutely beautiful. Your artistry never ceases to amaze.
This is so satisfying. The video. The music. These images. This is what AI should be used for.
Kinda depressing
@@davidenorbedo5555 no u
Please do more of this, it pleased my synesthete brain SO MUCH!
I can't tell you how mesmerizing that was to watch. The beat of the music to the image's changes from frame to frame is strangely soothing while incredibly interesting.
I can honestly watch this a thousand times in a row and not get bored of it!
I would love to see another video like this from you someday, if you want to do it again.
12/10 Brilliant job!
*Absolutely astonishing 😍😍😍*
DoodleChaos getting into AI art is one of the things I didn't even know was on my wishlist for the world. Absolutely fabulous!
Usually, I avoid supporting people on Patreon when they're already a huge success... better to help those where my contribution makes things happen that wouldn't otherwise, you know?
But if you're going to get into hefty machine learning models, maybe you'll need it, haha.
Okay, that was really cool. I always enjoy your video's of you making some kind of way to sync with music, and this, out of everything you have made so far, has blown my mind. Keep up the amazing work DoodleChaos.
the reasons ive subscribed to ur channel is all the different and creative ways u show the music. This video was a great example. Cant wait for ur next video!
Whaaat this is so interesting (like I've never seen something like this)
Its always a pleasure and an honour to be able to appreciate the videos you create for us to watch.
That said, I would love to see some of this new tech accompanied by some classical music like in the older videos.
Still, great videos :) thank you the hard work you put into them! :D
How!, You make the most interesting and awe-inspiring videos EVER. This was different and in a very postive way. I love how the terraforming of and and sea synchronized with the electronic music to form eye-catching videos. Keep up the synchronization between upbeat music and the interesting visuals. :)
It looks really good, congrats ! It's one of your best vidéo
This is amazing! You're one of my favorite UA-cam channels, I never know what to expect next but it's always so cool.
Another incredibly unique, extremely satisfying video to watch and listen too! Loved how you incorporated your logo 👍
1:58 "KoOraih" wise words
Just like this ratio!
Love the lil river dabs 1:36
And as always, doodle shoved us another very interesting approach to music!
Very nice job man!!
This is super cool. It’s incredible how amazing your video are.
Absolutely fantastic work and great choices
good but nothing will be as beautiful and satifying as your works of absolute are you are the best at this doodle! i also didnt realise the stone henge thing whas the doodle chaos logo!.
Thanks for continuing to make such cool stuff!
Thanks for amazing work! I especially love the part where your logo pops up :) (oh and floral equalizer, too!)
You keep getting better and better, love the change from what you normally do :D
At first, i didnt notice wut it was at 2:48 but then i realized for it was your logo picture. That... was pretty awesome if i do say so myself! Props to you man! :P
So cool!
My favourite bit was with the overview of the valleys. And when it "dipped" diagonally to the side 😄. Loved it
Wooow! What a creative idea! And it turned out amazing! Watching you play with different media is so cool and inspiring!
Great job! :) When you publish it always makes my day! You make the best music content in my opinion! Thank you for all you do for us! :) I appreciate all the time you spend on these video's and I'm sure a ton more people also do! :) Keep up the great work!
Thanks Enderglow, I appreciate that, have a great day!
@@DoodleChaos Thank you so much!!! :)
THIS IS AWESOME!! I wanted to make something like this for so long, to see it out in the world by a youtuber I love is amazing!
wow this is really taking up a notch! hope to see more like this
Wow, this is one of the coolest concepts I've seen in a while! Very mesmerizing to watch!
Bro it's as if the nature is dancing wtf
Like the river part
Amazing!!!
And then those sound waves! Damn dude!
Incredible job! I think it puts my old experiments to shame!
Damn, patrick's rock has grown much older in this video.
Great job man, this is mind blowing
Underrated and amazing. Great job.
Very awesome video. 3:00 it took me a while to notice it was the doodlechaos symbol, but when I did I thought it was cool. Looking froward to more music syncs, and do like the use of NCS songs. I like a lot of NCS songs, but some I don't, so far you have used some I like!
(edit, ish. The moving rock I thought was odd, but it works. The clouds do work better for moving though.)
Just amazing! Make more soon plz!
Your creativity is simply inspiring! Well done!
Wow! This is incredible and underrated!
Wow! This must have taken a while!! And it looks so beautiful!
This is super creative and I love it!
This is so cool. Subbed!
Gorgeous and really cool!
Never thought about using gaugan this way. Wonderful ! :D
This is so original and unique! What a cool idea. I can't stop watching.
Love this song, Good work!
Damn this is so cool! I love everything about this! 💕
Yooo, the visualizer looks so cool!
Now this is epic (and creative too, love it)
Interesting video! That was pretty cool!
Really cool, I liked the part with the doodlechaos logo changing around!
Hey Jake!!!
No matter what the medium, DoodleChaos shares his opinion to people through music and visuals without fail.
Amazing stuff!😁👍
Wow what a video and what a song choice!
That! Was art! You really amaze me and I am in love with your work! Keep blowing peoples mind! You rock!
I was thinking about this same music today and see this... is just magic. Good work♡
This is awesome, never seen anything like it!
I needed that smile today... thanks for the hard work as always, Doodle
💐🐱💐
The best part is that I could actually see a band creating a music video like this
I CRY that this only has 1/8 million views. DoodleChaos you are someone specially gifted. I don't know if ANYONE has done anything like this before.
more of this please xD its cool!
Just amazing!
It’s very amazing , you are so smart and ingenious ,this video is fire 🔥🔥🔥
This is amazing!!
This is AMAZING!
Incredible! :D
Good job adding your logo in there LOL
I always love watching your videos
That was really unique and cool, never seen something like that! :D
Outstanding. Incredible. Breathtaking.
Fun fact: This isn't actually AI generated. The music is DoodleChaos's gigachad vocals and instrumentals, and he simply transformed mountains, rivers, and forests to get the images.
this is awesome, im def gna try this out for myself. u just put me onto something big, thanks bro 🤘🏻
Sounds really good and looks really cool
I thought you did each fame by hand but the animation in after effects to an image sequence is a complete GENIUS idea. I don't know how you do it. This is incredible.
I really like this I hope you do more of this kind of video
I've been watching youtube since 2012.
I have seen a lot of things.
I've learned a lot of things about how different programs and skills work.
Generally even when I don't know entirely or are out right wrong about how something works I can still put forth a guess that I'm happy with.
This is the first video I have seen in my entire life where I have absolutely no idea how it happened or how it worked.
Like... you can paint a pretty realistic cloud to go over water if your patient enough...
But then the grass and water reflecting trees... and the Trees!?
Those trees are so detailed it must have been computer. But where how does the stuff move between it so well... Agh, at the end you revealed a bit with the colors and shapes and I guess the computer fills in those shapes with landscapes... but how are... some of those landscapes have to be from actual nature outside... but there's no way enough real world stuff exists to let the animation be smooth...
I have no idea. This is amazing and I don't have to know how it happened. All hats off to you sir... or ma'am... or whatever you prefer
Very well done
pretty different, why not?
turned out better than i expected lmao
good stuff as always
Very creative
Never seen anything quite like this
This is just so beatiful
Wow so creative, I never would have thought of this!!!!!