AI Video Keeps Getting Better! Motion Brush is Here!
Вставка
- Опубліковано 13 лип 2024
- Today we're taking a deep dive into Motion Brush, the amazing new feature in Runway's Gen-2. Not only will this be an overview of the feature, but I also have some pretty handy tips and tricks for you for maximum control of your video output.
We'll also take a look at Runway ML's new Style feature-- which may have been a bit overlooked with the hype of Motion Brush, but it looks to be a pretty great addition as well!
LINKS:
Video On Face Fusion: • The Best FaceSwapper y...
RunwayML: runwayml.com/
Final Frame: Finalframe.net
If you'd like, please join the Patreon Here: patreon.com/TheoreticallyMedia
Join the Discord: / discord
Follow Me on Twitter: / theomediaai
00:00 🎬 Motion Brush is a new feature in Runway's Gen-2 AI that allows for creative video generation.
00:42 🖌️ Motion Brush can apply motion effects like left, right, up, down, and proximity to images or objects in videos.
01:49 📷 Motion Brush can be combined with camera controls to create dynamic video effects.
02:58 🎭 Props in the initial image are important for successful motion brushing, as AI may not add missing elements.
05:02 🎞️ FaceSwapping in Motion Brush
05:25 ⏭️ Currently, Motion Brush cannot be applied to extended video segments directly, but you can use workarounds like speeding up or reversing clips.
07:27 🎨 Runway Styles offers various artistic presets to apply styles to AI-generated videos, enhancing creative possibilities - Наука та технологія
Thank you for this. So happy to watch something not about drama and motion brush is really cool.
Haha, the OpenAI drama? I’ll admit, I’m as fascinated as everyone else, but I don’t feel strong urge to make a video about it. Well, that, and by the time I’m done shooting and editing it, all the information would be completely outdated anyhow!
All of my attempts have been dog dirt so far. This is from a man who just used GEN2 to produce a whole lyric video (with some heavy editing), so I am not dissing them. Just being honest. The lyric video is for Warrior Soul and comes out next week for a song called "Blown". Just getting my plug in there!!!!!!!!!
Excellent! Ping me when it drops and I'll check it out!
It's a weird tool, I'll say...I think what it needs is an intensity slider. Like, the "harder" I brush something, the more that should indicate not to touch anything else.
Still, a great first few steps into what will be a major feature.
Another runway money grab smh
minds well use leiapix
@@TheoreticallyMedia You said to ping you when the lyric video drops. This is very much a budget video just to promote the vinyl release, but still very happy with it. ua-cam.com/video/Q038Mqk3dSo/v-deo.htmlsi=S9UiF77boYFxlf5i
Thanks for the breakdown, Tim!
And really appreciate the tool info. I totally forgot about Final Frame. That's going to be super helpful for a project I'm working on right now. Cheers!
1000%! And haha, yeah-- I kinda forgot about Final Frame as well-- until I was playing with this video and trying to extend a motion brush. It looks like some work has been done on the project as well! Considering you can now join two clips together (maybe more?) that's a HUGE timesaver!
@@TheoreticallyMedia wouldnt it be simpler to take a snapshot in premiere? Or is Final Frame easier and faster? Never heard of it though
:)
Great video as always thanks Tim. Actually went back to watch your face replace video too. Good stuff.
The Pinokio one? That little app is a gold mine!
It’s funny, I was thinking about Glitch when I saw the new Style feature in Gen-2. Man, combine that with Motion Brush?
Insane.
Great video, like your funny comments. 🙂 The examples you showed are very consistent and a great sneak peak of what is coming. 👍 When the workflow doing this stuff for longer scenes is enhanced and simplified, then I guess we will see a lot of artistry pop up in this field.
Totally correct on that longer workflow, but I do think there are a lot of opportunities right now as well. Technically, Gen2 (and most of the other video generators) put out about 4 second clips.
We got to 12 second on our Western Woman.
And for better or worse, if you watch most movies and TV shows- you probably won’t see many 12 second long shots. They’ll all be shorter for sure.
it was earth all along 🤣
The brush tool is AMAZING! Thanks for breaking it down👏
No more excuses for visual storytelling
100% correct! It’s all here! There is some kitbashing you need to do to get that “perfect” shot, but it is ALL doable!
8:04
That's me!! I'm super underwhelmed by video / motion AI at this time but I'm very hopeful that in a couple few years I'll be very interested in the technology because it will actually become usable for the tie and effort by then I believe.
Haha, well on the plus side, you keep pretty well informed about it! I think for some that “moment” is going to seem like it comes out of nowhere, but if you’re keeping an eye on it, you’ll spot it awhile in advance.
Personally, I think it’s next year.
@TheoreticallyMedia
Definitely I want to keep informing myself and that's why I'm subscribed to you !
@TheoreticallyMedia
Interestingly, the YT algorithm just tossed a fascinating video my way from an AI content creator I wasn't familiar with who put together a fascinating fantasy use case for AI video in the form of a BBC-like nature film. The YT channel name is:
Mickmumpitz
The AI video quality is still basically crappy but his latest video does demonstrate a fascinating use case as I mentioned and it shows the potential for how a combination of non integrated tools can be used for a comprehensive affect.
Great overview, i didn’t know about face fusion for motion graphics. I gotta remember the old last frame to extension technique. Cheers Tim
Final Frame will give you MUCH better results than Extend. It’s great for Pika and pretty much any other video generator too!
Face Fusion can be hit or miss- but it really is the best we’ve got right now. And, if you check out that linked video: it’s free and runs locally!
Thanks for the news made my day
I think we’ve all been waiting for this one!
Yet another excellent video. 🙌. Watching this and now wishing I'd touched on 'styles' in a recent video as well. Too many tools and never enough time. Everything has come along way over the past year.
Thanks Tim! gonna check it out right now.
Excellent! It’s pretty cool, I can see a lot of potentially interesting workflows coming out of it!
This so great.
Thanks, Tim!
1000%!!! Thanks for popping by with the comment as well!
Great video, some nice examples well explained. Thanks 👍
Thank you very much! Ha, if you like the examples, you might want to see the video I did on Krea.ai. Scandalous Godzilla seems to be a favorite: AI Art Just Changed Forever
ua-cam.com/video/4-2dSRjErE4/v-deo.html
@@TheoreticallyMedia already watched it :-)
@@Lazarusart ahhh, beat me to the punch! Thankfully, it was a punch and not a kick from Godzilla’s massively toned calves!
Nice walk though, and happy to see your wardrobe is expanding. ;)
Haha, winter is coming and it's getting cold!! One day I may even expand to the strange and frightening world of colors...
hmmm, that might be a step too far.
Thanks for this sir
1000%!!
I tried Motion Brush yesterday and I have to say it's not a game changer, but at least it's the right way to go. It only becomes interesting when I can not only move an object on the horizontal/vertical axis, but it also moves itself on his own object axis. In other words, it changes its own view.
I think we’re on a road that has needed to be paved since the introduction of AI video. Might be just the first steps, but it’s getting there.
Ironically, one thing that Gen-2 (and all the others) is really terrible with? Walking!
So…we’re on the path, but our characters step like newborn giraffes!
It works quite well:)
I’m also pretty impressed with the style thing as well. I need some more time to dig into it, but I think there’s something really cool in there!
Tim, you rock :)
Thank you so much!! Sorry it took a minute to get back to you- the holiday weekend really zoned me out!! (Too much food!!)
Nice breakdown!
This is also possible with pretty much the same level of quality and way more customisation in the open source project AnimateDiff.
Indeed! I do want to cover some AnDiff stuff soon (along with controlnet), but I also like to keep an eye toward the creators who might not have beefy GPUs or the hardware/knowledge to install and run it.
That’s one of the best things about this era: options for everyone to create!
@@TheoreticallyMedia That's totally fair. It's great there's so many options out there. For those looking to dip there toes into AI video content, Runway and Pika are a good start. For anyone looking to create a bigger project, open source is getting more an more viable/cost effective. Just today we have Stable Diffusion Video, which looks to compete with Runway 😁
It's not just there crowd here 😬
It's cool stuff though, good video =]
Haha, well-- at least you're keeping up with it, because it'll get "there" fairly quickly! At least you'll be one of the first to know!
So, MJ is still trying to give us a little more control...in its _image_ generator, right?!
Great video Tim. Thank you. 👍 Also, great job with Clint and...whoever the belle with the fractured elbow is!
Haha. Yeah those joints are a bit of mess- not enough pasteurized Milk in those days!
@@TheoreticallyMedia lol
I have a sneaky suspicion that Adobe will come in and wipe the table clean with these sort of tools. Of course theres an insane amount of work that needs to be done before it has any production quality but it's def moving in a solid direction. Also, with your face animation, it looks like you found out the coffee was spiked at 6am! A good exercise and fun project would have someone start a scene and then the next person picks up the last frame and then creates the next part of the story. Like the whisper game.
Oh, I’m sure Adobe is either reverse engineering right now, or looking to throw some money to just straight up buy this technology.
It’s funny, I know in one of my earlier videos, I posted up video inpainting months before Adobe announced it. I really want to go back and check, but I’m 90% sure it is the same tech, just acquired by Adobe.
In my mind, those of us who are actively following all these developments are basically seeing a preview of what will be industry wide in about 12 months.
As we play with this stuff, Adobe is looking over our shoulders. Haha, we’re STILL beta testing for them!
great video, can you tell me the app that allows me to text what i want an animated character to say, and it makes the character mouth move and talk, and it has preset voices
Suppose I were to have a couple of art figurines setup in front of a green screen stage, and I was recording stop motion animation fight scenes...
Is there any A.I. software (to your knowledge) that could replace the models in the video with other desired models?
Is there not going to be a video to video option for 3D Artist? There are certain poses of a character and/or object I can create myself, and if we could just keyframe certain poses and let the AI interpolate the in-betweens, that would be good enough for indie filmmakers
Not currently- but I suspect you’ll see something like that mid next year.
actually the Power of coffee and Gen-2 transformed you into Ryan Reynolds (who strangely is a bakground character in Yellowstone)
Is he really? Yellowstone almost got me to watch it when Harrison Ford showed up- but I also think he’s in a spinoff show? At this point, I feel so behind and it would take me forever to catch up!
nah, was messing with you. And i have to catch up this show too, but really, the AI made you look like Ryan Reynolds for 2 seconds @@TheoreticallyMedia (you're ready to star in Green Lantern 2 ;) )
I use Kaiber for some of my videos (psychology based). Once my yearly subscription is over I would like to try gen 2, if they reduce the prices
I do love Kaiber as well! There’s some new stuff in Motion and Transform they just announced, so I’ll be covering them pretty soon again!
But yeah: LOvE Kaiber. It has such a cool and unique look!
Isn’t the film clip bicycling across the moon from the movie ET the Extraterrestrial and was directed by Steven Spielberg? I don’t think Jasper Abram’s had anything to do with it
Haha, 100%! It was a little joke (mostly based on the JJ film Super 8), and I wanted to see if people were paying attention!
You win the chicken dinner!
"it was earth the whole time" hahah
Best twist ending ever!
If we can get ai videos to be consistant, and even just as short as 20 seconds, then a full blown movie could definely be made with that tech, combining things like ai voices and lip syncing tools, a script can even be made completely by ai. We're pretty much there already, i garentee by summer next year it will be a viable option for making movies
I would say 2025 is when we will see some wild shit with AI videos
Agreed. I am thinking next year. The first (and biggest) step I feel will be consistent characters/locations/wardrobe.
Once those things are locked, between lipsync and voice gen, we’re pretty much there.
I think the early versions will be a lot of characters standing around taking, at least until walking is figured out- but I’m already seeing a lot of hybrid 3d character rig generators popping up- so that might be solved quickly as well.
Super close to “if you can dream it, you can make it”
@TheoreticallyMedia games will probably follow afterwards. you’ll come home sit down and put in “big space battle movie constant action for 3 hours” and it creates it for you. crazy
omg, every video you make is like a giant leap in some capability or another 0.0
Oh thank you so much!! Just trying stay on top of this crazy roller coaster of technology!!
I don't know if I missed some sort of irony or humor here but... by JJ Abrams movie, you mean Spielberg, right...? Haha.
haha-- just wanted to see if I would catch anyone sleeping!!
Please make a video on emu video and emu edit
I’ll take a look at it!
Hi I have a question. You are an older musician I am a bit older but I like looking into AI ...HERE IS THE QUESTION...I have collected over 100,000 midi tracks on files songs from all different styles etc plus you can find even more on midi sites. Could you train an AI on all the midi information then ask it to make a song with a style and or prompt and it would give you a midi track with drums bass that you could then use?....that would be cool imagine you hum a tune you ask for a brass section etc but in midi format then you could add your own samples.
I’ve got some pretty good music stuff coming up, including rolling back over to Aiva (which I covered here: ua-cam.com/video/A5Bjungcuwc/v-deo.htmlsi=HyEQpi7juLVCe0th). I believe they have updated so they’ll take midi input now?
It’s funny though, most of the current music AI stuff tends to be more based around the idea of vocalizing and then converting that into instrument/Midi.
I presume there isn’t a focus on midi, since the models that have been trained on it probably have terabytes of them. But I’ll keep checking around for you!
What is that video in 08:23 from?
Funny enough, that was something I generated in Moonvalley. You can check it out here: ua-cam.com/video/E1G1jtUunjs/v-deo.html -- it does those really abstract things very well!
Thank you!@@TheoreticallyMedia
How to i get mation brush?
Just log onto Gen-2. Should be at the bottom of the input screen
It seems like whenever I try to extend four seconds, it changes the colors/ camera movements so drastically that it becomes unusable.
It’ll do that. To be honest, I think going the final frame route yields more consistent results…at least for now!
👋
Hi Louis!! Hope you had a great weekend! (And a long one this weekend- hopefully AI takes a break, I could use it!!)
How to reverse clip in free software? Really.
Yup! Davinci Resolve should do it. There are two versions: Paid and Free-- but honestly, the Free version will do just about anything you need it to. A ton of UA-camrs just use the free version. It is great software!
I'm trying to help you posting some of your videos on FB, I hope it's okay!? Most of the swedish people seems to iognore everything in AI....
Soooooooo glad thx
It's such a great tool! My mind is racing at the possibilities! It's funny, I didn't even get a change to play around with Generating a video, and then using the Rotoscope in Runway, then back to Gen-2 for background motion brushing-- but that is totally do-able as well!
You
Me?
At the beginning of your video, you reference 'E.T. the Extra-Terrestrial,' a movie directed by the talented Steven Spielberg, not J.J. Abrams. Personally, I view Abrams' directorial style, characterized by excessive lens flare, lack of satisfying story endings, and rapid-paced cuts, as lacking in true talent. This is especially evident when comparing his works with a masterpiece like 'E.T.' In my view, Abrams' approach, as seen in his treatments of Star Trek and Star Wars, falls significantly short of Spielberg's cinematic artistry.
Haha, it was a joke- wanted to see if folks were sleeping or not! You were not sleeping!!
It was a bit of a dig at Super8 as well. I totally agree with your assessment of JJ. Weirdly, I do enjoy his movies while watching them, but it’s only when/if I turn a critical eye toward them an hour after viewing, do you see all the glaring holes.
Well, maybe not the last Star Wars film he did. That was a dumpster fire.
It’s funny, I have a buddy that is an obsessive Jaws fan, and every year on July 3rd we screen the movie. And every year, I pick up on some new detail or directorial choice that blows me away. I mean, not even the big stuff like the One’r or the beach shot. At this point, it’s a lot of the quieter moments- which, as you point out: don’t exist anymore.
Just make a cup of coffee and film yourself
Haha. Yeah, I realize that this is a high tech solution to a problem that doesn’t exist in this case-
But, I was thinking about it the other night: What might the next Ken Burns documentary look like? Or any other documentary that sources historical photos?
We already do a lot of parallax effects on those types of docs, is this any different?