New AI Video Workflow With Epic Results
Вставка
- Опубліковано 24 сер 2024
- Today we're taking a look at a new AI Video workflow that I've been playing with, and it is one that I believe solves a lot of problems that AI Video currently faces. Namely, that you can't get specific actions out of AI Actors, nor can you direct specific camera angles or moves.
This method utilizes technology and platforms that are readily available, namely we will be looking at Runway ML's Gen-1 and Gen-2, plus Kaiber.AI and Midjourney. But we'll also take a look at a pretty cool app called Skyglass, that auto rotoscopes you into otherworldly locations. Utilizing that, along with Gen-1 is an AI recipe for some amazing footage.
All that, plus a free website that will greatly assist you in generating longer clips in both Gen-1 and Pika.Art!
LINKS:
RunwayML: runwayml.com/
EbSynth: ebsynth.com/
Kaiber.AI: kaiber.ai/
Midjourney: www.midjourney.com/
Replicate: replicate.com/
/ martin_haerlin
/ uncanny_harry
/ benjamindekr
Final Frame: finalframe.net/
AFFILATE LINKS:
Camera: amzn.to/3yXMDY2
Microphone: amzn.to/3K1jIZm
Audio Interface: amzn.to/3lDX9kf
Mac Keyboard: amzn.to/3XOeJR0
Coffee: amzn.to/3JZuBeq
Follow Me on Twitter: / theomediaai
-------------------------------------------------
Thanks for watching Theoretically Media! I cover a large range of topics here on the Creative AI space, Technology, Tutorials, and Reviews! Please enjoy your time here, and subscribe!
so glad to be a part of this space as it grows, ty for the workflow guide...I've just barely dipped my toes into the video realm, but the pace of advancement is staggering
It’s insane. The weirdest part about doing UA-cam videos is how quickly the information changes! I posted one about Gen-2 not having an image reference, and the next day it got added!
@@TheoreticallyMedia that's crazy lol, i appreciate you keeping up with everything but i don't envy the pace you have to maintain lolol
@@stevesmith1810 Haha, I've got a vacation next week, I'm looking quite forward to it!
Of course, I have the feeling that Midjouney v6, and a whole host of other amazing AI things are going to happen, and I won't be around to cover it...
But..I will be on a beach sipping fruity cocktails, so...I'm ok with it!
@@TheoreticallyMedia have a great and well-deserved vacation... I can't wait for v6 but I'm thinking you've got a couple weeks still...enjoy those cocktails!
What we need is an AI tween program where you give it first frame/last frame and then let it do it's thing.
Agreed! I think we’ll be seeing some time increases soon. It seems like that 4 second wall is something they all face, but I’m sure we’re one breakthrough away from 15 second videos.
Although, weirdly enough, I keep thinking: how long do we need? 3 to 4 seconds is clearly not long enough, but 15 also seems excessive. It’s going to be interesting to figure out what the sweet spot is.
Have people submit 5 photos for the AI to work with and make sense of the prompt
You can do this (sorta) with EB Synth. In fact it basically requires keyframe images.
@@ajkoch813 Yeah, I have EbSynth as a shoutout in the video. I remember they were first on the scene with this tech. I keep meaning to go back and check them out, but I keep getting buried under all the new tools!
@@ajkoch813 Here's my problem with EB Synth:
1) if you take away the download button, the entire Web site is one single UA-cam video. Super lazy and bad marketing.
2) No Linux support.
Both of these things make me think this is malware that collects personal data as opposed to a legitimate AI offering.
Hi Tim, just found your channel while looking for guides to Kaiber and I'm happy to say that I'm really liking your videos, it's great to find creators that genuinelly inspire and motivate 😃
Thank you!! That's 100% the goal!! We have all this amazing stuff at our fingertips: Go make stuff!!
Runway Gen-1 video to video result through Kaiber has become my go-to!
I love Kaiber’s video to video, I’ve often said it is the real secret weapon of that platform!
Hi Tim, I've been following you for sometime now and i just want to say that your videos are very helpful! Thank you for putting in a lot of effort to make them, keep rocking👊👊
Thank you so much!! I don’t know if you saw the announcement earlier, but I’m out on holiday this week. But your comment has me really jazzed to get back!
Great job on the comprehensive guide to your strategy, I feel we're on the verge of something quite special with AI video. And I think the more companies that offer something a bit different than others actually help all the companies build more robust solutions. Also, I will forever be using "Duct Tape Version" when necessary.
Totally agree! I think each of them having strong points really benefits those of us who enjoy kitbashing as well!
I’ll say, I was really impressed with Runway’s Green Screen/roto. It’s gotten a lot easier in AE, but you still have to keep an eye on it. The Runway app was so stupidly easy and accurate.
Kids today, they’ll never understand the struggle of the Roto Pen tool.
@@TheoreticallyMedia I totally agree with Runways Green Screen/roto. And you couldn't be more right about the struggles of roto. I would have to mentally prepare before any roto work. It's the tooth ache of motion design.
Also, I haven't dived deep into this but what if you were to change the frame rate to something like 60FPS on the footage beforehand. Do you think it would produce less artifacts or morphing?
Gotta watch your videos on a separate device so I can Google all the awesome you share. Nice video Tim!
Haha, dual screen for life!!! I know there’s been a push for larger monitors that are basically the size of two monitors, but I’m not sure my brain would accept that. I need 2 monitors to keep things organized!
Amazing video and explanation!!!❤
Excellent to hear!! Thank you!!!
It made me giggle. "I am available for voice over. I m really cheap. Cause im really bad at it" excellent !
Haha, I’m basically 3 hours of editing for 2 minutes of voice over work! I am NOT cost effective!
@@TheoreticallyMedia Modesty speaks :) By the way great video about adobe max :)
Outstanding workflow. Definitely filled in some workflow gaps for me, (final frame. replicate, etc.). Super appreciated!
Thank you!! I've actually had a few other little nuggets that didn't make the cut, and I'm hoping to so a follow up soon! I love AI Video prompting, but I'm really starting to become more convinced that video reference is the only way we're going to get past the current "static character looking at camera" vibe we have going on.
To be honest, it'll probably be a combination of the two.
“Hipster orientation day” 😂😂😂
Day one: using the correct beard balm!
The pellet with the poison's in the flagon with the dragon; the vessel with the pestle has the brew that is true.
Haven’t thought of that in ages, so I went back and watched the scene after your comment. It’s so brilliant!
All time Classic! 😂
Thank you 👏 now I’m going to make my New video clip 🎻🎶
Excellent to hear! Post it over on the channel discord when you’re done! Would love to see it!
God I love your videos and enthusiasm. You’re so right about this being a world changing moment in history. Imagine the world were going to be living in in 5, 10, 20 years. It’s going to be unrecognisable.
I mean, at this point, a WEEK from now is going to be insane! It's so crazy!
Hi Tim, I enjoy your video, thanks.
Thank you!!
Perfect Thank You 🔥
Excellent to hear!
I think in my humble opinion ai art will be a new movement in history . Mark my words that one day there will be art works created with the imagination of human new wave artists using ai tools to create the unimaginable, I believe those beautiful creations of art will take place in different museums around the world our future . Lots of people are having a bad aspect about ai saying that it will take over etc etc in other words they say it's should not be taken as serious art , but let me tell you that I really think we are in the verge of a new revolutionary new wave of art movement which I have faith in . Today impressionisim , surrealism , cubism paintings hang in the most prestigious museums around the world which in its time(early years) most people criticize it and not accepted until they understood it , a big example is impressionisim , during it's early years was highly criticized by the public, the public felt that the Impressionist paintings were vulgar and shapeless compositions that did not follow rules from the old masters , Claude Monet , Édouard Manet , Van Gogh and other painters from the time were rejected many times for their works not being considered a art form , which their paintings did not follow painting academic rules. All I wanna say is that today there is a lot of aspiring new wave artists that are using ai tools to create works , in my opinion it takes a a really brave artist to go poking around in the mind and who knows what might be unleashed from it's imagination , my name is Ricardo and iam a aspiring ai artist and currently iam working on projects using ai tools . If you are reading this and also have faith in this movement well all I wanna say to you is don't stop and believe in your self and create the unimaginable, and no I don't think ai is gonna take over or replace painters all I am saying is that we should give a opportunity to ai so we can be able to understand it better. I wanna see what are the possibilities of creating art works with our endless imagination and the algorithm of an ai machine that is enable to learn , analyze data , and make decisions to create . I think we have a lot to learn from ai and I believe ai wants to learn from us also , thank you , have great day.
Totally agree with you. I actually push toward AI becoming its own medium at some point. It doesn’t need to be limited or compared to traditional painting or filmmaking. It can go on its own path, and I’m sure it will. Ironically (to some) it will take a human mind to push it into that direction. I don’t know what it will look like, but I do think it is inevitable. At least I hope so!
This finalframe app sounds very cool, I am doing it manually screen cap + PS, let's see how much time I can gain. :)
Haha, same! I mean, it isn't "hard" by any means-- but it really does take you out of the creative flow state. Total time saver!
New Runway update for longer videos! That should help 😊
I'm excited about this!!
@@TheoreticallyMedia I know, that’s why I came and said something 😎
I feel that it's really now I'm the gold rush era. There are tons of opportunities, the rails and safeguards are not there, you need tons of ingenuity: like using Thin Plate Spline Motion Model which has a 256x256, limitation. So you tear it apart into pngs upscale them in a batch and then create a new video. But it's very interesting and you can just do it alone from your house.
This is my favorite part-- the Wild West aspect. Kit bashing, and breaking toys to see what they do. At some point, the walled gardens will be erected, and while it'll still be fun, it won't be as exploratory as it is right now. It really all feels like the dawn of the internet all over again. Thankfully without the annoying sound of a 56k modem!
@@TheoreticallyMedia it is very exciting! At the same can be overwhelming. Sadly I can't buy now a PC, so I'm stuck with a laptop, and man, it's overwhelming to find a reasonably priced cloud deployment of Stable diffusion that has all the features I need from A111. Or just the sheer number of projects. ControlNET? I didn't know it existed until last week!
Hey.. 😎 got a question: At 7:12 using Replicate…. what exactly over there did you use? Interpolation with which model?
Ah, sorry, Rife: replicate.com/pollinations/rife-video-interpolation
@@TheoreticallyMediaI think Topaz uses Rife as well?
Even I am using AI for my content , i made my video totally on mobile phone ♥️😂 that's so awesome
It's insane isn't it? I'm headed out on a vacation, and therefore leaving my computer/laptop behind. But there are plenty of mobile apps that I've been meaning to try out. Curious to see what I can produce on "just a phone" (we know that phrase is now a dinosaur!)
Hi
Is there a link to ( Wav2Lip: Accurately Lip-syncing video to speech), please?
I’m curious how I can make a video clip, it’s actually a clip I shot of surfing, and turn it into something creative with AI or is this possible? Just doing it for my friend that’s surfing in the clip so not trying for perfection either. Okay thanks either way for your time.
Check out Kaiber. I think they have a free trial. The video to video aspect will be perfect for surfing shots.
Thank you 👍
Tim again.. great content :). Also was doing the Last frame as image prompt Input in Gen2 previously. But usually there is quite a quality drop between iterations. Would you then suggest upscaling the image file before adding as i2v prompt? Wasnt super satisfied with the results.
Give it a shot, for sure. I'm still 50/50 on the technique to be honest. There is always going to be some level of image degradation, but I've found that some is worse than others. I really have not figured out what the special sauce is that makes one better than the other.
Another idea I've been thinking about it taking the final frame and bringing into PS to crank the sharpness up-- to a visually horrible level, but seeing if Gen-2 will the soften it to a "regular" amount.
@@TheoreticallyMedia oh thats an interesting approach. Please share if u give that a try. But to be fair.... runway would probably release a "use last frame as reference" beforr we figure it out hehe... fingers crossed.
Tutorial my brother!
More coming soon!!
I have made the first ai generated tv quality show
Drop a link!
depends on the size of the dragon small dragon maybe? Large dragon hell no
I don’t know, I’m pretty sure small dragons have those puppy teeth! Those things are sharp!
@@TheoreticallyMedia oh i had not thought of that lol
@@swardinc I mean, you’re right, large dragon does have that whole “breathe fire and eat you in one bite” as well…
High
I see what you did there! Haha. High back!
👋
Hi Louis! Always a pleasure to see the hand wave!!
Dutch, not Danish 😅
Oh no!! Sorry!! Big difference too!
Well, I guess I have to use that image again in a future video to correct it! Haha, which is fine. Maybe next time I can show some highlight videos of a match as well!