Wild AI Video Workflow with Viggle, Kaiber, Leonardo, Midjourney, Gen-2, and MORE!
Вставка
- Опубліковано 8 чер 2024
- Here's an interesting new AI Video Generator workflow utilizing Viggle.AI, Kaiber.AI, Leonardo, Gen-2 and Midjourney-- all inspired by Editor Colin Goudie's rip reel for Rouge One. By utilizing pre-existing films as references and styling them in a variety of AI Tools, we take a "kitbash" approach, but the results are promising, as you'll see in the examples. I'll walk you through everything I learned to hopefully save you time if you want to try this out yourself.
LINKS
Viggle Tutorial: • New Free AI Motion & A...
Viggle.AI: viggle.ai/
Kaiber.AI: kaiber.ai/?via=tim
Leonardo.AI: app.leonardo.ai/?via=tim
Runway Gen-2: runwayml.com/
AudioGen: www.audiogen.co/
Uncanny Harry: / uncanny_harry
Jeff_synthesized: / jeffsynthesized
Martin Haerlin: / martin_haerlin
00:00 Intro
00:28 Inspiration from Rogue One's story reel process
01:47 Recreating the "Are you not entertained?" scene
02:01 Overview of the A.I. generated scene
03:30 Using Viggle.AIfor the character animation
4:00 Issues with Viggle Output
4:58 Utilizing Leonado.AI for Character References
5:36 Solving the Shake with Kaiber.AI
6:54 Creating Backgrounds
7:35 Using Gen-2 for Background Motion
8:33 Tips on Comping Green Screen AI
9:53 Audio Generation and Free Tools
10:55 The Soundtrack
11:10 Concluding Thoughts - Наука та технологія
“Ai Kitbashing with Tim” needs to be a legit series on the channel! I’ve tried pretty much all of your process kitbashes and have advanced the quality of my output each time.
That's awesome to hear! And yeah...hmmm, a kitbashing show DOES sound fun!
This was really amazing! Thanks -- I used your kitbashing to do some interesting things in my last video!
Running the viggle output through Kaiber to clean it up is a nice trick. Thanks for the info!
As I delve into storytelling and video experimentation, VideoGPT seamlessly integrates into my process, enhancing my content with its subtle touch of sophistication.
0:44 confession.... Rogue One is the FIRST Star Wars film I ever watched in it's entirety (after decades on earth). This came after I watched The Mandalorian and got intrigued. Was so impressed after Rogue One, I ended up watching every non animated Star Wars film ever made!
How cool! That’s an experience I’m quite jealous of. My experience of Star Wars sadly has been one of diminishing returns. It was my entire childhood though, so idk who I’d even be without it haha
Rogue one is my favorite Star Wars movie. They did so many things right.
Shouldn´t you start with Episode Zero? Isn´t that the first movie historically speaking?
@@dsoprano13incredible, it is the only non-animated among star wars movie/series i tried three/four times to watch and never arrived past the first half an hour. My preferred is the empire strikes back. And i'm there since the beginning in 1977
@@MartinZanichellihhhaahhhaha
I think this is neat and all, if content creation is all you care about. To me though, it feels like letting a chess engine play for you. The fun to me of animating is watching it all come together.
Great walk-through. Pretty intriguing! Looks like AI video generating is progressing at a steady clip. Thanks for sharing.
Great video. Love getting inspiration for new types of workflows like this. (Even when it is snowing.)
Thanks so much! Still a lot of fine tuning with this method, but I wanted to get it out there to see what others might do with it.
I love this free-for-all era of kitbashing! So many opportunities for everyone!
3:15 OMG you are EVUL!!! I NEVER knew that film even existed, now you burn my brain with that visual !! LOL
My eyes, my eyes!!!
Haha, I'm so sorry!!
His hand motion with 3 fingers!!!! And so blurry! Tim, you are so bright! Open AI should be all over you for you to test their SORA and be an ambassador! I am sure you could create a good film with SORA
I keep tweeting at Sam Altman, but he ignores me! haha, I even offered to write him a check for 7 Trillion Dollars!
@@TheoreticallyMedia you know what, man! I truly want to see you on Mars building the colony. I am going to be working there by your side!
Brilliant workflow walkthrough! And Rogue One thrills if no other reason than Phoebe Waller-Bridge! (Many other reasons imho!)
Thank you so much! There’s a lot to still fine tune here, but I thought it’d be great to get the ideas out there for others to play with and add to! I love this wild era of AI video!
can't wait to try this out, thank you!
It's a LOT of fun! Can't wait to see what you generate!
just not long ago i started workflow similar to this, but the extra last step you did there is amazing!!!!!! thank you so much💚🙏
Excellent to hear! Still thinking of a few other details to add in as well! We're all on the front lines here, and it's really exciting to see all the crazy directions we're taking things in!
absolutely!! and i love how you getting out of the box with the use of the tools, and not only that, you learn the new ones so fast and then teach others that's amazing, big respect🤓🤝 @@TheoreticallyMedia
Nice job, Tim! Your film reminds me of those classic cars in Cuba that are built out of anything they can get their hands on for cheap or free. I think it's great that you're trying so many tools. Eventually I imagine that some tools will simply shine and be the preferred ones for this kind of work. Anyway, bravo!
Haha. You ever hear that Johnny Cash song “One Piece at a Time”- where he’s working at the Cadillac factory and steals one part every day, but it takes him 20 years to assemble the car!
non stop gold. thanks once again
Thank you!!
Small correction on Tarkin in Rogue One... It was not a deep fake, it was a full CG face replacement made by ILM. To my best knowledge the first time deepfake was used in Star Wars, was for young Luke Skywalker in "The Mandelorian" and later in an episode of "Book of Boba Fett".
Very valid point! And ha, yeah-- that Luke didn't work out too well the first time around. He did look much better in Book of Boba though! (didn't they hire the guy that did the better version?)
@@TheoreticallyMedia The reports to what degree UA-camr and master of Deepfakes Shamook was involved vary between "He was adiving" to "He did it himself". But he definitelly helped ILM to work out a better version in Book of Boba Fett.
Nice tutorial mate
Well done!
Thank you! Next reference film: Master and Commander! Let’s go through all the Russell Crowe films! Haha
Zardoz!!! Classic 70s scifi! I own it on BluRay, it's glorious 😂
Well played Tim. Do you think you'll be able to keep the character consistent through more short films? I am motivated to try the kitbash workflow.
More or less. Kaiber can be a bit finicky, but the 3.0 update is super early. Hoping they get that locked down soon!
Having a real hard time finding the Terminator Dance Off video, really want to see it 😆
I got you! twitter.com/Martin_Haerlin/status/1774785858572349692
@@TheoreticallyMedia Thank you!
Enjoyed the video, since trying out Viggle I've been contemplating a similar workflow. Using Facefusion and then Topaz with Iris over the top solved the face, but ive yet to find a better method to improve the body.
Comfyui just released one.
I havent downloaded it but it looks amazing.
I am actually going to have to watch this twice. Maybe a third time. I think I will try something with this method but I don't have all the software mentioned or alternatives. Do you know if I can do the lighting corrections and stuff in Wondershare?
Filmora? They should have color correction tools! It's been awhile since I've used it, but I'm almost certain it is there!
@@TheoreticallyMedia i will find out soon enough! Haha I won't complain if not though. I like the idea of this workflow for concept films or concept trailers.
Dat's a real purty guitar behind you.
Is there a reason why you didn't use the Speech to Speech in ElevenLabs using your own voice? I use that all of the time to get what I need and your tone in the recording does a great job in effecting the output voice.
I did try with my own voice, but it was still coming out garbled. My have been because I was yelling? It was the first 11labs major fail I’ve had in awhile. Surprised me!
Excellent!!
"I've lived with it for a very long time" 😆
Haha! Now others can share in my trauma!
Kitbashing is our friend. : )
Kitbashing is THE Best! Uncanny Harry had a pretty good quip about it: In 6 months our Kitbashing will be incorporated into the platforms-- so for now, we're all leading the charge and showing the way!
@@TheoreticallyMedia :)
What about shadows and lighting in Viggle;?
Good question! I think that it is established in your reference photo. I’ll test it out with some heavy film noir shadows on a character to see what happens.
Are there any other ways to reduce the stuttering/shaking output from Viggle without having to change the style/look of it too much?
Not as of yet-- but I'm hoping there will be some model improvements soon!
I love where your head is at and I'm super impressed by the work that went into this video, but man do I hope LTX solves all these workarounds
6:06 *Duane* Johnson? My man is really testing the limits of AI.
Ever see his Epic Cheat Day meals? I love looking at those. Sadly, I just want to eat them without doing the 6 months of workouts! Haha, I’ll just take the food please!
it may be better to use runwayml to get a cleaner greenscreen
I was thinking the same. Might have to revisit this once I have a few other tools to bash it with!
2:58
😆 🤣 😂
That's awesome!
Martin killed it there! Literal spit take from me!
@TheoreticallyMedia
😆 🤣 😂 Deservedly so....... just insanely funny to watch!
How would this process handle a fight scene?
I’ll try it out. Hypothetically, you might end up with the same character? Since Viggle only allows one reference. But what happens if we put in a reference with 2 people?
Might break, but I’ll give it a shot.
@@TheoreticallyMedia would it be possible to break the fight into two clips for each character?
@@TheoreticallyMediaall of these Style Transform suites seem to process at 8fps and then interpolate. So, quick movements are... difficult.
Have you subscribed every model! 😲
Haha, basically yes. I beta test for a few of them as well. But, yeah-- somehow I've turned all of this into a job, so I've got to stay on top of them all!
@@TheoreticallyMedia habibi! Too much money!! 😂
love
Thank you!!
Not getting access of LTX Studio! 😢
The big roll out should be this week and next! At least that's what I've heard!
@@TheoreticallyMedia can you please help me to get access!!
Tarkin wasn't deepfaked, it was traditional CG and based on a cast of Peter Cushing's face.
UA-camrs have made deepfaked improvements though.
Very good clarification-- I called it a "deep fake" as that is sort of what the media reported it as, but you are totally right, it is more akin to a CGI character that was mocapped.
That said, it is funny how random YT'ers (and not so random ones like Corridor Crew) have done the same with better results!
@@TheoreticallyMedia Didn't mean to be all "Ackshually..." about it. 😂
Had deepfakes been more of a thing back then, it would have looked a hell of a lot better! As it stands, he's right out of the uncanny valley, with only a few shots looking somewhat passable.
👋
Heya Louis! Thoughts on this AI video workflow?
@TheoreticallyMedia
It's not my cup of tea. I'm definitely willing to wait a couple / few years for SORA quality text to video to come to the little people like me......especially when we get that quality in Open Source!!! 👍
@TheoreticallyMedia
I didn't jump on board text to image until I saw sufficiently high quality being regularly generated by some of the tools.
Time is precious and I feel it's a waste of MY time to play around with tools that are too cumbersome or not sufficiently good quality producing tools.
That's just me.
I was blown away by SORA...... and that's the standard I seek in order to get heavily involved (time and $$ wise) with an AI video tool.
@TheoreticallyMedia
I definitely find there's a good selection of sufficiently good quality text to image tools available now and that's why I've spent a lot of time and $ on software tools subscriptions and hardware tools (like an HP Workstation with a lot of GPU under the hood).
Once I get access to RELIABLE and cost effective SORA quality tools, I'm all in on AI video...... trust me on that.
100000x easier to go out and shoot this scene
Haha. Not gonna lie, I often think that as well in the middle of one of these. But I think when you look at it as animation, or like, heavy VFX/Green screen films, there’s some time saving (and cost) here. Especially if you’re a “garage filmmaker”
*dwayne
Man, I had to see that 3000 times while editing, was kicking myself every time!
That said, I’ll say that the overall point is that the name seems to work more as a placeholder to get Kaiber to create a somewhat consistent character.
Not making excuses, apparently I cannot smell what he is cooking!
2nd
No Chicken Dinner, but I'll buy you a burger!
@@TheoreticallyMedia I will gladly pay You next Tuesday for a hamburger today. : )
This video made using viggle ai
ua-cam.com/users/shortsO5IeRNPpP7Y?feature=share
Where do you thing AI is going to be in 2040, 2050
Man, I can’t even fathom where it’ll be next month!
Third
No chicken dinner! As the third, you do have a vegetarian option though!
Furst
You have WON the Chicken Dinner! Sending out via Fed Ex tomorrow! You may get it....next month?
@@TheoreticallyMedia in UK :(... can u send thru dropbox? Or we transfer?
Damn brother, We still have a longggg way to go, 2:05 quality is awful lol Sora... Where are youuuu, we're drowning over here
Haha. Always remember, just where we are today. And I always caution: Sora (if/when we get it) is probably going to be pretty janky.
@@TheoreticallyMedia I can see that. I'm also pretty sure consistent characters in Sora is going to be in a real distant future. Like years lol Even if the technology allows it, Hollywood will not. If you think the strike was ugly from the script writers, imagine how bad the actors strike will be!
Tim I’m shocked you never talked to me for any content. I make way more content than these guys, I also helped Pika train data and was the head moderator and artist for them. We talked a bunch..
Ahhh- Microphonist! I didn't realize you were here! I'll drop a follow on your channel! Are you on X as well?
@@TheoreticallyMedia oh, wow! I didn't realize you were a Head Mod, Mikey. You were so seldom present, I'm surprised. And, a Data Scientist too? Wow. I clicked a lotta thumbs. Can I put that on my resume?
3 fingers and a thumb😅
Horrible to use other people's work without their permission just to replace them entirely
This is what literally everybody does to create anything, even if it’s just subconsciously and it never makes it out of their head before it becomes what they’re trying to create.
It’s also what people do when they use temp music on movie edits before the composer comes in to replace the music, very often replacing the music with something very similar.
I do understand what you mean, but this is basically just a higher Fidelity version of what people have been doing for decades, if not the entirety of human history
@@ChristopherCopeland The thing is. AI is not inspired like humans are, because AI does not like anything. It sees trends in the data its fed and outputs an approximate results of what something a human made might look like.
****immediately searches for Zardoz and...oh no....
Haha. I’m so sorry. I should have left that sleeping dog be!