New AI Video Goes Hard At Open AI!
Вставка
- Опубліковано 8 чер 2024
- There's a new AI Video model that has just dropped called Vidu, and it is clearly going right after OpenAI's Sora model!
Today I'm taking a deep dive into the Vidu model, not only looking at the research behind it, but also taking it head to head against Sora!
LINKS
Vidu (Note: Google Translated): www-shengshu--ai-com.translat...
Airhead: • air head · Made by shy...
Letters To My Future Self Walkthough: x.com/paultrillo/status/17838...
Chapters:
00:00 - Intro
0:43 - Vidu Demo Reel
2:04 - Vidu Background
2:40 - Vidu Technical Information
4:40 - Longer Vidu Outputs
5:24 - Second Example
6:07 - Third Example
8:53 - Fouth Example
7:27 - Vidu Vs Sora
8:17 - Vidu Vs Sora 2
8:42 - Vidu Vs Sora 3
9:19 - What it is really like working with Sora
10:00 - An AI VFX Breakdown - Наука та технологія
At this point, anything that ACTUALLY releases is immediately a Sora killer.
Even Sora 3.0 will be a killer
4:01 - "More TRADITIONAL video generators ... "
Meaning video generators from 4 months ago. 0.o
Haha. You have a point.
Been out for a year? will smith spaghetti is about a year old
You're always on the ball! I'm forever hanging out for your updates.
Haha, I don't know if I'm ON the ball, as it does move very quickly, but I try to run behind it as quickly as I can!
Great report, very timely and enlightening. Much to look forward to. Thanks Tim.
Much Appreciated!!
Definitely not a "Sora Killer", but still, progress. This is a good thing.
Haha, camp Temu Sora! Agreed though-- more of these models is better! And each one will get better and better. The race is on!
It’s not as impressive as sora but it would definitely speed up OpenAI releasing its public version, exciting times ahead 😎
That's my biggest hope. That a lot of these models start getting released and forces OAI's hand to drop Sora. The big swell of AI Video is happening this summer, I feel.
To be fair, it's obvious that Sora heavily curated their videos.
@@TheoreticallyMediaSORA is bogus. It uses so much compute no regular people could ever afford it.
It's never going to be released until it's somehow been optimized.
I would wager that this VIDU can also do SORA level generation but they have it nerfed so it can be affordable to the public.
@@mehfacetheir videos certainly required WAYY more compute to produce as well.
@@mehfaceOpen AI is saying they won't really SORA because it's dAnGerOuS but the real only reason is it's too expensive and they don't want to admit that it makes it sound like some clunky far off thing. They want it to sound like they already really have it.
It's to expensive, maybe movie studios will be able to afford it eventually but as things stand I doubt SORAs goof stuff will be available to the public.
Super excited to check out the Letters To My Future Self walkthrough, it is always so interesting to see others creative process! 😊 Also, I must agree that after 24hrs of careful examination the panda is absolutely playing Free Bird
I don't see the level of photorealism of Sora, but it is good if free.
Totally. It has some photorealistic elements...and some gens look really good (the detective one I really like)-- but I'm really curious to see more animated looks from Vidu. I think those might be really interesting!
I wonder if there is an immediate future for these products in creating the background computer displays you see constantly in movies and TV. You might think the relative simplicity of those usually vector videos might be a good fit. Thanks for this overview and all the work you do to keep us updated. ❤
Will these (Sora and vidu) have the ability to choose the frame rate the video be in (ex 24p,25p,60p)? Also how about a specific color gamut (log vs rec 709)
I tend to think it'll be at 24p. That seems to be fairly established by most of the video generators at this point. Haven't seen 60p yet, but I'll cheat that by running it through Topaz. As far as color gamuts, nothing that I've seen yet. Although, with Sora integrating into Premiere, that has to be a must at some point.
Tim is just Mit backwards... 😂 I gotta watch "notes to my future self" I dig the workflow. Can't wait to see the Adobe Sora video! Have a great week Tim Dawg!
How do you keep up? Thanks for keeping us in the loop Tim!
Haha, this one just happened to land on a weekend where I had a free Sunday afternoon, so I took a little time to record/edit it! But yeah-- there are 10000 other things still on the list!
Thanks for introducing Paul Trebo and Shy kid’s tutorial ❤❤
Paul Trello*
100%! Got to meet Paul very briefly during that Vegas trip from a few weeks back-- sadly, I didn't actually know his work at the time. Maybe better for him, since discovering it afterwards, I would have been all "Man, you're sooo cool! How'd you do this? How's you do that?!"
haha, again-- maybe for the best for him!
another great video by tim
Thank you SO MUCH! Always really appreciate see you here!
I am from 2050... and Sora was just a prank
I knew it! hows the price of bitcoin?
@@bigbadallybaby around million...!
@@Crazy_Truth
What does $1 million buy in 2050?
@@richardhall5489"What does $1 million dollars buy in 2050?"
1 semester of college.
@@richardhall5489 a tank of gas
Looking for someone who can write an AI script of a bear playing a guitar for me. Any leads?
Ask and Receive:
Title: "Strumming Paws"
Plot:
In the quirky town of Bearton, music is everything, but no one has ever heard a bear play guitar-until now. Meet Benny, a charismatic brown bear with an unusual talent and an old, dusty guitar he discovered in the forest.
Benny’s musical journey begins when he strums his first chord in the woods, inadvertently live-streaming himself using a lost smartphone. Overnight, Benny becomes an internet sensation, capturing the hearts of music lovers and the curiosity of animal behaviorists alike.
As his popularity soars, Benny is invited to compete in Bearton’s legendary "Rock and Roar Music Festival," a competition traditionally reserved for humans. With the help of his eclectic group of friends-a hyper-intelligent squirrel, a fashionista rabbit, and a grumpy old porcupine who doubles as a drum guru-Benny must learn to refine his raw talent.
However, not everyone is thrilled about a bear joining the festival. The reigning champion, a haughty rock star named Lance Lightning, sees Benny as a threat to his title and plans to sabotage his performance. Amidst this, Benny must navigate the complexities of fame, his instinctual habits, and a mysterious figure from the forest who claims to know the origins of his beloved guitar.
As the festival approaches, Benny and his band of misfits not only have to face Lance’s tricks but also rally the town to see beyond their prejudices and embrace the music in everyone, human or bear.
(To be fair, I've actually seen worse films!!)
@@TheoreticallyMedia I would legit make this. The bear needs to be played by Jack Black obviously.
Thanks for the video! I think prompt to end product (video) will be a while from now (so, couple of hours). It's like in voiceover. Creating _good_ AI voiceovers is actually pretty time-intensive. The workflow is slightly different, but _more_ editing and audio engineering are required in AI VO. Freelancers are charging higher for AI VO than human VO. But the larger companies wanting VO for brand anthem videos and the sort are paying more for _human_ VO's. We'll see how that compares to AI video, in a couple of minutes of course. Thanks again Tim. 👍
Though are we sure that the videos on TVs from SORA weren't After Effect post edited??
To be honest, we can't be sure. I think it likely was generated by Sora, the question I have is: how many regenerations did it take, and how long did it take to render?
Cool, with competition. I hope others like eg Runway etc also steps up soon.
I have the feeling that Runway is for SURE working on a really big update
Always so AI-nformative and Fun-iiiiiii 😁
you = on top of ur game 👏🏻👏🏻
Thank you so much!! Rushing ahead to keep up with that rolling ball!
Every few weeks, a new video ai. What happened to LTX Studios that you were reporting on maybe 4-6 weeks ago? Any updates from them?
they're letting folks into the beta now. I think mainly sourcing from the Discord, so join that for sure! They just reached out, so I might be doing another video on them soon-- which, hopefully means they're ramping up!
@TheoreticallyMedia Good, thanks. 👍
I watched one of those comparison videos on the best ai video generators that someone did a few days ago, but ai is so much in constant flux, that the best image, video and editing generators will continually be changing. The best ones a month from now won't be a month later, and the one leading the pack a month later will be a different one still.
But the long term winner of that competition will be consumers. As long as there's healthy competition instead of a monopoly, the market will always benefit.
How did you register on Vidu? it asks for a chinese phone to receive a code
Subject: Idea to Enhance Discovery on Your Channel!
Hey Tim,
Huge fan of your channel here! I really dig how you sprinkle in all those cool references across your videos-it adds such a great layer to what you're explaining. But sometimes, I find myself wanting to know more about them.
How about including a list of links for these references in your video descriptions? Even if it's just a quick LLM-generated list from the video transcript, it would be super helpful. I bet it could be a neat resource for others too, boosting discovery and engagement.
Thanks for considering it! Keep up the fantastic work!
Yeah, that is 100% on the to-do list. I've been mulling a newsletter and (super basic) website. The site would serve as a spot to archive all the stuff I've yammered about over the last 100 or so videos.
You'd be surprised (maybe not?) if you knew how often I was trying to track down one of my OWN videos to find something I knew I talked about at some point!
Who chose the music for the sizzle reel?
The one in this video? That's a track I recorded awhile back but never figured out what to do with, so I figured I'd pop it in here. YT sometimes gets finicky w/ striking based off audio, so I'll often strip out the background music and replace it w/ my own.
It’s just me or I’m bumping against join list every time while this particular one doesn’t work for me… I can’t send the form.
the form seems to be borked right now. Another comment mentioned putting a 1 on the phone number part? Basically Country code? Seemed to work for him.
@@TheoreticallyMedia my advice to any supernatural 3.0 AI engine ? get your 2.0 website running
Hmmm. Sora's quality and sharpness seems far better still, no?
I'll say, the captures I got from Vidu were 720 compressed videos. So, the actual quality is likely better.
And how exactly do we know if this is better quality than Sora if Sora hasn't been released ? (putting aside the few demo videos released, that doesn't tell much of the end product as it could be filled with lots of hype)
A lot of these tech demos make for a nice clickbaity thumbnail, which looks great alongside a clickbaity title, but I don't really believe the hype on a lot of these all-in-one, one-click AI video tools. These new AI startups are designed to hook you in with a viral tech demo showcasing some promising tech, and then they paywall it behind credits, most of them making you use the credits before you're even able to see the entire output. So in the end, while it's fun to play around with this tech, it's like film-making by playing a slots machine -- you put the money into the machine, and you pull the lever and hope the output is what you want. If not, then you keep spending credits until you get the output you want. It's the most perverse way to corrupt the creative process. Disgusting.
If you check out the back half of the video, where I talk about Shy Kids (Air Head) and Paul Trello, I think you'll see what I'm more excited about with this tech. Particularly the segment on Paul's work.
Well said 👏
This is something to keep an eye on. SORA and LTX and the two whcih I've been honed on, as I await their release.
8:54 I just noticed in the sora walk, her legs switches sides
Hey Tim! Thanks again for another awesome video!!!
I just like the competitive development. The more companies working on these models the better for the industry overall. Like any emerging market this is going to happen and the best ones will tend to refine and last the longest
Sick! Also Open-Sora 1.1 just released up to 16s too.
Thanks for adding that "whats it like to use sora" part...its clear its NOWHERE NEAR as consistent as OpenAi made it sound originally.
oh, I missed that! That's excellent to hear!
The SUBMIT button works, I realized you have to put a 1 in front of your area code if you're in the US.
oh, was it as silly as that?! Ok, back to submitting!
@@TheoreticallyMedia Still not working. I tried 1, 01, +1, +01 and using hyphens. Couldn't find a working combination.
5:25 country bear jamboree
Whoah whoah whoa, wait. Shy Kids short wasn't full AI? They had practical shots in there? No one every mentioned that. That's a pretty big difference.
There's a making of video on UA-cam. Pretty sure they still left in some distorted humans in the video despite amending it heavily.
Yup. TON of cleanup work. And from what I've read, they were looking at a ratio around 100:1 in terms of generation. So....yeah. Might have been easier just to shoot the thing.
Yup. I think in the interview, he even refers to it as a Slot Machine. There was a follow up interview where one of the Shy Kids said something to the effect of: I invite Studio Execs to try it out, just so they can see it isn't what they think it is.
What is the best upscaler/AI upscaler in your experience? I feel like they all want us to pay for them now,
How are these platforms not available. I wrote a superhero book years ago that's at an online retailer. I would love to see it made into a movie. I could use this software combined with D-id and Eleven Labs to bring my book to life and show it on YT. Just to see it live. I want this technology to be like Adobe, MS, or Google where I have Studio in one place rather than the dozen I use now.
Onion News:
Holywood technicians develop Sora killer using screen plays, cameras, actors and lighting.
Haha, I miss the onion!
I’m not sure if I’ll do AI video, but that might be a future path to take.
ouch on the auto focus in studio B, but love the content regardless... thanks for he reporting.
Hah! Saw that in the edit too-- the Canon seems intent on focusing on the mic! Gonna try to get that sorted out for the next video!
Sigh just when I had Studio A tuned in...
Hmm, the Google Translate doesn't seem to work with the Vidu site anymore. Does anyone know how to view the site in English?
the direct link to google translate might not be working right. Try the main URL and running it through Google Translate.
@@TheoreticallyMedia Yeah, that doesn't work right now either. That's alright though, I'll give it a day or two and then try again. Thanks for letting us know about this one anyways. I never would have thought to go looking for advanced AI tools on Chinese sites like that, lol 😉
Jo...I watched some vids of you in the Last week or so. You seem to be up to date of all the AI stuff and also show like new not yet released stuff like in this video. So anyways I decided to subscribe now, if I come back regularly anyway
This exactly what exponential progress looks like. Sora is showcased in late February and shows the world something we thought was years away even at current progress rates and OpenAI looks to be miles ahead of everyone else, and then two months later the industry is already almost catching up. Next up we will have opensource models doing the same in around 2-6 months as it seems they are consistently around half a year behind but not quite a full year.
And then in 1-2 years micromisation efforts combined with the current rate of progress we'll eventually have locally run, open source, realistic video generating models, and then all bets truly are off. Everything is about to change over the next few years, and you know what; I'm here for the rollercoaster ride.
It's an easy comparisson. Vidu vs Sora is exaclty like Leonardo vs Midjourney. I design books, so because I pay for Midjourney, I always give Leonardo a go first and sometimes, I get images that I "could" use for a cover, but then I go back to Midjourney and it's just not the same. I could be wrong, but I get the feel that a platform likr Sora, will shake the foundations of an entire industry, just like Midjourney did for illustrators, designers etc.
At 8:57 the right leg of the Japanese girl change place with her left leg. But I bet next year things will be scarily close to perfection.
One of my favorite ones was the Detective shot, but there is some clear morphing at the start of that shot as well. I think that's just going to be something we have to deal with for a bit-- but, invites some creative solutions to get around!
I want full length feature film blockbusters.
Thx for the summary. Is it me or you often say Uvid instead of VidU?🤣
Haha, WAYYYYY too much, or too little coffee in this video! haha, I messed it up a bunch, caught it in the edit and I was thinking about correcting it-- but man, I fumbled so many times I figured: just let it go and get roasted in the comments! haha
im guessing it's "VidYou" as in Video+You. Vidu. i dunno. sounds better.
Ah, Vid-You is a good read. Haha, I need phonic company names! I can't wait for someone to let me know how badly I butchered the company and uni names!
Ok this definitely places a fire under OpenAI, they can't hold unto Sora for too long or it might just become irrelevant!
We will have even more content creators as "AI" implementations allow people to make cool stuff without having to personally talk and stuff. AI can narrate your video, make visuals and sounds to go alongside with it.
thanks for this, I did not know about ViDU. But I am certainly tired of hearing about SORA, it is like the donkey and the carrot, I keep finding those youtube clips about it saying how good it is but it is not accessible.... grrr. Making a big hype and then letting people wait, I am already fed up. So good on VIDU!
Naahh, no, 3 seconds in and you know it is wallmart sora!
Why do so many AI image generators have resolution limits? They are really limiting the use cases here!
The beginning of the holodeck?
I think we'll see the very beginnings of it later this year. 3D is going to come in HARD.
5:08
Yeah, I'm a huge fan of MJ v4 also....... I really should play around with it more, it'sgotten way too little love from me since 5.1. Too much time in v6 changes a man! 😉
One of the things I've been thinking about is generating in v4 (for that dirty, grimy, surreal look) and then popping over to one of the creative upscalers to see if it'll "modernize" it.
That might be an interesting way of keeping that v4 punk rock look, but update it a bit.
Not to say there's anything wrong w/ v4 as is. I'm just happy that MJ lets us play with those old models!
@@TheoreticallyMedia
Oh......I like that concept!
Please experiment and share with us......... this could be amazing! 🙏
Also love to use MJ v4 quite often, because of the "cursed style", then I inject the pictures made as a Sref in V6
Looking forward to evolution🧬
Well underway!
Made by a private company? One step back... Comming from China?! Two steps back lmao. Tim is my best ref for AI world. Great job man!
I think it's probably pronounced, "Vid-U" (as in, Video You)
The music Ai tool Udio is similar in that it's a play on "audio," but is pronounced, "You -dio"
honestly i see this more of a way to make sora release sooner. many of the vids are "Sora esque"
All of this is very Uncanny Valley. Really curious, how would it handle impressionistic stuff, lighting effects, various styles of visuals... As of now, the video game cut-scene vibe seems to dominate!
Also, the "Tokio Walk" lady in Vidu's version is just visibly limping. Great for a zombie film 🧟♀️
EXCELLENT use case! I can't wait to generate a Zombie Epic!
You raise a good point there. It's funny how good cut scenes have gotten over the last few years, but there is always that uncanny valley aspect to them. I think that'll be status quo for quite some time.
Ladies and Gentlemen we find our selves in the era of Teasing, Ltx studio, Sora, Vidu.. and soon, cause now they will tease this new tech a lot , not perfected even 60%. and the lauching will take a loong time
My question about both Vidu and Sora: So lets say you have the prompt " man dressed in 1920s attire walking down the street etc" and then you want to do another shot with that same man turning the corner and walking down another street, and then walking inside a shop the next shot. I am guessing these tools will not be able to do that, they won't have that type of continuity at this point?
Consistent Characters are not a thing in Sora yet. I presume the same in Vidu? But that remains to be seen. It's why in that AirHead video they (creatively) went with a Balloon head. It's "easy" to have a character like that. I say easy, because even then they ran into a lot of trouble.
My general solution here is to prompt for very generic characters: "Man in a Black Suit with Dark hair" and then faceswap after the fact. It's a cheat, but it does tend to work. (The suit might change here and there, but usually a black suit is basic enough that no one notices)
With these tools coming out every now and then, sora will have a serious competition on their plate to deal with,
It is a good one for us creators.
It looks good, but as others have said - not quite on SORA level. I do like how there are now a few companies pushing the AI video creations. Pika, Runway and Haiper come to mind and i have a feeling Haiper will have an update in the near future (I do like Haiper), in general I think video length and quality will be a main aim with these companies. Give it another year and I thing things like AI Video generation and Music generation will be very, very good compared to what we currently have.
The more competition the better for all us AI users will benefit :D
I think it's pretty good 👍🏻
WOW~
They are holding us back by taking their time with Sora. Something will come in and fill that gap if it takes much longer. Everyone thinks their top dog till theyre obsolete..
Competition! Good.
couldn't agree more!
Vidu's demonstration is... questionable as a SORA killer and it isn't even local. Some recent projects worth covering, however, are Mira / Open-Sora / Open-Sora-Plan.
I want prompt generated 360 VR videos 🤓
Yes!!! And soon! I think this year. We’ve got skyboxes already, so it isn’t far off!
It video looked pretty rough, it's going to need a few more years to provide benefit for commercial purposes. The biggest issue with videos can be the distorted faces and limbs
A bit of very visible morphing going on here. Looks a little like a slight step up from Pika & Gen 2 that can make longer consistent content.
Not as good as Sora, but better than Runway. Anyway it is very good and good that it is not from the USA. It is bad if one country dominates AI, there should be a contest to the US global power to make it more balanced. Regards from Argentina!! Love your content!
That's a great take. And agreed on its placement in terms of ranking. I think Runway will be making a big jump very soon!
Thanks Tim! No Chinese AI for me though. Keep up the great content 👌
Totally understood! I'll keep an eye on it and report, so at least you can keep up from afar!
Sora's hype killer
great news but i see its only available for China
Did they update with that? There was no indication of that earlier.
It might be useful but framing it as a Sora killer is odd when the samples are noticably worse.
I don't think it was the best call to release footage directly calling out Sora. I get that it is kind of funny, and I think meant as a fun jab, but I also think it invited comparison, which...yeah.
Vidu looks like a great model (and 16 seconds!)-- it can be its own thing!
Looks interesting but can't use any of the tech.
What would be really funny if your background was actually an AI gen swap out.
Ngl, have thought about it! Maybe in an upcoming video!
How long until someone uses a game controller to create video in real time? Will that be an ai game or a real time movie? Is it possible ai could "know" enough about the real world it could simulate video predictively! My head hurts. 😊
Maybe someday tech will develop a VR suit with AI control. Some people may never come back to reality. Yikes
I know one developer working on a real time game. It is still really morphy and weird, but I’m keeping an eye on his progress!
Folks are working on it!
Ready Player One, indeed!
I would pronounce Vidu as "Vid - yoo" cause it's closer to Video
Sora from Wish
Certainly not a Sora Killer.
the google translator link starts in German 😮
Vidu looks very AI tbh
It’s true. That’ll be the case for a bit. But I’m ok with it too. I think AI video can/should have its own look. Doesn’t mean you can’t tell a cool story with it!
It's better than anything except Sora but since we don't have Sora... awesome! :)
Haha, totally! That was kind of my joke with the Thumbnail. Sora Killer...which, y'know hasn't even been released yet!
Looking forward to the time in the future, where we make fun of these videos. It's amazing from today's standpoint but will look age very badly I reckon
Every once in awhile, I like to go back and watch my videos on like, Gen-1 or Gen-2 when it first launched. We've got a LONG way in a short amount of time.
(I also don't watch for very long, since I hate watching myself!)
Why doesn't "Vidu" used the English language instead of Chinese for broader reach? Even their sign-up page is in Chinese!!!
i gotta start learning chinese and arabic at this point.
I feel that. Luckily, we'll all be wearing universal translators soon! As it turns out, that Humane Pin might be worth something after all!
@@TheoreticallyMedia Well, seeing that a rather simple text-to-video tool (don´t remember which one, was mostly for educational/ commercial use and sticking stock-videos together) could translate my german, rhymed texts into english, also very well rhymed texts, within seconds and in perfect context... I don´t think you´re exaggerating when you say "soon"...
I feel like Sora's sizzle reel was cherry picked a TON and showcased what they are wanting to achieve with their release, which is why it might be taking a while. While Vidu released theirs directly out of the box. This is all speculation on my part but OpenAI has a lot to live up to since they've set the bar high for themselves.
Oh for SURE Sora was cherry picked. In that Shy Kids BTS-- I mean, the amount of work they put into the post process? And, from what I'd heard, the ratio was somewhere around 100:1 for shots.
In some ways, you get the feeling it probably would have been much easier to just shoot the damn thing!
@@TheoreticallyMedia Imagine having the ability to do a script breakdown, (like a real shoot), and you have the art departments breakdown with set design, props, etc. From there you can load that up images, references or a style and it builds the scene around the breakdown and then when you prompt for that location it refers to the script notes and the breakdown and adheres to the details. Would be like having a real art department on set. Same goes for the vanities, wardrobe, etc - even a script supervisor could be working in the background. But that is going beyond far at this stage of the game. But being able to do a full script breakdown by department would seem the only way to truly get a decent short film or feature to the next level.
nah this is like on second place
and 4 months later ... OpenAI might have made improvements..
"Temu Sora" 🤣🤣🤣🤣
If they have access to the same hardware/chips, they have access to the same software. Just reverse engineer it all. Glad to see these vids and hopefully SORA will realize they can't charge too much.
Sora actually has already been reverse engineered. Like, there's an understanding of how that model was made. Vidu apparently pre-dates Sora in terms of development, and does use a different method. Kind of fascinating really-- I think we'll be seeing a lot of new models popping up soon. And for sure, someone is already using the Sora approach for something in development.
One of the first applications for video generators might be porn movies. The initial flaws won‘t matter much there.
Wrong! He's playing Stairway.
Haha I was going to say Wonderwall!
Still the best AI UA-camr. I find all the other ones so irritating to watch, so glad you have a personality at least. Gosh I sound like such a bitch. Sorry I’m nice really. Can’t wait for the sora premier video. So curious to find out when that’s coming out
More Pika than sora
So there are some pretty impressive clips in there. But I'm just going to say this, like most things from China, it just feels like something cheap. I would say these are first Gen and give then a little bit of credit, but I would guess you'd call this second gen, and is definitely not as good as Sora. But who am I to complain as I don't even have access to Sora. Like most people.
Same. Still waiting on Sora. If we get this before it, I'll be using it for sure!
It’s not as good as Sora, but their reveal did use better prompts.