There is a problem with rating videos for how they look instead of what the prompt requested. If we are giving five stars to something that did not follow instructions but still looks good, it reinforces behavior that will make effective prompting more difficult as the model evolves. Most noticeable in this video was camera replacement
One suggestion: Please don't edit your voiceover together with the videos, because when you remove parts of you talking, you also cut the videos, so it looks like the videos jump in a weird way, and it's not clear if it's a quality issue with the generated video or the editing.
There are so many artefacts and inconsistencies in these shots that I really don't think they're useable yet, outside of thrown-together low effort stuff.
Even at this point though, I see a reasonable use for this. One of the benefits of prompted AI for art is that it may not create something you want, and the artifacts seem to always be there, even for still images, however, if you have a general vision of a scene, you can enter a prompt, run it 10 times and see if something comes close to what you want, and then use it as inspiration. It might be a hot take to suggest we use AI art for inspiration for self-made art, but how many times have artists seen things in the real world that triggered a spark of creativity, inspired by the art of others? All the time! Asking for something close to what you are feeling can more quickly lead you to your final destination. This could be true for people who paint, make movies or even write music.
Exactly. Video quality is max 1 star out of five, especially if it doesn't match the prompt, movement is janky or illogical things happen or appear. It's just "cool looking stuff on video with usability or memorability of zero"..
Give it 5 years and everyperson will beable to adapt everybook into their own series. People won't even need to write their own screenplays. Just pop Harry Potter into a LLM, generate a Screenplay, pop that into a Video generator and you got yourself a movie/series...
It's the same thing, essentially. You're a neural network, it's a neural network. It's piecing together visual info it has previously seen, just like we do when we dream.
These ratings might approach making sense if you replaced every "out of five" with "out of ten". Maybe "out of twenty" if we allow for duration and storytelling.
Most of the prompts lack creativity. I want to see how the AI performs with something way out there that it hasn't had a lot of training on. Idk, surreal, like "A glass baboon riding a spaghetti unicycle in zero gravity"
I think your prompt is going to be rendered quite easily, what about this: "A hyperdimensional tardigrade wearing a monocle made of crystallized time, juggling quantum singularities while tap-dancing on the synapses of a dreaming god, as fractalized echoes of forgotten memories rain upwards in a symphony of bioluminescent nostalgia."
@@zdenekburian1366 In theory yes but the problem with verbose prompts is that the renders are limited to 4 seconds. The engine is going to try to parse the key points and ignore the rest. What you end up getting is a mess that looks nothing as intended.
@@RasielSuarez ahhhhaha i asked claude 3.5 to answer with a joke, making fun of your intersting but offensive (for AIs) suggestion and it gave me that comment to post ahhhaha
@@zdenekburian1366 Using AI to converse in UA-cam comments is how the internet dies. I'm guilty of it too, to a degree, I've pasted answers to specific questions that needed factual answers (in reddit), but eventually a big percentage of comments is going to be people doing this. Remember this time of mostly human interaction. It will be missed.
It ain't all over. It's scaled the slopes of the Uncanny Valley, danced like it's selling nails and stomped on the tera of hyper realism. Still, it's only the begining (and you've already gone and lost your mind). We can't even predict the outer limits of this stuff, but I don't think we have the imagination to envision the horrizon. Buy the ticket, take the ride. Just be sure you're strapped in Scouts!
Those videos were INCREDIBLE! I am a smart human, but not artistic at all, and I could NEVER make a video as good as any of those you showed. Just the little details like the sand, the clouds in the sky, the movement of the water down the waterfall - incredible.
Your ratings are terrible even by your own conditions you set. Are you seriously not noticing all of the AI video glitches and inconsistencies in the 5s you rated?
yea, same thought here. This model is doomed with that kind of rating XD Something is fckd up to the point it's basically unusable - yea I like it, 4 out of 5. Another one with no glitches - yea, it's ok but there is not much movement, 3 out 5. Logic died long ago. Probably the same moment one's getting convinced about "meta" in YT titles. Shocking.
as he stated the rating is relative ... you can't expect high quality video out of generative models at the moment. What you should look for is how consistent it can keep track of concepts between frames (and for how long it can keep track of them). Like that monster shot where the monster turns his head you can clearly see that an extra eye got generated. Almost as if the model lost track of the "head" and "position" concept. Anyway the progress is really good considering a little over a year ago models were generating morphing dream collage.
@@user-ty9ho4ct4k o ya I agree that the numbers scale is pointless. Its his arbitrary scale, when he explained it at stat I interpreted it as a bucket between 2 choice "good" and disappointing. With the in between as unknown ...
@@DarkInovator And according to what you just said, these ratings seems to be random ;D I see only one criteria "oh, I like it" or "I don't like it". And it have nothing to do with actual consistence or quality, look closer ;)
Because it encourages goons to whine about it in chat which (j/k)......... DRIVES ENGAGEMENT .... simple really. Learn to ask WHY? Prior to reacting. I have noticed, he does it all the time (lots of other tubers do too), every time he says something obviously stupid, or overly supportive of a given project, or idea. Don't react ....but instead ask yourself why did he just make that obviously contentious statement. Just cheap tricks to drive engagements by using YOUR attention to get on the algorithm's good side.
The only reason I said no to even trying runway is it's complete lack of few free prompts, and a yearly subscription plan with no monthly, cancel anytime plan. So it's, you can't try it to see if you want to buy it, all you can do is chuck £145 and hope. Nope. Imagine going to a supermarket, where you have to pay for what's on your shopping list before you are allowed in, then your £1 'can of beans', has two beans in. You wouldn't even go in, knowing from others that this is how that shop works.
0:16 If it's just "decent. it's [pretty] good" why is your title "The King of AI VIdeo is here. It's all over..."?? At least you didn't call it stunning.
I'm so over all this generative AI produce. To begin with it was interesting, even impressive but it's still pointless and weird. We already have a whole industry of CGI which is degrading the movie industry imo.
@@musicbro8225 I disagree, with the crap movies that are release nowadays once this technologie allow for full production it will unleash an era of unheard productivity and I have great expectations about it
I think it's a mistake (and not helpful) to give high ratings when the model produced things that weren't in the prompt (like the debris in the bullet shot) or missed things that you asked for, or contained weird morphing. Things like "Oh the shot wasn't exciting enough" shouldn't lower the rating, because you didn't ask for exciting. I'm a big fan of your videos but this one was a bit of a miss for me.
The movie industry is finished within five years. What costs 200 million dollars to produce today will cost thirty cents worth of electricity to make. Actors will still be able to get work in live theater.
It cannot do image to video which is essential for film making as character consistency is prime. Right now it's only a good toy to play with not for professional work.
Runway Would be cool if they didnt want $100 + to Make videos. LM studio or SD4 Needs to figure out how the GP can use this with-Out having to worry about Spending there Rent on it.!
Impressive demonstration of Runway ML's Generation 3 model! The quality of AI-generated video clips is astounding, though some glitches still need ironing out.
WesBot Prompt: “Rate videos for us humans on a 1 to 10 scale. Give us both a ```prompt following``` rating and a separate ```video quality```. After you have the original values, subtract at last 1 from them 90% of the time”
You don't see curvature of the earth from a sky dive. It's curvature of the camera's lens. When you point the camera so that the horizont reaches the bottom of the picture the curvature goes even up (it switches 180 degrees upward to the sky). So it's a camera effect. The curvature of the earth is way lower than you think.
17:43 - Not bad, a relatively good one, the car moving by is pretty impressive how coherent it is (despite coming out of nowhere all of a sudden but its still good)
By "YouSum Live" 00:00:00 Runway ML Gen 3 video generation capabilities 00:00:16 Quality and diversity of video outputs 00:00:51 Rating system based on AI video glitches 00:01:03 Drone shot of waterfall and cliff scenery 00:01:20 Realistic documentary shot of a crab 00:01:42 Monsters at a house party Dreamscape 00:02:11 Drone shot between neon colored buildings 00:02:37 POV snowmobile ride on a snowy road 00:03:00 Cinematic time-lapse of woman reading a burning book 00:03:24 Testing various prompts for video generation 00:04:01 Burlesque dancer drinking a beer scene 00:04:37 Woman manipulating flames with hands 00:05:02 Monster playfully hopping into view in a forest 00:05:24 Woman looking out of a train window at dawn 00:06:00 Underwater futuristic shipwreck scene 00:06:40 Extreme closeup of a woman's eye blinking 00:07:28 Historic footage of California during the Gold Rush 00:08:02 Vintage SUV driving down a mountain road 00:08:38 Testing prompts with different vehicles 00:13:58 Cat blacksmith scene 00:15:38 Racing video game through a city 00:16:24 Review of various video prompts 00:16:52 Rating different video scenes based on quality 00:17:01 Bullet's point of view in a busy street scene 00:17:20 Bird's point of view in a busy street scene 00:17:54 Skydiver's perspective captured effectively 00:18:13 Motorcycle racing in rainy night, impressive visuals 00:18:40 Gender specification in historical context critique 00:19:01 Animated toaster-human interaction, lacking interest 00:19:40 Mythical creatures at a house party, nonsensical but creative 00:19:57 Monsters at a house party, Muppet-like characters 00:21:16 Testing lip sync technology 00:22:00 Experimenting with different prompting mechanisms 00:27:01 Runway ML's recommended prompting guide 00:27:56 Testing various camera styles and movements 00:30:43 Encouraging experimentation with prompts for better results By "YouSum Live"
15:27 - Not too bad but it does not look like the cut si actually running, looks like someone is holding the cat above ground and running with him forward
I've worked with Luma Dream AI quite a bit, and I've watched a livestream by MattVidPro who was generating videos with Runway Gen 3, and my impression is that Gen 3 is not much better, if better at all, than Luma. It would be interesting if you could compare those two, maybe make a shootout with your rating system, giving the same prompts to both models and then rating each video, then we'll see which model gets more points, Luma or Runway.
2-3's at best. This stuff all looks plainly AI with weird problems and inconsistencies. These video generators also just tend to look more and more like video games than actual video as they develop, which isn't great.
those cats at around 15:00 are nightmare material. the first one has a second head where its right ear should be and the second one has the ears facing backwards.
Can it make 180 3D VR Immersive video format? For prompts like ‘X-Wing flying through Death Star tunnels’, ‘ Riding on the back of a dragon’, ‘Chased by a TRex’, ‘Indiana Jones minecart chase’, etc.
I'll agree on the video front - like that dude running with the city in the background creeped me out big time. His man tits were not right. Pecs. That's the word. That's quite a broad generalization to make in regards to ai images though - they are kinda good these days. Made with a bit of care to details they are close to indistinguishable - very little giving them away. I mean, all the images on the internet are AI now, basically. All of them. And the people - AI. I'm AI even, shits wild
I hope someone creates a video bot arena so we can start comparing models and getting ratings. Probably too expensive to run and a lot of generations will be pretty obvious since there are so few good models. But it's something that will be very helpful in the next year or so.
"Formally, a latent space is defined as an abstract multi-dimensional space that encodes a meaningful internal representation of externally observed events. Samples that are similar in the external world are positioned close to each other in the latent space. . To better understand the concept, let’s think about how humans perceive the world. . We are able to understand a broad range of topics by encoding each observed event in a compressed representation in our brain. . For example, we don’t keep in mind every detail of the appearance of a dog to be able to recognize a dog in the street. . As we can see in the illustration below, we keep an internal representation of the general appearance of a dog, in a similar way, the latent space tries to provide a compressed understanding of the world to a computer through a spatial representation."
SnorriCam - (also chestcam, body mount/bodymount, or bodycam) is a camera device used in filmmaking that is rigged to the body of the actor, with the camera facing the actor directly so that they appear in a fixed position in the center of the frame. 🙏
As far as I can tell, image and video generators can make some very impressive looking stuff, but making something very specific is next to impossible. Yes it looked very good, but try to imagine something in your head in as much detail as you can, and then try to create that with a image/video model. I'm not sure that is even possible at this stage.
I think having multiple ratings would be better, such as one for aethetics, another for detail accuracy/consistency and another for prompt accuracy. so the dog on the skateboard might be 5/2/3, something like that. i also noticed that these genartive ai's do best when there is a single subject that it focuses on and not multiple subjects that are interacting with one another or even interacting with a specifc environment. So it always messes up fight scenes or any movement beyond traveling through a static enviroment. Perhaps if it was trained more on dancing, couples figure slating and boxing/fighting it would do much better. For specific environments, video of skateboarders, parkour, gymnastics apparatus and weight lifting. less pov first person shooter and more oblique view of mortal combat as an example when trained on video game training data.
14:22 "This cat is a jerk". Great comment! Just to say, if a cat acts like that, it's probably being mistreated or otherwise not having a good environment. Just the fact that the owner straps a camera to it says a lot about its existence.
8:50 It reminds me of camera sticks that the camera software mostly filters out, but that artifacts do creep in once in awhile. Maybe the LLM reviewed tens of thousands of video clips of this "camera stick" phenomena and included it.
I feel like the ratings you gave were a bit arbitrary. Next time perhaps you could set up a criteria to follow. For example: Did it include every aspect of the prompt, did it adhere to the logical context of the prompt, did it bring everything together in a believable way, are there visual artifacts or inconsistencies, did the video exceed expectations? I don't think you should be afraid to be critical, especially when these products are going to continue to improve indefinitely.
17:13 - This one is not looking convincing, its nice looking but does not look like actual bullet flying midair in slowmo, it looks like a normal speed footage of a pen like object that is floating/flying through air in a relatively slow speed.
Gotta say the female soldier looked fine, so I am not sure what you meant. It looked like her hair could have been put up in the helmet. Anyway, love your videos keep up the great work.
Though... judging by her hands, her species counts in base 12. That might've been the deal-breaker ;) (Just kidding. Yeah, I thought that one was pretty good too, apart from the not-very-WWII gear.)
Honestly it seems that there re a lot of re-used assets regardless of the prompt. Also it seems like every clip moves at the same speed. The sports car, bike, cat, and waterfall seemed to have the same pace. I can't really say I'm impressed when a dude with a camera or computer can film/animate actual breathtaking shots and videos. The technology is cool but is very much in its novelty "party trick" phase and doesn't consistently create at a competent level as of yet.
1:30 Shiny ! .....apparently some crabs decorate themselves as camouflage?! Decorator crabs are crabs of several different species, belonging to the superfamily Majoidea (not all of which are decorators), that use materials from their environment to hide from, or ward off, predators. They decorate themselves by sticking mostly sedentary animals and plants to their bodies as camouflage, or if the attached organisms are noxious, to ward off predators through aposematism.
@@kliersheed Well it's looking like it's not as far off as many think. About a year ago a concept like that was in the realm of science fiction. For it to be within reach and no longer talked about as sifi in such a short amount of time speaks volumes about about how quickly the tech is compounding. I would guess the timeline will be somewhere between 1 and 10 years. The vr obviously will be secondary, but speaking about a game or concept you want to experience and having it in real time will likely be reality very soon.
I don't need to use a hyperbole. I can use Runway Gen-3 to make Madelyne Pryor & Nate Grey scene. This is not a false reality humans. This is transformative technology for me.
I believe bullet digging water came from matrix bullet time where was sort of watery substance around bullets symbolizing air shock wave or something like that
you have a natural fade in and fade out of such objects in videogames if they come to close to the player POV field. could totally be a game-data feature. also often happens with twigs and stuff so you can still see where you are going and dont feel like having them stuck in your face xd.
the people in the video and animation world are cooking this guy today , so fun to watch, stay away from commenting things you don't have knowledge of, stick to coding bro
The "before plunging down ..." in the waterfall prompt is probably the issue -- the model misidentifies the referent, the subject of "plunging." So the water is plunging. Initial text comprehension seems not great; probably Sora is much better there.
i think the real test is if you can't tell if its generated. They were all obviously generated, with all sorts of weird ai gooey feels. i give half of your scores and pretty much all zeros for professional use besides psychdelic music videos or cinematic dreamscapes
The IA is not good as gpt may be, so i think sora would be much ahead. When you ask for a bird gun, the AI cant imagine how to render it. It is really a problem of "lack of understanding" by the AI, more than video rendering issues.
I think this is more of a critique on your ratings than the technology, but a lot of these were pretty awful and you were being very generous lol. Grading on a curve maybe. Thank you for showing it to us regardless. Interesting stuff as always.
If you rate 5 the robots think they have done a perfect job. If you rate 4 the robots, think they have almost done a perfect job. Surely they shouldn’t have to be explained
Given the comments I think rating is very very hard. I’d say the technology to do this at all is near magic but the gap to really useable and controllable consistent professional video with no artifacts might be some way off. You could potentially spend a lot of frustration to get what you want. Long tail problems
oof, seeing fan-made movies with some crazy good stories in a few years, Evil marvel superheroes anyone? Someday streaming games via control input prompts?
There is a problem with rating videos for how they look instead of what the prompt requested. If we are giving five stars to something that did not follow instructions but still looks good, it reinforces behavior that will make effective prompting more difficult as the model evolves. Most noticeable in this video was camera replacement
One suggestion: Please don't edit your voiceover together with the videos, because when you remove parts of you talking, you also cut the videos, so it looks like the videos jump in a weird way, and it's not clear if it's a quality issue with the generated video or the editing.
There are so many artefacts and inconsistencies in these shots that I really don't think they're useable yet, outside of thrown-together low effort stuff.
this, i wouldnt give more than 4 start to be best (the girl reading the burning book) the crab is a 1 start the leg bits are disappearing very bad
The point is how blazingly fast this is improving
Yeah, your SFX job is safe for another year or so for sure
Even at this point though, I see a reasonable use for this. One of the benefits of prompted AI for art is that it may not create something you want, and the artifacts seem to always be there, even for still images, however, if you have a general vision of a scene, you can enter a prompt, run it 10 times and see if something comes close to what you want, and then use it as inspiration. It might be a hot take to suggest we use AI art for inspiration for self-made art, but how many times have artists seen things in the real world that triggered a spark of creativity, inspired by the art of others? All the time! Asking for something close to what you are feeling can more quickly lead you to your final destination. This could be true for people who paint, make movies or even write music.
@@gubunkithe woman with a burning book had one of the biggest errors - she has extra fingers.
Your ratings were WAY too high.
I know right? This may have been mind blowing 6 months ago, but don't underestimate how spoiled we are
Exactly. Video quality is max 1 star out of five, especially if it doesn't match the prompt, movement is janky or illogical things happen or appear. It's just "cool looking stuff on video with usability or memorability of zero"..
yea 10% them i would rate it 4 the most, rest are mostly 1-3
Yeah, they should automatically remove at least 1 star from all of his ratings for a better data representation lmao
Human: Can a robot create a piece of art?
Robot: Can you? Can you create a video without a camera?
Human: Your videos are garbage, 1 out of 5
Give it 5 years and everyperson will beable to adapt everybook into their own series. People won't even need to write their own screenplays. Just pop Harry Potter into a LLM, generate a Screenplay, pop that into a Video generator and you got yourself a movie/series...
It's like a having dreams but you just woke up and trying to remember what it was about.
Exactly. A lot of AI video models have that dream feel to it.
It's the same thing, essentially. You're a neural network, it's a neural network. It's piecing together visual info it has previously seen, just like we do when we dream.
Some backstory: that normal looking man at the party was the REAL monster!
MiB flashbacks
Yeah that Beardo guy, is harshing the smoke monsters vibe.
Why?
Just bc he showed up uninvited to a monster party and still vibe with them without judgement?
How does a two headed cat that looks like the are walking on a moving sidewalk get a 4? 😂
The dreamscape monster party shot with an iPhone was my favorite. Looked like something out of another dimension. Wild!!!
Damn you are generous!!! 👀 Especially considering that one gotta pay to randomly get actual good / useful footage.
These ratings might approach making sense if you replaced every "out of five" with "out of ten". Maybe "out of twenty" if we allow for duration and storytelling.
Bruh... These shots were insane..? Do you even remember where we were two years ago
@@MeetCutie-zm9ef you fail to consider where will we be three months from now.
@@Juttutin Well I for sure hope you're right on that one, but we do need to start somewhere. And I think this "start" is a pretty fucking good one
Most of the prompts lack creativity. I want to see how the AI performs with something way out there that it hasn't had a lot of training on. Idk, surreal, like "A glass baboon riding a spaghetti unicycle in zero gravity"
I think your prompt is going to be rendered quite easily, what about this: "A hyperdimensional tardigrade wearing a monocle made of crystallized time, juggling quantum singularities while tap-dancing on the synapses of a dreaming god, as fractalized echoes of forgotten memories rain upwards in a symphony of bioluminescent nostalgia."
@@zdenekburian1366 In theory yes but the problem with verbose prompts is that the renders are limited to 4 seconds. The engine is going to try to parse the key points and ignore the rest. What you end up getting is a mess that looks nothing as intended.
@@RasielSuarez ahhhhaha i asked claude 3.5 to answer with a joke, making fun of your intersting but offensive (for AIs) suggestion and it gave me that comment to post ahhhaha
@@zdenekburian1366 Using AI to converse in UA-cam comments is how the internet dies.
I'm guilty of it too, to a degree, I've pasted answers to specific questions that needed factual answers (in reddit), but eventually a big percentage of comments is going to be people doing this.
Remember this time of mostly human interaction. It will be missed.
@@theterminaldave absolutely; if i could rule the world, i'd outlaw AI in no time, same as smuggling of nuclear technology
Wes, your ratings are wild. What are you smoking? I want some 😂
It ain't all over. It's scaled the slopes of the Uncanny Valley, danced like it's selling nails and stomped on the tera of hyper realism. Still, it's only the begining (and you've already gone and lost your mind). We can't even predict the outer limits of this stuff, but I don't think we have the imagination to envision the horrizon. Buy the ticket, take the ride. Just be sure you're strapped in Scouts!
Most youtube videos use clickbait nowadays.
i just want life to become a non problem, whole life.
Those videos were INCREDIBLE! I am a smart human, but not artistic at all, and I could NEVER make a video as good as any of those you showed. Just the little details like the sand, the clouds in the sky, the movement of the water down the waterfall - incredible.
the girl with the book on fire has 5 fingers and a thumb on her left hand...
Your ratings are terrible even by your own conditions you set. Are you seriously not noticing all of the AI video glitches and inconsistencies in the 5s you rated?
yea, same thought here. This model is doomed with that kind of rating XD Something is fckd up to the point it's basically unusable - yea I like it, 4 out of 5. Another one with no glitches - yea, it's ok but there is not much movement, 3 out 5. Logic died long ago. Probably the same moment one's getting convinced about "meta" in YT titles. Shocking.
as he stated the rating is relative ... you can't expect high quality video out of generative models at the moment. What you should look for is how consistent it can keep track of concepts between frames (and for how long it can keep track of them). Like that monster shot where the monster turns his head you can clearly see that an extra eye got generated. Almost as if the model lost track of the "head" and "position" concept. Anyway the progress is really good considering a little over a year ago models were generating morphing dream collage.
Lol. There are literally no criteria. He is just saying numbers. Maybe it's not worth thinking too deeply about.
@@user-ty9ho4ct4k o ya I agree that the numbers scale is pointless. Its his arbitrary scale, when he explained it at stat I interpreted it as a bucket between 2 choice "good" and disappointing. With the in between as unknown ...
@@DarkInovator And according to what you just said, these ratings seems to be random ;D I see only one criteria "oh, I like it" or "I don't like it". And it have nothing to do with actual consistence or quality, look closer ;)
22:42 - Nice, that's a relatively good one, almost looks like actual real video footage from a movie
The Burlesque Dancer also grows another finger while holding the bottle, freaky.
I have no problem with that.
Well fuck ye, i bet we wish we all could grow more fingers to drink more beer 😅 Bring on the beer fingers gene count me in.
almost all of these were like a 2 MAX.
there are issues all over the place.
Really? Where are the issues at 2:45?
@@StefanReich middle right of the video, bottoms of trees disappear into snow while leaving the tops floating. at 2:48
In one of the bike scenes where day changes to night and there is rain, the shadow of the biker in the wet road changes weirdly
18:16 - WOW ! this one is the best so far, incredible, now this is actually a true 5/5 stars !
Way too generous with the scores here
Yep these look absolutely horrible. Gives me headache. Good step forward but still horrible.
Because it encourages goons to whine about it in chat which (j/k)......... DRIVES ENGAGEMENT .... simple really.
Learn to ask WHY? Prior to reacting.
I have noticed, he does it all the time (lots of other tubers do too), every time he says something obviously stupid, or overly supportive of a given project, or idea. Don't react ....but instead ask yourself why did he just make that obviously contentious statement.
Just cheap tricks to drive engagements by using YOUR attention to get on the algorithm's good side.
The only reason I said no to even trying runway is it's complete lack of few free prompts, and a yearly subscription plan with no monthly, cancel anytime plan. So it's, you can't try it to see if you want to buy it, all you can do is chuck £145 and hope. Nope.
Imagine going to a supermarket, where you have to pay for what's on your shopping list before you are allowed in, then your £1 'can of beans', has two beans in. You wouldn't even go in, knowing from others that this is how that shop works.
0:16 If it's just "decent. it's [pretty] good" why is your title "The King of AI VIdeo is here. It's all over..."?? At least you didn't call it stunning.
ABC - Always Be Cheating
Yeah, it gets annoying
Or else you won't click it.
@@prasannakarthik7721 Not true. I click Luma videos all day and I don't care at all about the title
UA-cam algorithm favors titles like that. It's not really his fault, he has to work the system to get it in front of people
A tool that doesnt do what you ask it to do is not a good tool... those 5 stars are meaningless
I can't believe anything i see anymore
Nothing is real
I'm so over all this generative AI produce.
To begin with it was interesting, even impressive but it's still pointless and weird.
We already have a whole industry of CGI which is degrading the movie industry imo.
@@musicbro8225 I disagree, with the crap movies that are release nowadays once this technologie allow for full production it will unleash an era of unheard productivity and I have great expectations about it
@@huguesviens
Nice to hear comment from someone with clue.
thank you for explicitly defining the grading scale from the start :)
I think it's a mistake (and not helpful) to give high ratings when the model produced things that weren't in the prompt (like the debris in the bullet shot) or missed things that you asked for, or contained weird morphing. Things like "Oh the shot wasn't exciting enough" shouldn't lower the rating, because you didn't ask for exciting. I'm a big fan of your videos but this one was a bit of a miss for me.
The movie industry is finished within five years. What costs 200 million dollars to produce today will cost thirty cents worth of electricity to make. Actors will still be able to get work in live theater.
Not without real world models. So far except for some hypothetical JEPA stuff, no one has any idea of how to do it.
"Approach waterfall and then plunge down"
*Vague waterfall shot with slight zoom, no plunge*
"Bravo, 10 out of 10, stunning!!"
It cannot do image to video which is essential for film making as character consistency is prime. Right now it's only a good toy to play with not for professional work.
That flame one is impressive. The lighting etc.
Runway Would be cool if they didnt want $100 + to Make videos. LM studio or SD4 Needs to figure out how the GP can use this with-Out having to worry about Spending there Rent on it.!
Impressive demonstration of Runway ML's Generation 3 model! The quality of AI-generated video clips is astounding, though some glitches still need ironing out.
Thank you for the video examples of the prompts! Much more helpful than the still images on the site alone. :)
3 days later "It's all over again ! MInd Blowing .... blah blah yada yada"
WesBot Prompt: “Rate videos for us humans on a 1 to 10 scale. Give us both a ```prompt following``` rating and a separate ```video quality```. After you have the original values, subtract at last 1 from them 90% of the time”
You don't see curvature of the earth from a sky dive. It's curvature of the camera's lens. When you point the camera so that the horizont reaches the bottom of the picture the curvature goes even up (it switches 180 degrees upward to the sky). So it's a camera effect. The curvature of the earth is way lower than you think.
KING that nails backdrops yet fails in almost everything that is on display on those videos... 🤣
17:43 - Not bad, a relatively good one, the car moving by is pretty impressive how coherent it is (despite coming out of nowhere all of a sudden but its still good)
By "YouSum Live"
00:00:00 Runway ML Gen 3 video generation capabilities
00:00:16 Quality and diversity of video outputs
00:00:51 Rating system based on AI video glitches
00:01:03 Drone shot of waterfall and cliff scenery
00:01:20 Realistic documentary shot of a crab
00:01:42 Monsters at a house party Dreamscape
00:02:11 Drone shot between neon colored buildings
00:02:37 POV snowmobile ride on a snowy road
00:03:00 Cinematic time-lapse of woman reading a burning book
00:03:24 Testing various prompts for video generation
00:04:01 Burlesque dancer drinking a beer scene
00:04:37 Woman manipulating flames with hands
00:05:02 Monster playfully hopping into view in a forest
00:05:24 Woman looking out of a train window at dawn
00:06:00 Underwater futuristic shipwreck scene
00:06:40 Extreme closeup of a woman's eye blinking
00:07:28 Historic footage of California during the Gold Rush
00:08:02 Vintage SUV driving down a mountain road
00:08:38 Testing prompts with different vehicles
00:13:58 Cat blacksmith scene
00:15:38 Racing video game through a city
00:16:24 Review of various video prompts
00:16:52 Rating different video scenes based on quality
00:17:01 Bullet's point of view in a busy street scene
00:17:20 Bird's point of view in a busy street scene
00:17:54 Skydiver's perspective captured effectively
00:18:13 Motorcycle racing in rainy night, impressive visuals
00:18:40 Gender specification in historical context critique
00:19:01 Animated toaster-human interaction, lacking interest
00:19:40 Mythical creatures at a house party, nonsensical but creative
00:19:57 Monsters at a house party, Muppet-like characters
00:21:16 Testing lip sync technology
00:22:00 Experimenting with different prompting mechanisms
00:27:01 Runway ML's recommended prompting guide
00:27:56 Testing various camera styles and movements
00:30:43 Encouraging experimentation with prompts for better results
By "YouSum Live"
15:27 - Not too bad but it does not look like the cut si actually running, looks like someone is holding the cat above ground and running with him forward
I've worked with Luma Dream AI quite a bit, and I've watched a livestream by MattVidPro who was generating videos with Runway Gen 3, and my impression is that Gen 3 is not much better, if better at all, than Luma. It would be interesting if you could compare those two, maybe make a shootout with your rating system, giving the same prompts to both models and then rating each video, then we'll see which model gets more points, Luma or Runway.
Great examples and analysis. Thank you.
2-3's at best. This stuff all looks plainly AI with weird problems and inconsistencies. These video generators also just tend to look more and more like video games than actual video as they develop, which isn't great.
those cats at around 15:00 are nightmare material. the first one has a second head where its right ear should be and the second one has the ears facing backwards.
🤣
*shifting shapes and hair that randomly pops up from under a shoulder strap* 5 out of 5!
Thanks for the update Wes 🎉
8:29 - Looking good overall but you noticed there's no driver inside the car? LOL
Can it make 180 3D VR Immersive video format? For prompts like ‘X-Wing flying through Death Star tunnels’, ‘ Riding on the back of a dragon’, ‘Chased by a TRex’, ‘Indiana Jones minecart chase’, etc.
Scuba diver was in white in the middle upper right, had the long flippers on.
21:45 These are the drones we're gonna have to fight in the future. Their hands getting longer like some street fighter shit.
Nightmare juice. For me AI images and videos feel off in ways only in seen in my worst nightmares. They creep me out.
I'll agree on the video front - like that dude running with the city in the background creeped me out big time. His man tits were not right. Pecs. That's the word.
That's quite a broad generalization to make in regards to ai images though - they are kinda good these days. Made with a bit of care to details they are close to indistinguishable - very little giving them away. I mean, all the images on the internet are AI now, basically. All of them. And the people - AI. I'm AI even, shits wild
18:44 - Good one, nice
The blue monster has six legs, that’s cool. The white SUV is driverless, you didn’t specify with or without.
You're super generous!
I hope someone creates a video bot arena so we can start comparing models and getting ratings. Probably too expensive to run and a lot of generations will be pretty obvious since there are so few good models. But it's something that will be very helpful in the next year or so.
"Formally, a latent space is defined as an abstract multi-dimensional space that encodes a meaningful internal representation of externally observed events.
Samples that are similar in the external world are positioned close to each other in the latent space.
.
To better understand the concept, let’s think about how humans perceive the world.
.
We are able to understand a broad range of topics by encoding each observed event in a compressed representation in our brain.
.
For example, we don’t keep in mind every detail of the appearance of a dog to be able to recognize a dog in the street.
.
As we can see in the illustration below, we keep an internal representation of the general appearance of a dog, in a similar way, the latent space tries to provide a compressed understanding of the world to a computer through a spatial representation."
SnorriCam - (also chestcam, body mount/bodymount, or bodycam) is a camera device used in filmmaking that is rigged to the body of the actor, with the camera facing the actor directly so that they appear in a fixed position in the center of the frame. 🙏
As far as I can tell, image and video generators can make some very impressive looking stuff, but making something very specific is next to impossible.
Yes it looked very good, but try to imagine something in your head in as much detail as you can, and then try to create that with a image/video model. I'm not sure that is even possible at this stage.
I think having multiple ratings would be better, such as one for aethetics, another for detail accuracy/consistency and another for prompt accuracy. so the dog on the skateboard might be 5/2/3, something like that.
i also noticed that these genartive ai's do best when there is a single subject that it focuses on and not multiple subjects that are interacting with one another or even interacting with a specifc environment. So it always messes up fight scenes or any movement beyond traveling through a static enviroment. Perhaps if it was trained more on dancing, couples figure slating and boxing/fighting it would do much better. For specific environments, video of skateboarders, parkour, gymnastics apparatus and weight lifting. less pov first person shooter and more oblique view of mortal combat as an example when trained on video game training data.
14:22 "This cat is a jerk". Great comment!
Just to say, if a cat acts like that, it's probably being mistreated or otherwise not having a good environment. Just the fact that the owner straps a camera to it says a lot about its existence.
this is scary good and we are still very early in this type of development!!! wow this will change everything!!!
'It's all over"/"Just Beginning"
1$ per generation is absurdly expensive for a model that misses so much.
8:50 It reminds me of camera sticks that the camera software mostly filters out, but that artifacts do creep in once in awhile. Maybe the LLM reviewed tens of thousands of video clips of this "camera stick" phenomena and included it.
I feel like the ratings you gave were a bit arbitrary. Next time perhaps you could set up a criteria to follow. For example: Did it include every aspect of the prompt, did it adhere to the logical context of the prompt, did it bring everything together in a believable way, are there visual artifacts or inconsistencies, did the video exceed expectations? I don't think you should be afraid to be critical, especially when these products are going to continue to improve indefinitely.
Jeeze Louise . Like it . I’ve been using Pika , so time to try this out.
We've come a long way in AI video, and there's obviously a long way to go still.
17:13 - This one is not looking convincing, its nice looking but does not look like actual bullet flying midair in slowmo, it looks like a normal speed footage of a pen like object that is floating/flying through air in a relatively slow speed.
AI voice over?
Gotta say the female soldier looked fine, so I am not sure what you meant. It looked like her hair could have been put up in the helmet. Anyway, love your videos keep up the great work.
Though... judging by her hands, her species counts in base 12. That might've been the deal-breaker ;)
(Just kidding. Yeah, I thought that one was pretty good too, apart from the not-very-WWII gear.)
13:25 - Ok, that's a good one
My ONLY interest is continuity. Can it continue to generate additional clips in different scenarios using the same character?
"cat moving it short grass" that was exacly what it showed. she's moving... in short grass.
The monster rave has a huge eye glitch so it deserves a 4 not a 5
6:57 - That's a good one, I'd give you that
Honestly it seems that there re a lot of re-used assets regardless of the prompt. Also it seems like every clip moves at the same speed. The sports car, bike, cat, and waterfall seemed to have the same pace. I can't really say I'm impressed when a dude with a camera or computer can film/animate actual breathtaking shots and videos. The technology is cool but is very much in its novelty "party trick" phase and doesn't consistently create at a competent level as of yet.
1:30
Shiny !
.....apparently some crabs decorate themselves as camouflage?!
Decorator crabs are crabs of several different species, belonging to the superfamily Majoidea (not all of which are decorators), that use materials from their environment to hide from, or ward off, predators. They decorate themselves by sticking mostly sedentary animals and plants to their bodies as camouflage, or if the attached organisms are noxious, to ward off predators through aposematism.
Is there retaining memory on this so you can repeat characters?
From these five-out-five videos I'm not sure whether a perfect video would be a 6 or a 10 out of five.
that bird view is reminding me of "birds aren't real" campaign LOL where they said birds are fakes robots to spy all people
🤣
It's really going to be insane when you can create entire games by just speaking. Paired with some good vr and it will be a godlike experience.
my greatest fear is dying before i get to experience that to the full :D
@@kliersheed Well it's looking like it's not as far off as many think. About a year ago a concept like that was in the realm of science fiction. For it to be within reach and no longer talked about as sifi in such a short amount of time speaks volumes about about how quickly the tech is compounding. I would guess the timeline will be somewhere between 1 and 10 years. The vr obviously will be secondary, but speaking about a game or concept you want to experience and having it in real time will likely be reality very soon.
I don't need to use a hyperbole. I can use Runway Gen-3 to make Madelyne Pryor & Nate Grey scene. This is not a false reality humans. This is transformative technology for me.
I believe bullet digging water came from matrix bullet time where was sort of watery substance around bullets symbolizing air shock wave or something like that
What explains the tank's turret's unhinged footage? Lack of data, lack of representation in the data-set, and/or lack of training.
you have a natural fade in and fade out of such objects in videogames if they come to close to the player POV field. could totally be a game-data feature. also often happens with twigs and stuff so you can still see where you are going and dont feel like having them stuck in your face xd.
the people in the video and animation world are cooking this guy today , so fun to watch, stay away from commenting things you don't have knowledge of, stick to coding bro
Today video and animation people come to laugh, tomorrow they are coming to watch, soon they'll stay at home laid off.
The "before plunging down ..." in the waterfall prompt is probably the issue -- the model misidentifies the referent, the subject of "plunging." So the water is plunging. Initial text comprehension seems not great; probably Sora is much better there.
Thank you!
The "train through the Italian landscape" video has a weird flow field. A tree is even moving in the wrong direction.
i think the real test is if you can't tell if its generated. They were all obviously generated, with all sorts of weird ai gooey feels. i give half of your scores and pretty much all zeros for professional use besides psychdelic music videos or cinematic dreamscapes
The IA is not good as gpt may be, so i think sora would be much ahead. When you ask for a bird gun, the AI cant imagine how to render it. It is really a problem of "lack of understanding" by the AI, more than video rendering issues.
Profoundly generous ratings.
Show us MORE.
Please. 👌
Turning text to video and feeding a new omni model will be the next strategy. let the ai dream.
Wes watches chinese cat fights lol
12:40 him running into a wall definitely training in ue5, it's trying to apply physics and logic to a simulation walking in place.
I think this is more of a critique on your ratings than the technology, but a lot of these were pretty awful and you were being very generous lol. Grading on a curve maybe. Thank you for showing it to us regardless. Interesting stuff as always.
If you rate 5 the robots think they have done a perfect job.
If you rate 4 the robots, think they have almost done a perfect job.
Surely they shouldn’t have to be explained
Given the comments I think rating is very very hard. I’d say the technology to do this at all is near magic but the gap to really useable and controllable consistent professional video with no artifacts might be some way off. You could potentially spend a lot of frustration to get what you want. Long tail problems
oof, seeing fan-made movies with some crazy good stories in a few years, Evil marvel superheroes anyone? Someday streaming games via control input prompts?