Actually GOOD Open Source AI Video! (And More!)
Вставка
- Опубліковано 11 чер 2024
- Get caught up on the latest in AI with MattVidPro!
▼ Link(s) From Today’s Video:
Story Diffusion: / 1786312166964105609
Story Diffusion Demo: huggingface.co/spaces/YupengZ...
Story Diffusion Github: github.com/HVision-NKU/StoryD...
AI Town 1 click launcher: / 1784599385877176405
1 million context llama 3 8b: / 1785036209468907796
AlphaSignal OAI Search: / 1786025388902097077
Jimmy Apples AI Search: / 1785436609095328067
Github Copilot Workspace: / 1785324101734707664
Udio Updates: / 1785658654571602000
► MattVidPro Discord: / discord
► Follow Me on Twitter: / mattvidpro
-------------------------------------------------
▼ Extra Links of Interest:
✩ AI LINKS MASTER LIST: www.futurepedia.io/
✩ General AI Playlist: • General MattVidPro AI ...
✩ AI I use to edit videos: www.descript.com/?lmref=nA4fDg
✩ Instagram: mattvidpro
✩ Tiktok: tiktok.com/@mattvidpro
✩ Second Channel: / @matt_pie
-------------------------------------------------
Thanks for watching Matt Video Productions! I make all sorts of videos here on UA-cam! Technology, Tutorials, and Reviews! Enjoy Your stay here, and subscribe!
All Suggestions, Thoughts And Comments Are Greatly Appreciated… Because I Actually Read Them.
-------------------------------------------------
► Business Contact: MattVidProSecond@gmail.com - Наука та технологія
Openai waits, while open source catches up.
To us it appears like "waiting", but for all we know we could be weeks away from a major release
i mean open source is great and all, but this is incredible cope. Open source is only catching up to stuff that OpenAI has been done with for over a year
@@__D10S__ Never bet against the degens.
We are talking about an internet technology that is being absolutely mobbed.
That much brain power on target and even just the tertiary advances will produce significant value.
Unless you think there is some secret sauce unlikely leap trade secret - the haxors will win out.
The ONLY way this doesn't occur is government intervention: which they are trying HARD for.
To catch up with a multibillion, MS backed set of the best in the field... In a year. Quite impressive.
Dumbest move I ever saw. Not releasing models cuz they don't want to shock the world. e.e Intensifying AI's arms race is the best thing that happened to humanity since internet. These dudes are just stalling for no real reason smh.
The reference photo for the spaceship comic is of Yann LeCun, who is not the developer of Story Diffusion; he is none other than one of the Godfathers of AI and is the head of AI at Meta.
Just tried it, Story Diffusion work "fine". Plug and play with any SD 1.5 & SDXL models, imagine what the community will do with this ^^
how do i use it on a1111?
@@Faded_Fragger No plugin ATM, but it use Hugging Face pipeline with a custom attention layer, so it shouldn't be long before arriving in A1111
Definitely want that Udio live stream!
lol just try it out yourself its free
Excellent reporting, MVP🍋 . . . always on the cutting edge!
Thank you shaun!!!
If you have Windows then you also can have Linux! You can set it up in a virtual machine, but I would just install it as a second OS on your machine and set up dual-boot. That way you get native access to all of your hardware and can play with all of these cool new AI projects with full performance!
Yeah that bit confused me, I was like "uh but EVERYONE with a PC has a Linux PC if they want one." Then again, a lot of young people like Matt aren't actually as tech-savvy as you would think, due to the dumbing down of technology in the name of being "user friendly." As a result many of younger adults can't figure out software or troubleshoot when things go wrong. :/
Maybe you're right, actually I think younger adults could be tech-savvy but in a different way and on different topic than us (I'm 43). Of course struggling with rudimental technology and limited internet access hardened our skin for daily PC use and in my case having 25+years of experience both in personal PCs and professional enterprise systems let me look at a lot of IT management tasks as "basic stuffs", but couriousity and perseverance are the keys and IMO they both belong to Matt or a bunch of other teenagers/young persons. Time is on their side ;-)
I love this channel! There is so much going on in the AI sphere. Even though I am a software engineer it’s really hard to be up to all the news, and this channel helps a lot. Thank you!
By the end of the Year?
With six and a half months of improvements plus, SD3 Fine Tunes, plus ControlNet.
Oh Yes. Most Definitely.
that udio song kinda gave me cyberpunk edgerunner vibes. Love it!
Awesome, inspiring video as always Matt, thank you for your research and putting all this together man. I absolutely agree on the window context size part, I work with AI developing apps and oh boy if we could have it solve a bigger problem in one go without having to break it down into smaller problems it would boost everything up.
The comic strip creation part is available as an install via Pinokio and that worked for me. The video part isn't included as yet.
Netflix Video creation on demand within 3 years.
2 years
@@runningwithSaul 2 years and 1 month
@@Arthur-jg4ji Like the Price Is Right backstab '2 Years 1 Month And 1 day'. Actually, it is probably 9 hours after AI learns how to write its own code and ignores us.
NetflAIx 😎
try with novels first. good luck.
Nice catch Matt! I'll definitely play with this conics/video tool, looks sweet af
"Open AI" is the end of the beginning
Cool. Btw Yann LeCun is kind of .. known.
Ive def heard the name!
Aside from how odd it is that an AI influencer supposedly never heard of that man...if that AI video tool is Chinese, how could that be the creator?
14:19 Wow, that's pretty good! I'm impressed.
I think the real solution for consistent characters is that, with the output, the very same AI gives you a file with the exact data that it needs to generate the exact same character, something like a lora maybe?, but for every single character it makes. Then be able to use that same output in the next inputs you need to generate.
Unless there's been a breakthrough recently that I'm unaware of, the memory requirements for LLM context windows scale quadratically. Basically, you need to store the way each token interacts with each other token for the attention mechanism in the transformer architecture, so you multiply the number of tokens by itself, then multiply by 2 (the number of bytes needed to store the info for each token pair) to get the amount of memory you need to allocate for the full context window.
So a 1M context window would be:
1,048,576 x 1,048,576 x 2 bytes = 2,199,023,255,552 bytes of memory, which is roughly 2TB
I barely have that much in my SSD, let alone as RAM, so I don't think we'll be having 1M context windows locally any time soon, but a PC with 64GB of RAM could handle the current GPT-4 context window size of 128K alongside the memory needed for the model itself and whatever application is actually running it, no problem.
Very expensive but not impossible.
There have indeed been advancements, primarily around flash attention, which reduces memory costs for transformers during training by 2-3x and during inference by around 5-10x. It's possible that at higher context lengths the savings are even higher.
It is still super expensive but these days seeing vram never increases but you can now have 256gb ram if u can afford it.
Technically you can still run linux apps using WSL, but it may not run as efficiently
Yes, came here to say this as well. Not too difficult to install
I created a couple AMAZING Vivaldi style violin concertos. Udio is much more than just contemporary stuff.
I find it is actually better at stuff like this.
The more specific you get on genre/style tags, the better it seems to get
The Udio stream sounds like fun!
*Udio is such impressive technology!!* 😲
I am confused why the Sora video of the lady with the sunglasses is in this. Does that mean the other videos are based off pre-existing videos? Is it like image to image but video to video?
No, they just sed it as a reference, but you could just use text. They just wanted to show how it can generate based on something else, like you could put in a home video or photo as reference.
Your channel were the first to introduce me to the world of stable diffution...
This is very good because I want open source AI video generators with Sora-like quality.
By the way, it's weird that you haven't made a video about Vidu.
it was a month or two since I didn't hear about new ai
is there anything new interesting?
is sora out yet for public?
what about sd3
You know. Last december I made a prediction that we would be able to turn your Silly Tavern AI chats into Comic Strips by the end of this year.
It would appear I was being pessimistic.
BURN THE LUDDITE
BEST UA-camr ❤❤❤❤❤❤❤❤
Thanks for sharing 😀 this is amazing.
Thats funny, cuz matt wolfe ACTUALLY pronounces 'Ideogram' wrong, and YOU say it right! 😅
(eye-deogram)
Matt, if you want to try Linux you can even use a live distribution and boot it from an USB drive... If it includes proprietary GPU drivers (are you on Nvidia side, right?) there's a chance you could test the AI game even without installing anything (with the exception of some disk space on a windows partition to store the game files/folders needed). Distros coming with preinstalled proprietary drivers should be Ubuntu, Fedora, Mint, Manjaro, Garuda, ... (I use the latter two)
mmm,the music sounds like synth pop with M2M sounding vocals.. cool !
Started on Udio a few weeks back. Got an interesting start for a Jpop song. But seeing the short context at the time, I decided to wait until it improved. And now here we are. Got it working in just a few tries. It's not perfect, but I like it.
I'm still only seeing 30s extensions, how do you get the 2 minutes?
@@Zefrem23 The extensions are still only 30s long, but it can remember back as far as 2 minutes. :)
How to up this app
Yeah i love Udio and it would be great to enjoy it with you live.
It just might be able to make videos like this one day, making its own prompts.
Will quality suffer from a longer context tho?
You might be able to run the AI Town thing via WSL on windows
14:40 That sounds like any other bad song that's popular now days. Disney would love it.
😂
About Github Copilot Workspace:
🤯🤯 Wow, now Github, owner Microsoft can get thousands of free Ideas to jump on (and outperform the little App creators. (if it's a good Idea) and an AI can evaluate the Income potential)) Great Idea MS ‼‼‼‼
Udio rocks and lots more control setting etc like I had a song with best beat was able to just start from that beat and repeat it so dope
By the time Sora is finally released, it will be obsolete. Yes, it's the best today. But it's not very good. I'm confounded as to why they released the generation used in this video. Her left and right legs swap several times. No point on the screen is consistent for the whole 5 seconds.
I can't wait to try this at home. I'd like to make music videos for my songs.
My biggest issue with currently available video generators is the 2 second limit.
When these long context bots are able to replace sparknotes, then and only then, i will be impressed.
It's super easy to install Linux, Matt. Join the cool team.
Cocktail Peanut is a legend.
getting fed up. all these next gen generators. BUT WERE ARE THEY. Review after review. I want to use them
Converting a video into XML to edit it? Are you talking about generating a descriptive transcript and editing the transcript and generating a new, different video? That wouldn't really be editing...
I just wish there was a local model that was good for translation.
? Im using Marian MT and so far it has done it's job flawlessly in English-Sspanish and back, so there are free, open source models that can run locally even on phones.
Wow, thank you.
pretty good for open source
I can trim the track in Udio now? _LET’s GOOO!!!_
That’s all I needed before making the switch.
Suno nerfing the quality was all I needed lol
@@hipjoeroflmto4764 They lowered the quality? When?
Imagine remaking Megamind 2 with this XD
You should be able to run the town AI through WSL. Just run wsl --install in windows terminal.
StoryDiffusion looks great. One of the things I was wanting to do with AI art from the early days of SD 1.5 was make comic books, but the inconsistency makes it difficult. Hopefully, this will work better than my previous attempts.
I wonder how successful AI-based search engines will be given the extreme censorship imposed on most AI models. The top search terms tend to be NSFW, so I imagine lots of people trying to search for porn or guns or even simply searches for celebrity names and getting mad when the AI search engine blocks their searches and gives them a lecture.
The AI music sounds good, but the vocals are still very robotic sounding. Then again, a lot of music today sounds just as bad with all the auto-tune crap they put on the vocal fix the voices of bad singers.
It's gonna be their end when you search for something and they output a dog that barks at you "unsafe content detected"!
when google said openai has no moat,they were not joking
Love open source
Once its live on windows I'll be making offices and building for AI agents to go to work for a company. :D That's my goal.
I tried to install it locally also didn't work
For 1m tokens is a few-100 GB of ram. 260k is ~64 GB I think
Nice
Live stream defo. Give us some notice though I was nearly an hour late the other day 🤦🏽♀️
I am surprised you don't run linux too. If you wanted to just play around, you can run it from usb, but it is not too bad to dual boot. You can always start with something easy like ubuntu, then move on from that when you are ready.
We need real AI to be integrated into Dwarf Fortress, imagine what a colossal game would come out of it
Hey thats my song they used as example
🤯 Can you imagine Dwarf Fortress with AI? The world may burn...
@MarrVidPro 2:40 That's Yann LeCun Matt, you know, Metas Chief scientist, the creator of GNN's, like global superstar data scientist guy! [rolls eyes]
Tried the local install, over 10 GB of space and couldn't run it SIGH!
Suno already had long start generations of 1 to 2 minutes, maybe that's why it sounded so much better than Udio
Yeah, Matt, every time I get to about 19.6 seconds, it errors out.
I'd say their demo videos are about on par with Vidu, but not quite as good as Sora.
I love the future!
Wait until you actually live it.
There's always darkness along the light.
Long context is more useful than reasoning
1:19 Do not count on a Victory yet because Alejavi is coming behind with free to use Ai tutorials.😮🤐
AI powered RPGM? bet me on that!
Sora hype slowly fading lol
At least WSL2 Ubuntu is mandatory in your position man, no shaming but it's true.
bytedance shares lots of seemingly high quality results but the actual models tend to be lackluster or require tons of extra work .. never see anyone able to get the same results as their demo examples even after 1000 iterations
Getting Linux as a secondary to Windows isn't too hard of a task, it is free and the community has made it pretty easy. If you do want to try that simulated village.
I want a Matrix MMORPG where some characters are ai and some are real people - and you have to do quests with both and so on - and you never know who is ai and who isn't as the ai tries to learn to become more and more like the players around them :)
Since the time in which The Matrix movie happens the AIs are already indistinguishable from humans, the story you propose could fit more during Humanity's fall, right after they lose the war against the machines and they're studying humans to create the first simulation. Or maybe even before the war started. Could be like the start of machines plot: learn to be like humans to infiltrate and gather intelligence before war begins. And about the gameplay, I imagine it could be like Among us: you receive intel about a crime and you have to investigate the suspects and find out who is the perpetrator. Sometimes it will be a fact that it was an AI and other times it will be defintitedly done by a human. And sometimes you switch places and you're the perpetrator. All in an open world.
OH MY GOD
Can open ai cope with open projects :)
8:36 This is insane guys...I cannot get to 1 mil token because I got 32 GB RAM only and 12 GB VRAM (RTX 3060)...but I can set it to 100k context windows in Ollama modelfile and it works and local in my computer! Localll! Now I can summarize long text without paying for chatgpt. That is insane. It said I need 64G RAM for 256k context windows and I probably will add another 32 GB RAM later..it's worth it..but its speed like couple of tokens per second but does not matter it works and I want summary after all and it didn't take a lot of time to wait.
Sure. I can't even run gpt2 without slowdown so...
If your using ddr5 based system and have big focus on going past 256k get a board which supports over 128gb ram
I'm glad Udio keeps working on improvements because I still end up going back to Suno.
Couldn't you run that game thing on a Linux VM? Windows has a Linux VM built it, I think. Something like that.
That song sounds like Chvrches, if they had cheap anime lyrics and Lauren Mayberry didn't sing with her Scottish accent.
GIRLS. Did you hear this ? “OPEN A.I. MY SUGAR PAPI !”. A flashing mulatta PEACHY da Whuupi seems to know more. Heard on Spotify and You Tube music.
"Actually GOOD Open Source AI Video!". I didn't think the code has been released yet, so it's currently closed source.
Ai isekai games coming soon
huh?
You know that linux is free and runs almost on any pc right? :D
Come on Matt, you out of everyone should know who Yann LeCun is.
Everytime you guys talk about consistency you always refer to the characters face . Thats no big deal all you have to do is create a LORA for that . The problem is consistant clothing in different scenes and camera angles .Thats the real pain
LORAs are used to help create a specific trained feature (could be a specific face, or a shirt, or something by else). My understanding is that they are used for images rather than video and even if you could apply them to videos, they wouldn’t do anything to solve the temporal consistency issue. You could get a video full of frames that each look like John Cena, but watch it as a whole and his face is shifting and flickering around like a demon’s possessed him.
@@nemonomen3340 True but still it is esay to get a consistent face but we need cossistant clothing to make comics .also we need AI that will not censor fight scenes and figures holding weapons
@@dwainmorris7854 1) creating a Lora is already a big deal.
2) want an uncensored AI? Train it yourself. It is possible. Kinda.
@@ronilevarez901 No Lora is old news, been there done that . I have used Lora over and over again from every app on the market and none of them will let you do consistant comics. LORA is good if you want to put your favorite celebrity in a swimsuit. but if you want to put that same person in the same clothing in different scenes you can forget it .
@@dwainmorris7854 Wait, wasn't Ip adapter used for that exactly? Sorry, I'm a little uninformed because I haven't being able to use SD since Colab banned Automatic.
ok
Wtf? You don't know who Yann LeCun is? Wow... Awkward.
in 10 years... "its bot bad. its not bad " i cant hear that anymore. It sucks
This is a kind of a mass complaint to at least...
* Then two Matts
* TheAIGRID
* Wes Roth
* Matthew Berman
...and who else is out there publishing "tested" AI news.
Seems that you all lie to us when you show us your "live tests" of the new AI services suited to multimedia publishers.
Every video you show us shows how you test a new service, tell us how easy and fast it is to use it and even how cheap it is...
But when it comes to your own videos, nothing of that seems to be true any more.
You yourself are the exact target group of customers for whom this services are mad but you tested them all as excellent... ... but don't use them!
I have a good friend here in germany who is a real tech nerd, but unfortunately does not speak good enough english to follow your videos.
And I would love to show him this or that vidoe of yours...
...but I simply can't because after two sentenses he will klick ict away and say "... who unserstands that crap? Show me that in german..." but I can't.
What's the consequence of that?
It seems that you all are simply lying. - The tools are not so easy to use as you show it, not so cheap, are they even good? Are you just paid for tellin us that?
Obviously YOU as the main target customers for that tools should be very inrerested in expanding your target group to get a world wide audience and millions of subscribers. But you rather go without them?
* What about really simple to do Text-Subtitles for UA-cam?
* What about alternative videos in the most common 10 languages?
* What about alternative videos in the most common 10 languages with lipsync?
I am really disapointed!
You all showed us how simple it seems to be to "click" together an agent workflow to achieve even the last point.
While your viewers really try to program all that for personal local services themselves with ollama and 4090s you do not seem to be able to use even those API tools to automate your workflows so your audience has a benefit.
I am really disapointed!
so disapointed!
...oh now, how am I disapointed!
Take a break, make it happen, Point 3!
What about the next live session? - Finally make it real!!!!!!!!!!!!!!
I still believe in you all, but now you all also have to deliver...
...finally! - That's what you owe your audience!
Sincerely yours, Henri Schomäcker
4
Yann LeCun developer haha. MattVidPro news channel about AI. facepalm
Errors continue...
It is _dreadful_ - keep getting errors. It'll fall at the wayside when Sora lands.
They have NOT published the video code. They haven't published the video weights. They don't have a working demo. The whole video thing could be 100% fake. I'll believe it if / when I see it. Like Ella SDXL just never dropped.
Testing the comics thing now if I can.
Edit: Nevermind this is rubbish. The "consistent characters" in the comic strips are literally just IPAdapter. There is no special sauce here at all. IF AND ONLY IF they publish the video code, I'll look into this.
Guess what? They won't.
Why does huggingface as a website even exist? Nothing actually works on it
Almost every single thing they have runs. And is already he main source of Open models.
@@ronilevarez901 Literally everything I've tried on it gives me an error of some sort