I Talked To Adobe About Sora, Gen-2, Pika, and the Future of After Effects
Вставка
- Опубліковано 8 чер 2024
- Some big AI News today! Stability.AI has released Stable Diffusion 3! I'll be not only showing you some examples of the images it is generating, but I'll also dig into the documentation to show you some of its upcoming features!
I've also got an interview with Adobe regarding the new Gen AI Video tools that are going to be included in the newest Premiere Update. I'm asking questions you all asked in my last video, including: When will we get Sora?! (Spoilers: no answer yet!)
All that, plus: Microsoft's entry into the AI Avatar Race, and a look at the world's first AI Art Esport Competition!
LINKS:
Adobe/Sora/Gen-2/Pika Look: • Adobe Just Changed The...
StabilityAI: stability.ai/stable-image
Microsoft AI Avatar: www.microsoft.com/en-us/resea...
Curious Refuge: / @curiousrefuge
Chapters:
00:00 - Intro
0:41 - Stable Diffusion 3 is Here!
1:15 - What is new is Stable Diffusion 3?
2:26 - SD3 vs SDXL Shootout
4:53- Stable Assistant Chat
5:25 - Interview with Adobe
7:45 - Adobe Answers Questions
11:30 - Adobe Interview Wrap up
11:55 Microsoft's new AI Avatar
14:43 - AI Esports! - Наука та технологія
Excellent questions asked, Tim! Well done. Many of the answers were evasive, but the questions were knowledgeable and on point. 👏🏼
I was saying before, this was like a first date. Can’t come on too strong, but we’re developing a relationship! Hoping that I can dig a little deeper next time around!
I know they’ve got to be dodgy on some issues. Like, there’s a multimillion dollar ad campaign for a new feature, they’re not about to spill the beans to a random YT’er (ha!), but I’ll keep digging and speculating!
What I’d love is to get to a point with the big boys where I can actually TALK to the right people. I think they’re often shielded from end user feedback. And not to shred them or anything, but give actual useful ideas!
@@TheoreticallyMedia Sounds awesome, Tim. Keep up the brilliant work!
Excited for this! It will definitely motivate me to learn these tools better and keep my subscription. Well done going out there and getting the interview!
Great summary, with insight into what's NOT said. BTW, take a close look at the TEETH in the MS talking heads. On some (and in other demos) you can see the lower teeth, and occasionally the upper teeth, moving around and swapping places. It's an orthodontist's worst nightmare. Also, a note: While your production values are quite excellent, many times I wished your circular head overlay wasn't in the frame. We don't need to see your face all the time, not that there's anything wrong with it. But whenever there's a shot of a control being clicked, or some portion of the subject being discussed, please just let us remember who's talking, and show the graphic without anything obscuring the content. After a while I started wondering if it's a new feature of your production setup. Anyway, these are really useful and intelligent reports -- keep 'em coming!
Ha! Yeah, I'm trying to get better about my big dumb head! Thanks much for the feedback, always appreciate it! As a solo channel, sometimes I get stuck in these routines of production, so it's super valuable to hear critical feedback like this!
SD3 reportedly has "safeguards" which many people thought would be restrictions/filtering to prevent some content. It was never explained what those were; are NSFW content moderated now? or is it something deeper even. SD/SDXL's whole appeal was its ability to create unfiltered content with zero restrictions.
Online, connected to the servers, SD was always restricted. Only running off your GPU can you use it without ANY restrictions whatsoever
Brilliant coverage Tim, as always, so informative and great questions posed to Adobe. And can I say I personally love your LOTR filter. Now get some rest!
Tim, thanks so much for the fast update, even in your sleep-deprived state. That microsoft tool was extraordinary! Things are moving so darn fast, your channel at least gives me a chance of keeping up with the creative AI news. I'm curious if there was anything about AI and script writing at NAB? I know there's NovelCrafter and SudoWrite, and of course just using Claude or ChatGPT, but anything coming down the pike we should know about specifically for screenplays?
Generative ai in Premiere is insane! It was insane in Photoshop and it'll be 100x cooler with video 🤩
I can’t wait to see what we can do with it! Really excited about all the possibilities- and if they’re looking at it like a plugin, I almost wonder if we don’t get a breakout panel with those Pika/Runway controls. Like, motion brush right in Premiere? That would be wild.
@@TheoreticallyMediawhen can we expect to get our character prompts reusable aka generating video with the same character ? I mean that would be the definite game changer.
Saying that a not released yet product is "insane" denotes somebody is an overhyped fanboy or maybe simply a social media agent working for the corp. Fortunately at the current rhythm open source models and tools are growing by the time that adobe is hurriedly able to launch something buggy and plenty of glitches there will be open source tools way ahead of sora. The future of AI is open source and the sooner everybody understands it the better.
@@punchtalestudio LTX Studio.
i reply to my own question. its possible with pixverse amid a lot of stress
Stability $0.07 cost per image is crazy!
Yeah, kind of unsurprising though, considering they've been bleeding money for as long as they have. I love that commitment to Open Sourcing for the future-- but it is clear that didn't work out too well for them.
Just hoping this doesn't evolve into an "Open" AI situation.
Great AI journaling. Thanks again my guy.
Much appreciated!!
thanks for your hard work while being jet-lagged 💟
AI Pictionary with real time GenAI is actually genius! I wonder if we'll see it sold as an upgrade to the board game.
Something that I deperately need is VIDEO OUTPAINTING. I work in a corporate environment, and often get videos recorded in portrait orientation. I would love to be able to expand the video. Would that be possible with the new Adobe feature?
Such a great idea! I'll hammer them with that next time I get a chance. I mean, technically, I guess you can via static outpainting, but that only really works for locked down shots. Real video outpainting? That's a gamechanger for sure!
12:45 man this is getting GOOD, I've noticed it's become very hard knowing what's a real video and not anymore...
I’m saying 6 months before we see an AI avatar where we won’t be able to tell the difference.
Hook that up to a LLM, and virtual girlfriend/boyfriends are going to set the world on fire.
Buckle up, things are about to get really weird! (As if they aren’t already!)
Excellent video! The avatars are...short of insanely real and in that uncomfortable area. So, they're really good! Generated audio in Premiere had me going. Meanwhile, I'm waiting for AI transcribe in Audition. I don't get it! Thanks. 👍
Thanks again Tim for this insightful report. These developments are things to wake up for in the morning. Btw, there's a hitch in my channel and I'm prompted by Google to use this name till they resolve it incl missing profile image.
Video excellence, deeply impressed. 🎥👍
Much appreciated!
@@TheoreticallyMedia You’re welcome.
omgggg i am so excited about SD3, is it going to be free? i mean if i run it locally?
Great! Thanks again 😊
1000%! Thank YOU!
Asked the questions I would have, well done. I agree with other comments, he was evasive, but he has to.
I hope it gets here within the year, but we shall see.
so, to be clear: this is NOT coming from Adobe, but the rumor I heard on the show floor is that the update drops in May. I gotta get all these rumors out now, since Adobe is starting to talk to me about an NDA! Haven't signed it yet though, so I can still say whatever I want!! HAHAHAHA!
@@TheoreticallyMedia May would be outstanding!
Haha! Nodding like I care. You crack me up.
Ted is the WORST! What does he actually do all day anyhow?! Haha
UDIO can generate speech that sounds even better than eleven labs, that combined with vasa1 :O
That Microsoft talking avatar is impressive, but not as great as the EMO lip-sync model. The camera / character rotation control is a cool option though! My first question here would be: Can we upload your own character/ A.I. created photo? And one thing I find strange with these examples (was also the case with that EMO lip-sync program) why are the footage examples all in a square aspect ratio? I will need that in landscape!
Man this is gonna be a good video lol thanks again good sir..what a day! Also just seen Llama 3 too its been actually really good for my use cases (so far).
Other creators: heres SD3 release, lets read it and thats all.
You: heres SD3 release, lets read it + how to actually use it.
This is why i follow you.
SD3 looks like its going to be a beast when the weights are released my god it looks like inpainting, layerdiffusion and got a much better clip maybe the one cascade has (i recall reading something about that?)
I went from being a bit mid on SD3 to being super excited about what it has to offer! Those new features are going to lead to some really interesting ideas and workflows once it hits GenPop!
And haha. Yeah, I’m the guy who reads through the documentation. It’s a sickness!
13:07
Wow.....very impressive.
Right? I bag on it a bit, but I do (overall) think it’s a pretty amazing model!
Creative upscaler! And wow search and replace 🎉
Search and Replace is such a wild idea! And I think-- because it will have to contextually look at the whole image, probably going to be a LOT more accurate-- AND: Masking Image to Image?! That's wild!
New Kaiber update i think with a new photorealism model
Did that go full release? I'll have to dive in for a full Kaiber tutorial revamp! It's been awhile, and my old one is pretty outdated at this point!
(Sigh...with AI, I gotta keep redoing these tutorials like, every 6 months!)
@@TheoreticallyMedia right?! Its so tough to keep up but yes they also said something about improved "wavy" fix
Ted… what a drone…
He’s the worst! Did you know he’s the guy that keeps microwaving fish in the lunchroom?!
@@TheoreticallyMedia Jan says he wraps it in lettuce BEFORE microwaving, too…
2:56 all i want to know is if they have improved finger generation... LOL.
Haha, well 7 fingers works to your advantage if you're a wizard!
12:42 well, it's better than HeyGen or even D-ID from what I can see so far, but the movement makes it obviously AI. Still haven't seen anything come close to EMO (Emote).
I thought EMO looked a little cartoony. Maybe perfection is a lot harder than most ppl think. Still very very early days.
You should have demanded Adobe to update their speed ramp in premiere. its such a small tool with few steps yet its absolutely impossible to understand. it should be intuative. I have to go to Davinci to make a speed ramp then go back and import it again to premiere. insane.
Believe me, there are so many of those older but simple effects that for some reason Adobe has refused to give us simple solutions to. Clip retiiming should be as simple as a click/shift/drag, but instead we get keyframes in a dropdown.
That said, hopefully with what is looking like simplification of tools in PS will eventually lead to the same in PR.
VASA-1 = the video of Mona Lisa doing the Anne Hathaway Paparazzi rap ?!!😂
Ref : "Prompt Engineering Future" page
Oh, I forgot to post links! I’ll hop on that! Yeah that one is hilarious!
@@TheoreticallyMedia
"Prompt Engineering" page posted it on the FB AI Animation Group.....
Now I realize Microsoft created it not them.....😄
Vasa-1 could be coupled with Wondercraft ai, that creates audio synopses of webpages?
wow
why when releasing SDXL, they show great pictures produced by SDXL and now when comparing with SD3, the images shown as produced by SDXL are only the crappy ones ...
Haha, MARKETING! Agreed though, SDXL is still very good. I always say there are use cases for all the models. Like, I LOVE Midjourney v.4-- Does it look dated? Yup, but it also has this gritty surreal quality that I love.
I can generate in v4, then take it to a creative upscaler to get results I'd never get in v6.
Every Image Generator is a special snowflake! (It's mostly why I don't do "vs" videos anymore!)
Those talking heads are still stuck in the Uncanny Valley. Creepy stuff.
Agreed. Also, I want to go watch fellowship of the rings again.
…but I should sleep.
Hmmmm, maybe I’ll do both!
Adobe really has to step up for After effects. its such a cool program but has been neglected for years.
Did you see the interview I did in the last video where I asked them about AE?
While I don’t think they’re getting rid of it anytime soon, my thought is that we’ll be seeing more AE type things ported over to PR.
BUT: there is a lot of 3d stuff happening in the Adobe Research side, so I expect AE to get a big update…in a few years?
Sadly, it does not feel like AE is a priority for them right now.
Oh, sorry- this WAS the video with the interview! Haha. I thought your comment was in the PS/Firefly update video!
@@TheoreticallyMedia :) I have lots of things to say about PS too. I use almost all of adobes apps on a daily but im ranting out to the ether so no worries.
@@Experternas Haha, spoken like a true Adobe user. For years I've said I was going to ditch Premiere/PS/etc and use the alternatives. It's been so long now that I've pretty much given up...sigh.
@@TheoreticallyMedia I've used Adobes apps since the 90s even though back then Macromedia owned Flash and Dreamweaver and other stuff, so Adobe probably only had Photoshop and Acrobat. However being diverse is best. I abandon any app if something better comes out. Davinci is a superior alternative to Premiere in most aspects.
Anyways, All hail AI, its super fun. Keep up the good work.
For anyone wondering when they might be able to get access to VASA-1 I have bad news:
How to Use VASA-1 Now - Is It Possible Or Not?
Despite its potential, Microsoft is cautious about VASA-1's capacity for creating deepfakes and has no plans for a public release. Instead, they aim to use this technology to develop interactive virtual characters and improve deepfake detection tools, underlining their commitment to responsible Microsoft AI technology development.
That’s kind of unsurprising. That said, have you seen the new HeyGen model? It’s pretty remarkable.
I still want VASA for MSTeams meetings! It’d makes some calls far less painful for me!
@@TheoreticallyMedia I've not seen any of the HeyGen updates since it was first released, was really impressed though. I've mostly stuck to Sadtalker as I can run it locally for free, hoping one of these other models will see a release. The ability to control the output with VASA-1 would have been game changing.
@@Kenb3d1 That's one of the craziest things I've seen in AI Avatars yet! On the plus side, now that we've seen it can be done: We'll be seeing it pop up somewhere else! Also, the fact that these Avatars are being created from one-shots? That's wild.
I would have asked better Questions than that like are you going to merge AI into Adobe 3d, like what Blender is doing . Are we going to have different styles for generation, like photorealist or cartoon animation and what is the price point for all this
We’re moving slow here, haha- it’s like a first date! I’ll put your questions in a doc for some follow ups with them!
: )
Thank you!!
Hope Adobe buys Cinema 4D and Zbrush.
Haha, I’ll pass it along! Not that I think they’ll make a decision like that based off a comment, but you never know!
With all these AE tools coming into PR, I do still wonder what the plan is for AE…I could see more 3D modeling stuff integrated in.
I use my custom Chat GPT that generate images and this are far away from Dalle 3
Problem with Dalle 3 is content guidelines and cost. SD3 can run locally.
@@HistoryIsAbsurd yes. well I am working with my custom chatGPT.. sothe images are very good. MJ can get better ones, but not exactly what I prompt. as for SD . yes, but I dont have 2000 usd to purchase a video card
How can I make money using AI! 🤔
The question on everyone's mind!
Digital communication is now totally compromised. The gov will have access to vesa. Face to face meetings making a comeback to ensure Comms authenticity. Well until we get nexus 6, like next year.
Haha, I'm actually all for in-person meetings. As easy as virtual meets have made scheduling, I also think you lose a lot of communication in them. Also, let's face it: There are TOO many Zoom meetings that should have been an email!
As far as the Nexus-6, seriously: if the first real android comes off the line and ISN'T named Roy Batty, I'm gonna be really mad!
@@TheoreticallyMedia I'm waiting for Pris.
Adobe guy said nothing, just flapping his gums in the wind
There was the bit about generative sound. I think if you read between the lines, you can kind of see where they are right now. They’ve entering very cautiously, and there is still a lot of road to pave.
Real caution is not showing anything til it's prime time. There's tension in the hype and managerial mush. If it lives up to hype why not paint ambitious/audacious image of future or let it speak for self? Is this as amazing as it seems or a small upgrade in convenience instead of something revolutionary?
@@watsonhartsoealso runway and pika dont cut it using real footage? Too much morphing.
It's only sora we are interested in.
The adobe guy would make a great politician 😂😂😂
Zed's dead, baby. Zed's dead.
Haha, SAY WHAT AGAIN!!
Hmm these samples of SD3 arent that impressive, maybe midjourney v4 ish
once the model weights are released and the public can start playing with it, we'll start to see what the model can really do. It always takes the elbow grease of the community to jazz those base models up!
Re: gen AI inside, premiere. Thats twice now that they use a vague description in regards to 3rd party integrations. Almost sounds like wishful thinking vs real partnerships
@@BabylonBaller well, I've seen with Pika Socials they're flexing and showcasing-- so, that's one of the three onboard for sure! Sora is a big question IMO. That can easily fail to materialize on OpenAI's whim.
@@TheoreticallyMedia Ahhh so its not all smoke and mirrors.. Lets see where this goes then!
I am REALLY pissed at Midjourney. I created images of a Politician that I will not name before Midjourney changed their policies. I was a PAID member and created the image. Yet when I tried to download it they warned me that I was violating their rules!
First, do we or do we NOT own our images we create on Midjourney if we are paying members? Because if the answer is we do not own them, then I am stopping wasting my time with them.
Second, what the HELL are they so afraid about images of politicians?!!!!!!! Have they not heard of satire that has been used on all sides of politic throughout our ENTIRE history?
Third, futuristic stuff should be hard.
But WHY can't the AI engines do military history correctly in terms of uniforms, weapons and accoutrements? I dare you to see the AI produce an historically accurate weapon!!
I think it is stupid LEFTIEs who think if you can ignore history of war it will go away. When the opposite is true. Those who do not learn from history are doomed to repeat it. And there is so MUCH historical references online for the AI to learn from, it has to be a decision by WOKE Karens who work in these organizations!
Would love to see a video from you to address the military historical issues!
I was with you up until “stupid LEFTIEs.”
The only thing I can trust right now is software I can download to my own computer and use on my own resources without an Internet connection or telemetry.
You couldn’t download an image you previously generated? That seems kind of silly.
I’d agree with the other comment, I’d caution to tone down on the political rhetoric. You are raising valid issues, but you lose half your listeners when you go that angle.
But again, you’re raising valid points. Even the historically accurate image generations. I know the concern there is that people will start generating false history narratives, but at the same time- bad actors on that front can already do that with just a little extra work in Photoshop.
I do think we’re just in an era of growing pains. One of the things a number of companies are trying to get going in the content aware detection system. Basically, a pixel identification that says “this images is AI generated” but cannot be seen by the naked eye.
I think that once that’s put in place, you’ll see a lot of those guardrails slowly being lifted.
Thanks a lot. You're out of focus in the view :)
those demo prompts are garbage... not a serious way to tell how good SD3 is
Eh, it’s a prompt. At the very least a good baseline indication of how the model compares to SDXL.