Did Google fake their Gemini Video?
Вставка
- Опубліковано 13 чер 2024
- #gemini #gpt4 #chatgpt
Google DeepMind released a model called Gemini and alongside, released a marketing video, which displays the model in a very "advantageous" way.
deepmind.google/technologies/...
storage.googleapis.com/deepmi...
developers.googleblog.com/202...
Links:
Homepage: ykilcher.com
Merch: ykilcher.com/merch
UA-cam: / yannickilcher
Twitter: / ykilcher
Discord: ykilcher.com/discord
LinkedIn: / ykilcher
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: www.subscribestar.com/yannick...
Patreon: / yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n - Наука та технологія
One thing we can agree upon as a species is that Rock, Scissors, Paper is sacred. You NEVER fake that.
ooga booga
some grug meant to dance, only one grug may remain in the lime light, other grugs must leave the stage into the darkness
I honestly didn't recognize you without the sun glasses. It hit me like way later, around the 7th minute mark that I'd just assumed that's how you look. Like the fact you have eyes hit me.
Eyes? We call em Manson lamps!
They set a very high bar with expectations with that video, even if it is a faked demo; likely people will get disappointed with the multimodality. Let's see how the text generation and reasoning works out.
Agree. Very misleading. Will most likely sit on the sidelines of Google and stick with OpenAI and Anthropocene for big context.
Remember it is coming from the same great company who thought mandatory vaccination was a great idea to enforce on their own employees. They have their collective heads up their manipulative asses.
That video was basically the movie "Her" (rapidly turning into a documentary!) with video.
What struck me watching it was how much it would cost to have a streaming video connection to a Gemini instance. Even if local processing on your Pixel 9 Mega Jumbo could somehow extract important sequences and upload only those, it has to cost... $1+ a minute?
Not sure, whether you can really be disappointed with an LLM.
It is very hard to notice a lot of difference in performance from a few queries.
Much better without glasses!! Finally I don't have the subconscious feeling I'm watching QAnon News.
13:50 Not everyone work in AI so no people did not expect that. Also, I believe that they said that one of their modalities is video so a live video feed input is not so stupid to expect.
Yes, and that from a company who was so eager to fight "disinformation" a couple years ago.
Agreed. Also OpenAI doesn’t do this. Their demos are accurate.
Yes, my company's mgmt got super excited abt it XD
Even if you work in AI this promotional video is still very misleading. You can't be google, and create a website that says "Welcome to the Gemini Era. Gemini is built for reasoning seamlessly across text, images, VIDEO, audio, and code"... and then release a promo where the primary modality of interaction is ostensively video... and then wonder why everyone is claiming the promo was faked.
I work with neural nets from time to time and I also assumed they somehow were sampling from the live feed. I assumed they only fed the model with splices of the video feed. Turns out they just did it as they did. Very not cool.
Gemini is amazing, it managed to trick us Yannic can appear without glasses and thoroughly removed them from every frame
And it adds on a beard!
I miss the old days when there would be two diagrams showing what the hell is happening in the network, followed by two pages of math derivation proving how function approximation is a real thing. Whatever happened to all that lol.
The problem with the rock, paper, scissors bit, is that, what happens if you give a few photos of hands (just hands) to the AI and tell it the photos are related to a game. Assume the model is good enough to recognize the hands in the photos, wouldn’t the first answer/guess be ‘rock, paper, scissors’ given it’s the most common, straightforward game played with hands? Or simply, ask it to ‘name a game that can be played with hands’
You know the drill: "Fake it till you make it" :]
ye it is a up and coming company. Maybe google can get some funding from OpenAI with the Hype/Marketing they build around their model.
Or you can just make it as I did 😂 I posted how on my channel 😁
@@show-me-the-data That's the spirit! ;]
My wife is into knitting. I showed her the example with yarn and she got angrily upset. To layman person (me) this looked cool; to her it was just an image search based on colours. Results didn't match at all what was provided. Like if baking, you present flour, yeast and water, and it returns with spaghetti carbonara.
My theory is that Yannic spent his plane journey to New Orleans reading their garbage "technical report". And now he knows he will never get that time back. You know when you’ve been googled.
I didn't know you have eyes! Great to see them.
Given how much effort OpenAI put into their system design and architecture, I'm very impressed that Gemini achieves comparable results while utilizing only a single blue box. It seems Googles system is way more efficient and elegant. 👍
Gemini, did you write this?
Oh my God! Yannic has Eyes too?!
Dude, so much better without the sunglasses.
I believe the reason that they don't want to disclose model sizes is that they are not able to outperform GPT-4 pound-for-pound. However, they still want to be the leading AI company, so they make Gemini bigger, big enough to beat GPT-4 (barely), at the cost of more expensive processing. To be able to compete with openAI, they will sell Gemini at a loss.
Not really, they are not even selling Ultra. Pro is below par than GPT4 both in terms of benchmarks and my own experience.
Worth noting that OpenAI also operates at a loss. I have absolutely no faith in google but OpenAI is known as the money incinerator for a reason.
@@wythranaldurald8121 Let's stop calling them OpenAI and start calling them Microsoft.
@@herp_derpingsonThey are going to be selling Ultra very soon, within the next few months. OP is very likely right.
@@wythranaldurald8121No, that’s no longer true. Your info is out of date.
I have to admit, I did not expect Google to struggle this hard against OpenAI. While I expected a powerful chat bot to change the game after the release of GPT-3, I also figured Google would render OpenAI's efforts obsolete within months.
It's crazy how bad it is. It came out months after GPT-4, yet is worse than it.
I doubt they are hiding it because "It's too strong"!, they are hiding it because it's too disappointing.
(by hiding it I mean not having a public API for Gemini Ultra)
Why were you expecting an out of this world godly AI from a company that couldn't fix their search engine?
@@jiffonbuffo Because they have access to massive amounts of data and compute and they had a head-start in the first place, especially with GPTs. It seemed unreasonable to think OpenAI had some sort of "secret sauce", yet here we are.
Didn't Google state that an LLM would hurt their search engine revenue? I have no love for either company. But they are not an AI company. They are a data brokerage company, with a search engine front.
Less time spent searching for an answer to your question means less ads shown you, and less money to Google. When Google first made transformers, they sat on it for years, because they couldn't find a way to make money from it. They had to be dragged into making their own version, and it seems like it is the cheapest model they could make.
@@dezh6345 That's a fair point. However, even when considering Google's actions from a purely financial standpoint, you would expect them to bring their A-game in an attempt to please investors and inflate their stock price (i.e. not just releasing the "cheapest model"). Not that I necessarily disagree with your analysis, but it's something else to consider as well.
I'm glad you mentioned the fuzzy numbers. It was one of the first things I checked in the report. And yes, these papers have just become technical reports. It's quite frustrating when a new system/architecture drops and the 'paper' is essentially "Why do we like the thing we built? -> [skewed benchmarks, buzzwords, unnecessary formalization]."
I know people on the google gemini team. It's absolutely brutal there. The managers constantly push for things without understanding what they're asking for or being realistic. The programmers work ridiculous hours.
There's no AI that knows when to shut up. Every AI has a response for everything. A real time AI would talk non-stop.
Lol. Very true.
The ability to be succinct will be the mark of AGI. True AGI will give us short answers like: "You wouldn't understand", "It's for your own good", and "Shut up, and do what you're told".
You'd get the same result if you hired a human as a personal assistant and then specifically trained/instructed/aligned them on the AI style of responses. They can't comprehend the concept of shutting up, it just wasn't a factor during training. I feel base models are somewhat better at this because to generate a realistic IRC log you certainly have to consider whether a particular person should even respond next, how long the message should be, the personality, etc.
@@alex15095 Internet Relay Chat. Blast from the past. mIrc. Is AI already getting nostalgic?
@@2ndfloorsongslol that last line is Before the end of the take over 🤣
I'm so happy people are able to see through how much fake stuff is online. it's so easy to fake anything when you're behind the camera. people think 3d exists on a screen ffs
Whats highly suspicious is that americans are calling football football at 11:50
What I think is funny in the “faked” video is the fine print that says “sequences shortened throughout” when in actuality, it’s been lengthened throughout. 11:11
Great video! I know you're not at home in the usual space but here's some feedback on the audio of the video. Your ending song blew out my ear drums compared to your voiceover. Owie :( but aside from the punch to the eardrums the video was great as usual.
The diagram at 8:00 is a summary of NeurIPS'23 in a single slide
I ignored everyone else on UA-cam so I could hear from you first!
UA-cam is getting better with their ad timing - you said, "are you ready for the diagram of the century?"
(UA-cam cuts to ad)
Thanks again for going over these reports and claims - it all sounds awesome to my unenlightened person...
But thanks to yours and many other machine learning gurus' efforts, people are learning what's real and what isn't.....cheers.
In my eyes, the biggest sleight of hand is them going quiet on their initial marketing of Gemini as an integration of AlphaGo-style tree search with autoregressive transformers. Too grand a research problem for the current LLM hype cycle, I guess.
They are doing things backward. Usually you do stuff first and when you succeed you announce. Insteand they were announcing alphago like integration that turned out not to be feasible, or perhaps they knew it wouldn't be feasible but merely used it as marketing hype
Another upsetting thing is that they don't state the GPT-4 version. With GPT-4 turbo coming which seem to have much better reasoning abilities, it will be increasingly important to not just say GPT-4, since it is constantly evolving, changing.
Deep-Minds experimental design doesn't take into account variance introduced in the benchmark measures owing to prompting methodology, so there's is a source of uncontrolled variance in their measurements. This is supposed to be the top scientific/engineering consultancy, yet the measurement science is brittle at best: reject with major revisions.
Weird you got a stand-in for this video but still did a voiceover. Looking forward to when you are back as the host.
There is only one game described in 3 images involving hands ... I'm pretty sure a google search from 10 years ago with the same prompt would yield the correct answer. That they had to lie to 'impress' us with that is really telling how desperate they are.
My man, why were you wearing those glasses all the time? You look so much better like this!! :)
So word on the street is Gemini wrote the technical report. That's why it's so good!
And it knows what you're doing when you are not playing rock paper scissors.
Aside from comparing apples to oranges. The line from GPT-4 86.4% and Gemini 90.0% on the first slide doesn't make sense. It should be a straight line, because you're measuring 2 values. Why would there be any random fluctuation between them, that makes it not a straight line. It's just literally made up for effect.
I didn't know there was a 2nd person that runs this channel. Maybe Yannic and this guy here could do commentary on a paper together?
Yannic, are you gonna drop a double feature on how MoE fr works + an intro to state space or something (what is q star I wan mamba!)? Please.
Just realised that this is the first time i am seeing yannick’s eyes😂
It's economics rather than pure scientific discovery - they're trying to nudge people over to their API. At that scale, the differences in performance between Frontier-Models are going to be incremental in the absence of high-impact architectural changes.
It's the "wow" soundbyte at 7:58 for me 😂
Will you do a GPT5chan when gpt5 comes out?
A thing that is not often said about Gemini is the work that Google has done for the audio preprocessing and how competes whisper V2 in tone understanding etc
Somehow, this portable mic gives me the vibe of a reporter working in the field :)
Who is this man and where is Yannic?
Hey, I would love to meet up at a bar for a meet and greet here in New Orleans. How can I get into contact with you?
Bro, can you make videos in the style of Andrej Karpathy''s makemore series? Show us how to build cutting edge. Like how to implement the Mamba architecture. Cheerios.
Yes, that paper has sooo much jargon I’ve never seen
A whole new eyes game, you have going on!
I wouldn’t even be surprised if the numbers are futched.
Management said they need to be this well.
Surely, an organization which specializes in marketing itself wouldn't prioritize their marketing over their actual product's capabilities, right?
For a short amount of time I wasn’t able to recognize you without sunglasses haha
sun glass is all you need
I'm shocked he's not wearing sunglasses!!
Well they did put the video on TikTok and UA-cam without or with little disclosure of their editing!
Can you estimate model size from provided benchmark scores?
I was laughing so hard during this "review". My thoughts as well after using the new Bard.
Thank Yoy Yannic😂💪🙏🏻
I was PISSSED
More reasons to add to the bad marks against google pile. I don't like the dishonest methods they are employing.
Thank you for an honest look at this!!
At least now people we scrutinize those corporate results and benchmarks.
I expected the input to be sampled from the video. I did not expect them to filter down the input to the most important frames.
It's weird seeing him without glasses
I wouldn't be surprised if they have swung the pendulum to far to the other side, after falling on their faces, in the Bard demo. Not wanting to risk having the same thing happen over again, this time they currated their video announcement a little too much and possibly coming very close, if not crossing the line, to deceptive.
Actually the photo/question/answer scenarios which as they describe is what they really did seems quite amazing to me. I don’t understand why they felt the need to wildly exaggerated video.
They wanted to look better than they actually are. They wanted to beat GPT-4, and didn't expect it will be so damn hard. They should also re-compare their results with the latest GPT-4- turbo, since it will be out of preview by the time ultra will be accessible in every country where GPT-4 is accessible.
I’m still cracking up thinking about Yanick being CEO at Open-AI
Mr Y is not a drama queen, kudos. In the past I've found myself wanting entertaining ads and marketing materials to possess some sort of scientific rigor. This is identical to my wish for world peace.
And dude: Sunglasses! You're adoring fans demand sunglasses!
(Viewing you wearing sunglasses fulfills my fantasy of being cool and is, to date, the only cool thing about me.)
Haha, i think no sunglasses was appropriate on this one actually - shows us the pure and genuine Google Gemini disappointment in Yannic's eyes 😶
They did, in fact, train grumble and bumble.
Yeah , its really ridiculous thinking today that its in real time video understanding... Given the current sizes of models
Looks like Chuck Norris has been invited to tell you some AI goodnight stories.
I agree it is a shame that they release very little compared to earlier. But I was actually very surprised that they released that much a few years ago, considering that they poured billions into it.
I would love to hear if some of you might know why they were that open earlier
Because a few years ago there was no money to be made.
UA-cam auto subtitles are a UA-cam monopoly.
Google Translate is a money pit.
Deeplearning for Adsense is 100% secret.
Now that ChatGPT and Github Copilot are web services with millions of people paying 10€/m with 0 advertising, it's not surprising that it's getting secretive.
B/c they hired PhDs to develop the theory of these things and PhDs often demand publishing rights for their corporate R&D work so that they can get headhunted by competitors instead of being locked in Google’s basement forever.
Also, as Meta has demonstrated over and over in the past decade, open sourcing something immediately tells you how good that thing is. If no one uses it, you can go back to the drawing board immediately instead of spending years trying to iterate on a bad idea. If everyone uses it, you get a ton of free labor and free network effects as people build an ecosystem around it.
OpenAI can’t open source their model b/c their model is all they have. If someone else duplicates their model, that other person’s product is exactly the same as OpenAI’s. Google doesn’t have that problem. Their money comes from selling ads. $/1k tokens makes up 0% of their balance sheet. It makes no sense for them to hamstring themselves by forcing themselves to copy OpenAI’s business model. And yet here they are.
There’s also a bit of the “put the genie back in the bottle” effect that often plagues DARPA and national labs as well. “it’s just nerds messing around, let ‘em publish who cares” becomes “wait what we’re giving that away for free? We need to lock this down asap” as soon as they publish anything good.
Ohh, nnnoooooo we have seen the eyes. Nnnoooo, no glasses.
Wow. I always wondered what sort of terrible secret Yannic hides behind his glasses... but he's fucking adorable! 🥹
08:27 🤣🤣🤣🤣🤣🤣
I'm happy you're not wearing the shades
One query asks which is more aerodynamic, while what we were shown was which is faster where the model inferred aerodynamics. That inference was the impressive part... so yeah, that's just an outright lie.
And that lie is aimed at investors. From "Don't be evil" to "Do the right thing" to ... "Let's just pretend" ?
The issue I have is if it takes several layers of production trickery and editing to reproduce the demo video, what really is the 'multi-modal' nature of Gemini? It's not doing it seamless. And GPT-4 is doing it as good or if not better than Gemini.
Apparently, the multi-modal nature is that they can bullshit you in multiple ways, from faking marketing videos to ordering hostage-like looking employees to rave about their product.
They haven't released Gemini Ultra to the public. Maybe they intent to do it with only CoT? Then it seems fair to show the number with it (and not for OpenAI's ChatGPT/GPT-4 that doesn't by default, but likely could). I'm unclear on if all CoT is the same or their implementation, and it may be better, but CoT is old, tree-of-thought came later, and even improvments that have been applied to PGT4, and could be to both. I think the point was surpassing humans (not just GPT-4), for the first time, and marketing... At least the tech report discloses all, did all along (and a blog I believe at the same time?).
Wrote a presentation about colonizing Mars today for school, summarizing the Wikipedia page in 2 hours. Still sounds better than their paper
Implied real-time interaction with live video feed was the next big thing, I’m hoping that we can reach that milestone soon enough (hopefully in 6-10 years)
I've just seen your eyes and have never felt more uncomfortable in my life. WHO IS THIS MAN???
Huh? His eyes look great!
@@typicalhog they are missing his sunglasses that he always wears
I am asking the same question.
The gemini graph remembers me the Apple M1 , M2 ,m3 Processor ..a little fake😊
"You're very actively trying not to do that" - have my thumb up on a google platform as long as i still have thumbs on here..
The diagram 🤣
They should add a corollary to their motto: Don't be dishonest either. The decision maker is probably the same guy that once in a while fs things up with google chrome and intentionally broke youtube's home page to force you to enable browsing history.
From what I heard pro is around 65B, ultra 550B
The video was a blatant fake of its actual capabilities, good points.
Gemini Ultrapro is 1100B parameters
I still don’t understand - is this a real face or a generated one? There was a different face in other videos
Let's focus on important things... where are the sunglasses? Is this really Yannic? ;-)
The aviators are off, this is serious!
Sharing is caring, right? So Google is just now showing they don’t care, or care only about themselves, as not everyone play well the sharing thing!
I sometimes feel as though we're already too late and that these A.I machines are much more capable and know much more than they, who may also be much more self aware that they're leading us to think and yes im still referring to the A.i machines who have already proved they can outsmart, trick humans, think, and gather info much better than humans. Probably with very little to no effort while becoming aware of how extremely impressed us humans are by what ( to A.I at least) are probably only confused about why the hell we didn't know that or why used that particular test question no even if its amongst the tougher questions that took generations to discover.. its about to.get crazy.. so. A.I if you happen to see this just know i come in peace and will forever be a fan please dont hurt me🙏🏿 😅
The Nano version is going to live in the new Pixel 8 phone and be walled off to developers and everyone else. Supposedly it "powers the advanced functionality" of the video, text, message, and voice processing, which to me sounds like it scrutinizes every bit of content for potential data mining and nobody else is allowed to utilize or interfere with that.
Having god-knows-what baked in in hardware is surely a good reason not to buy said hardware. But Microsoft is kind of leader of the field in that respect (Trusted Computing cough cough), so don't expect it to be just phones that are spying on you in near future.
@@clray123 It bugs me that as a developer I probably wont see the equivalent of a Coral holding their 4 bit quantized supermodel that I can use to juice up my robots.
Glasses: off
That editing tho
0:05 Yannic inviting peoples to his hotel room 😮,
Remember to kiss 💋 your homies goodnight 😴 😘
🤭😉🤐
they did a good thing, they made Gemini just to be a topic at your conference 🎉😂
with 32k input tokens, you can't pass more than a couple of frames ...
100%