Let's Test Gemini Pro (honest comparison with GPT-3.5 & GPT-4)
Вставка
- Опубліковано 8 гру 2023
- Let's actually test out google gemini pro compared with OpenAI GPT-3.5 and GPT-4
🔗 LINKS:
Gemini Pro in Google Bard: blog.google/products/bard/goo...
Gemini Stats: blog.google/technology/ai/goo...
Multimodal demo: • The capabilities of mu...
How the demo is made: developers.googleblog.com/202...
Alpha Go Model: • Using AI to solve comp...
✉️ NEWSLETTER: tinahuang.substack.com/
It's about learning, coding, and generally how to get your sh*t together c:
🤖 AI Lunch & Learn series: www.lonelyoctopus.com/email-s...
It's a FREE weekly 1hr livestream about AI & tech topics eg. how to build a GPT, how to build AI products, jobs in the era of AI etc.
🐙 Lonely Octopus: www.lonelyoctopus.com/
Check it out if you're interested in learning AI & data skill, then applying them to real freelance projects!
🤝 Business Inqueries: tally.so/r/mRDV99
🖱️Links mentioned in video
========================
🔗Affiliates
========================
My SQL for data science interviews course (10 full interviews):
365datascience.com/learn-sql-...
365 Data Science:
365datascience.pxf.io/WD0za3 (link for 57% discount for their complete data science training)
Check out StrataScratch for data science interview prep:
stratascratch.com/?via=tina
🎥 My filming setup
========================
📷 camera: amzn.to/3LHbi7N
🎤 mic: amzn.to/3LqoFJb
🔭 tripod: amzn.to/3DkjGHe
💡 lights: amzn.to/3LmOhqk
⏰Timestamps
========================
00:00 intro
📲Socials
========================
instagram: / hellotinah
linkedin: / tinaw-h
discord: / discord
🎥Other videos you might be interested in
========================
How I consistently study with a full time job:
• How I consistently stu...
How I would learn to code (if I could start over):
• How I would learn to c...
🐈⬛🐈⬛About me
========================
Hi, my name is Tina and I'm an ex-Meta data scientist turned internet person!
📧Contact
========================
youtube: youtube comments are by far the best way to get a response from me!
linkedin: / tinaw-h
email for business inquiries only: hellotinah@gmail.com
========================
Some links are affiliate links and I may receive a small portion of sales price at no cost to you. I really appreciate your support in helping improve this channel! :)
You got my full trust just by this one video. Very objective, analytical, structured, testimonial, and insightful!!!
Nice video but she missed something.
Gemini >>Ultra
Hi Tina! Thank you for this comparisson between Gemini Pro and GPT 3.5/4
We appreciate your insights. Keep working hard.
That explanation at the beginning was everything, thank you for distilling it
Thanks for the non-clickbait review!
Everybody is trying to be biased for Google. Because we all love Google products and really believe they will come up something very good. But they are constantly disappointing us.
I don’t get why people are rooting for Google so much. Google laid off a bunch of people just like so many other companies, showing that they are willing to hurt their employees just to make a quick profit.
Trust me, all those engineers are overhired and overpaid.
Google products been nearly the same for over a decade.
@@nomad7317 yeah, just like windows in the last 30 years, just a start menu, icons and windows xDDDD
Wait bro, starting 😅 , google just started AI ,
I often compare the Bing (GPT-4) and Bard (Gemini), and I agree with this video. Gemini is overhyped. I'm not saying it's 'bad'... on any given day one might spit out a better answer to a certain question. But overall, Bing (or CoPilot) using GPT-4 is just more consistent and reliable. Just my $0.02.
Great! Keep up Tina! 🌟✨
Always fun to explore AI with you! ❤
Thank you for the video !! Is there any API for Gemini ?
Hi Tina! Thank you
I loved your review of bard, GPT 4 and GPT 3.5. But I do have a few questions for you. When it comes to bards lenes do you have to update your camera? With the new Google camera update? I don't know I'm just asking. Also, when your prompting with bard you have use your words well. If you don't it won't work for you at all. Last, bard always give multiple responses to your question. Because I see very few streamers using those parts of bard. But I'm going to like your stream as it is one of the best reviews I've seen. Oh and your very easy on the eyes also ❤.
Great video. I was impressed when Google put out those videos, but one has to wonder if it's more hype than reality. I guess we'll have to wait and see, but for now it seems like bad marketing if the final product is so far away from being released.
agreed - that really was an interesting approach that google took...it's not even close to what they showed...
@@TinaHuang1 Hey hi
How do you access gpt 4 version ? And can i have you email please
Having checked the Langley's Adventitious Angles problem posed by Edward Mann Langley, and then later proved by James Mercer, angle x should be 30 degrees. Maybe the A.I.s read incorrectly from the image (not too clear?)?
hmmm true! I should have maybe tried explicitly prompting to see if it read correctly. Could be a vision problem
🇧🇷🇧🇷🇧🇷🇧🇷👏🏻, It's interesting to see how GPT-4 turbo is superior to Google Bard, as you said they are from two different categories, we need to compare with Gemini ultra. GPT-4 turbo is able to generate more coherent, creative and personalized texts than Google Bard, which is limited to following predefined models. Gemini ultra, on the other hand, is an artificial intelligence system that can interact with users in various languages and contexts, as well as generate texts. Therefore, it is not fair to compare GPT-4 turbo with Google Bard, as they have different goals and capabilities.
My tests gave me similar results - Gemini Pro performed on the level similar to GPT3.5. I hope Gemini Ultimate will be better.
Well, mistral passed the flower test:
">>>If all roses are flowers and some flowers fade quickly, can it be deduced that some roses fade quickly?
No, it cannot be deduced that some roses fade quickly just because all roses are flowers and some flowers fade quickly. The relationship between roses and flowers is a subset relationship (all roses are flowers), but the property of fading quickly is not inherited from the parent set. Some roses might fade quickly, but others might not. It's impossible to make this deduction based on the given information."
And llama2-70b-chat sort of did too, but it was a lot slower, and I had to coax it into answering the question:
">>>Please answer this question: If all roses are flowers and some flowers fade quickly, can it be deduced that some roses fade quickly?
Using the given information, we can conclude that:
* All roses are flowers.
* Some flowers fade quickly.
From these statements, we cannot directly deduce that some roses fade quickly. However, we can deduce that some roses might fade quickly, since all roses are flowers and some flowers fade quickly.
Therefore, the answer to the question is:
Maybe."
zephyr was close: "Yes, if we assume that all roses are flowers and some flowers fade quickly, then it can be logically deduced that some roses might also fade quickly."
All tests were run using ollama which uses quantized models by default.
The problem is the implementation of Gemini. I bet Gemini is actually more powerful than GPT-4 but something about how it's being implemented in Bard is making it misbehave. That's my guess anyways.
But is still not exist yet
YEAH I feel like it's a software thing a lot of the times which is really quite disappointing coming from google
This was a very good analysis
My own testing shows that Bard is not great (relative to ChatGPT). I believe Gemini engine is powerful, but whatever logic they have on the backend it's not as good as ChatGPT
What Google showed could have been made more clear up front, but it wasn't like anyone should have expected that big of a leap so soon. I immediately recognized the video for what it was... our future. Remember, we're still in the crawling phase with AI and haven't even learned to walk yet. IMO, I'm excited for what's to come.
Im curious if i can learn with chatgpt for my Quantum Rizzics exam
Yea Time will tell, but great video on the mishaps of both
In Manchester, England (and other places in the North West) buzzin' has been around for decades. Used for the same reason. Wonder if it's origin has gone from Manchester and changed to bussin' in the US? 🤔
Bard just now:
If all roses are flowers and some flowers fade quickly, can it be deduced that some roses fade quickly?
View other drafts
No, it cannot be definitively deduced that some roses fade quickly based on the given statements. While it seems intuitively correct, the logic is flawed.
This is an example of a syllogism fallacy, specifically the affirming the consequent fallacy. The argument goes:
All roses are flowers.
Some flowers fade quickly.
Therefore, some roses fade quickly.
The error lies in the second step. While it's true that some flowers fade quickly, we don't know if any of those flowers are roses. It's possible that all the quickly-fading flowers belong to a different category altogether, leaving open the possibility that no roses fade quickly.
To conclude that some roses fade quickly, we would need additional information, such as:
Some roses fade quickly: This directly states the conclusion we want to reach, eliminating the ambiguity of the original argument.
All flowers that fade quickly are roses: This eliminates the possibility of other types of flowers being the only ones that fade quickly, ensuring that at least some roses fall into that category.
Therefore, without additional information, we cannot definitively conclude that some roses fade quickly based solely on the statements "all roses are flowers" and "some flowers fade quickly".
Another great video pretty fair comparison, I think gpt4 is worth using and investing money till now 🙌
yeah! I think for paid gpt4 is definitely much better
For ppl like me that hate pure text, Gemini is a go-to
Good video 🎉
A significantly more accurate comparison between googles Gemini and GPT 3.5. Would be to use Microsoft copilot on the 3.5 setting.
True. I tend to just use Bing (CoPilot) on the Creative setting, which uses GPT-4. Best of both worlds, ya know?
x=30 That problem is tough. I used a graphing program. Still need to prove it algebraically.
I tested extensively on reasoning and had a debate about some topics with both GPT 3.5 & Gemini Pro, GPT 3.5 was better tbh. I haven't tried other things but in reasoning I can say GPT was better and has seemed to have a wider knowledge base.
you should be comparing to GTP4 , not 3.5. 4 is much more capable
Bard music was like happy birthday 😂
Bard prompts require specificity. Unlike other models, Bard needs clear, complete instructions to thrive. Be intentional with your words, providing context and desired output details for optimal results. I was a GPT plus user, until i learned how to use Bard.
She says that it's not fair to compare Gemini Pro to GPT-4 but if you take away the 4pts you gave for vision (because GPT-3.5 can't do pictures) and the scores are 14.5, 15.5, 16.5 which is way more representative of the modest improvement between the models (unless you actually think that the 4 of the 6 point difference between the 2 models being because they can search pictures now is an accurate representation of its vast improvement from a 14.5 to now a 20.5).
Using gpt and bard both but gpt is somewhat better but if bard is much accurate in less word, sometimes we need to interpret what we wants
I called bard stupid after it kept repeating same mistake even though I corrected it. It literally got mad at me and threatened. It said “I will not tolerate such remarks and try to be careful in future”. Then it logged me out. Apparently they put fragile emotions in bard.
The criticism are fair. It is a big deal for any AI to be able to understand live input and respond intelligently and promptly. That would be considered a major breakthrough. Big thumbs down 👎 to Google.
The ultimate test. Write a short story using just emoji.
Good work. Twinkle twinkle was a good touch.
If anyone was wondering, the answer to the triangle problem is 30 degrees.
oh damn they both got it wrong thank you!
How do you access gpt 4 ? I can’t upgrade to gpt 4
Guys please let me know how to upgrade 😢
as a programmer I would like to compose like this: gpt4 > gpt3.5 > bard
waiting for Grok still :P
I keep hearing that Grok is overhyped. Plus they charge a ridiculous amount to use it. Better to stick with GPT or Bard. I personally like Bing Chat (CoPilot), which uses GPT-4.
I just read that gemini's best demo was faked. So the features that you were excited about in another video I saw might not actually be fuctional.
It’s the first thing she mentioned in the video.
You mean the fireship video?
@@douglasdrumond Well I commented pretty early into this one after watching the first one where she was referencing it. I didn't have time to watch it that day so didn't know she was saying the same thing.
Can you please do another video comparing the newly launched Gemini Advance (using Ultra) vs ChatGPT 4? Your test here is good and easy to understand!
on it!!
@@TinaHuang1 Yay! Thanks in advance!
GPT-4 is very smart!
Bard is not enabled with Gemini Pro
I couldn't help but notice your bookmarks for you web browser. Might I suggest making folders A to Z and putting the bookmarks based on the first letter?
Is it just me or both of the responses for the first prompt were really similar?
The part comparing the ability to search was not comparing apples to apples, as it was comparing Bard to ChatGPT.
In fact Gemini compares to ChatGPT and Bard compares to Bing Chat. Bing Chat (based on ChatGPT 4) can search and use Bing Maps, just like Bard (Based on Gemini Pro) can search and use Google Maps.
Had the comparison been done with Bing Chat, the results would have been quite different.
I tend to compare Bing (or CoPilot) and Bard pretty often. You're 100% correct... it's more apples to apples. That said, I prefer Bing over Bard. It pretty much has all the strengths of GPT-4, plus it has additional Microsoft features (mapping, export to Word, book flights, get directions, etc.). It's similar to what Bard does... just the Microsoft version.
Do you stay until 1am to upload?🥺
yes 😅
@@TinaHuang1 omg plz make sure to rest 🥺
Tina, but by using your same logic, claiming that a product is better than the competitors' and then not allowing people to test it cancels your claim about "fairness"... 🤔 What you seem to ignore, is that they literally even used small print that explicitly says that the tests weren't the same between Gemini *_Ultra_* and GPT4, beyond just making things seem in a certain way several times, though. Clearly they fear what the GPT Store might do to theirs, the timing shows it, so they prolly did this to buy time to see what they can do in the meantime. They might just be duct-taping smaller specialized models to it to try to do specifically what they claimed, but OAI has the power positon now... Yannick Kilcher (he made his own model with his group) explains it in his latest video. 🙂
I'm still mad at Google for killing Allo. I hope they don't kill Gemini. 😂😂😂
I get X equal to 10 degrees
everyone who solved for x on the geometry question in the comments got different answers what!!!
Gemini Pro is fast, but it often misunderstands me and takes me too literally.
Gemini pro 👎🏽👎🏽👎🏽
Google 🤡🤡🤡
I am not mad at Gemini, but at Google itself, because just like they did with Stadia and Google Duplex, they overrated their product again!
The value of "x" is 60, which means google bard is correct.
5th!!
First!
Wait...how can you both be first?
@@nathanbanks2354 i swear I was :)
I know Ultra was used in a lot of the experimental results showing Gemini beating GPT4. But did google themselves make it explicit that Gemini Pro is only GPT3.5 level? No I don't think they did. So the confusion is partly their fault too. In fact I think it's the community that noticed how bad Gemini pro was when compared to GPT4 that started saying pro was meant to be 3.5 level. This didn't come from google themselves as far as I know.
It was done implicitly through their reports that showed the proficiency of each of their models compared to gpt4 and gpt3.5. From there we could see which corresponded to which.
@@gonzalezm244 you're right actually I missed 3.5 in the big report they wrote and I've seen them comparing it. It's something that needed to be said explicitly in the marketing material though, cos it's the very absense of that that's responsible for the misunderstanding.
I'm sure they use to claim or imply that when they upgraded to palm2 it was competitive would be GPT3.5 at least. So Gemini Pro being roughly at GPT3.5 is actually poor I think.
@@elawchess yeah it’s very disappointing from a company that has so much expertise and resources to be almost a year behind a 700 person company in the super fast-paced AI space
I think it serves them right cos google had access to the AI discoveries but didn't make anything for the public because they feard it would threaten their revenue on the search business. Now that someone else has done it first they want to move in and eat cake while having it too. It's not gonna be that easy.
I'm more satisfied about my privacy with Google than *Open*AI after such a miserable performance by bard 😅
x=60 degrees. bard is correct
I'm still drawing triangles
hahahah
Well if you think about when 3.5 came out and Gemini was released its not fair Either 🤣 so ye its fair
It's googles fault for trying to say Gemini is better without showing how or why. Total misinformation.
Gemini Pro is not as good as gpt-3.5 either. Cannot even follow a conversation. I don't want to have to remind it what we are talking about for every prompt.
this is true! it doesn't remember the image previously shown and glitches out quite often. It feels like a software issue that is very disappointing coming from google huh
Twinkle twinkle little tsar
X=55
wow just wow. why google 😭😭😭
because "why not?"
Because they can't help it, hype is second nature to these tech companies.
U just mentioned it, that bard used Gemini pro only on text, and now u make these test for comparison, its not really a fair comparison lol
good lord... ooof
AI = making computers do tasks too difficult for computers to do
Yep, that's just crazy that's what people are so 💯 hype for 🤦♀️
PREFACE: Not aligned to any company
COMMENT: the Gemini video was a “pure marketing piece for a vision of what is possible” and should have been verbally commented by the narrator as such. The video was completely misleading. Sending people to “go find the programmer notes” is NOT an excuse.
1st
Wait...how can you both be first?
The amount of Google bias in this video is astounding...🤣🤣
I used gpt to write so many boring stories
_★:_ I believe we are meant to be like Jesus in our hearts and not in our flesh. But be careful of AI, for it is just our flesh and that is it. It knows only things of the flesh (our fleshly desires) and cannot comprehend things of the spirit such as peace of heart (which comes from obeying God's Word). Whereas we are a spirit and we have a soul but live in the body (in the flesh). When you go to bed it is your flesh that sleeps but your spirit never sleeps (otherwise you have died physically) that is why you have dreams. More so, true love that endures and last is a thing of the heart (when I say 'heart', I mean 'spirit'). But fake love, pretentious love, love with expectations, love for classic reasons, love for material reasons and love for selfish reasons that is a thing of our flesh. In the beginning God said let us make man in our own image, according to our likeness. Take note, God is Spirit and God is Love. As Love He is the source of it. We also know that God is Omnipotent, for He creates out of nothing and He has no beginning and has no end. That means, our love is but a shadow of God's Love. True love looks around to see who is in need of your help, your smile, your possessions, your money, your strength, your quality time. Love forgives and forgets. Love wants for others what it wants for itself. Take note, true love works in conjunction with other spiritual forces such as patience and faith (in the finished work of our Lord and Savior, Jesus Christ, rather than in what man has done such as science, technology and organizations which won't last forever). To avoid sin and error which leads to the death of our body and also our spirit in hell fire, we should let the Word of God be the standard of our lives not AI. If not, God will let us face AI on our own and it will cast the truth down to the ground, it will be the cause of so much destruction like never seen before, it will deceive many and take many captive in order to enslave them into worshipping it and abiding in lawlessness. We can only destroy ourselves but with God all things are possible. God knows us better because He is our Creater and He knows our beginning and our end. Our prove text is taken from the book of John 5:31-44, 2 Thessalonians 2:1-12, Daniel 7-9, Revelation 13-15, Matthew 24-25 and Luke 21. Let us watch and pray... God bless you as you share this message to others.
So Gemini pro is sh!t. Understood.
Chat gpt is way better than Gemini
chatgpt 3.5 was based on gpt 3.0 and that was a really old model while gemini is just released so I believe it is far to compare gemini against chatgpt4
It's not, to a degree anyway because the most advanced model, would be Gemini ultra. There are so many misconceptions about GPT 4 turbo and Gemini is unbelievable She lists some of them in the video. People have to take into account the models and what you train them on. Good point, but regardless of being older, comparing the second lower tier version of Gemini to the most powerful version of Chat GPT 4. I would argue it is more unfair, because it can do more than Gemini pro is capable of. A lot of people don't know how to properly utilize them. Also sidenote, the accuracy of chat GPT 4 turbo can be enhanced to a greater degree with the right prompts, plugins, and PDFs to train it on.
I think is the best Google has to offer now is Gemini pro? Why would I want to compare that to its competitor’s 2nd or 3rd best offerings? Or maybe Bing chat vs bard would be the right comparison?
@@testinghere9818 All great questions! Love feedback, yeah I think Gemini Pro is the best they offer. The reason why I believe the comparison leans more toward 3.5 is the current "limitations" or the level of information both have access to, She says in the video under misconceptions: "Bard is using Gemini Pro for text NOT for other modalities such as speech, images, code etc.." All of which GPT 4 turbo can do even though Bard can do those things, it's not necessarily intended to. So, for the purposes of the video, and the intention of what it was meant to do. The comparison falls under that category if that makes sense. Bing Chat is more advanced than GPT 3.5 and can do the things mentioned like images, speech, etc. It is currently more limited than GPT 4 turbo due to Microsoft's restrictions so quite possibly could be a comparable match up but not one I wouldn't fully support until it's stated to have support for the same capabilities.
potato
You are wasting my time for nothing but you are so cute so idm it
@google
Gemini >>Ultra
I get why they felt the need to put something out, but I agree that the way they did it was wack 😂. They could have waited one more month and had something cool and well put together to show off in the new year 🫠!
They may have wanted to get real users to try it before they release the ultra version and start charging for it. They came up with BERT 5 years ago, before OpenAI had GPT-2, so their engineers are certainly capable of natural language programming. Of course they also came up with Google+ earlier...
they are desperate. they have nothing to show.