I tested both and in my findings, gpt-4o mini is about DeepSeek-V2 Chat level (which is half the price still). Nemo -12b is about Claude-3-Sonnet level (for 2-3% the price). More options are good, and small efficient models are welcomed, but I really wanna see a few more large flagships.
Native audio/video *input AND output* is huge. Looking forward to that! I'm gonna try 4o-mini in my robot project. Currently using Claude Haiku, but as much as I love Claude, Haiku doesn't do well with function calling.
Interesting. It looks like it is much lower cost, and still maintains 2nd place in MMLU, GPQA, DROP, MGSM, MATH, HumanEval, and MMU, with 3rd place in MathVista... overall, it is very impressive for the cost. (Will definitely try it out). Thanks for the video.
Thanks for the Video bro. One Question, am I correct that Gemini Flash still has that free-price tier, right? Since my workload is high and I was considering if makes sense empty out the free tier from Flash for easier high volume jobs? Am I missing something from here? Thanks!!
Thanks for the update. How many languages does it support? For example does it support turkish, dutch, polish, thai and ...? Good for us, prices will drop to other competitors also like google and anthropic. Hopefully Mistral or Meta will release an open source competitive multimodal llm.
Hahaha agreed, Sam Altman likely did this intentionally! This is indeed cost effective! The funny thing about Gemeni flash context caching is that the content must consist of at least 32,769 tokens. This means that in many real-world cases, like in RAG where the context is fetched from a larger datastore, and it's smaller than 32,769 tokens, you won't experience the caching benefits. This seems more like a sales/marketing constraint rather than a technical one, reminiscent of those pesky terms and conditions with small font sizes that you overlook when purchasing the product.
Could you make a video on what kind of apps one can build on top of these and make money? Also, how can one build apps on these? Also, how one can build skills to deliver requirements for genAI and other LLM based jobs on freelance websites like upwork? I have played tekken. Characters like Haihachi, Yoshimitsu. It also has a movie on primevideo.
Using the gpt-4o-mini API, when I ask, "Which model are you?" I get the reply, "I am based on OpenAI's GPT-3 model, which is designed to..." 🤔 Even in the playground, the same thing happens. However, when I select gpt-4o-mini in the Android app and ask the same question, it says it is based on the gpt-4 architecture. This is very strange! 😱
This isn’t a fair test, OpenAI (for whatever reason) never programs the model to know its own name, which is simply a knowledge issue and has little to do with intelligence.
Why people are talking about Microsoft Phi model. It's jusr 2 B model. Are there any use cases thay can be done via such small models? One use case i found was sentiment analysis & ticket classification. But are there any other use cases? Also can we use 2B modle using my normal laptop with 8GB ram?
![The moment we stopped understanding AI [AlexNet]](/img/n.gif)
I tested both and in my findings, gpt-4o mini is about DeepSeek-V2 Chat level (which is half the price still).
Nemo -12b is about Claude-3-Sonnet level (for 2-3% the price).
More options are good, and small efficient models are welcomed, but I really wanna see a few more large flagships.
Native audio/video *input AND output* is huge. Looking forward to that!
I'm gonna try 4o-mini in my robot project. Currently using Claude Haiku, but as much as I love Claude, Haiku doesn't do well with function calling.
Interesting. It looks like it is much lower cost, and still maintains 2nd place in MMLU, GPQA, DROP, MGSM, MATH, HumanEval, and MMU, with 3rd place in MathVista... overall, it is very impressive for the cost. (Will definitely try it out).
Thanks for the video.
Thanks for the Video bro. One Question, am I correct that Gemini Flash still has that free-price tier, right? Since my workload is high and I was considering if makes sense empty out the free tier from Flash for easier high volume jobs?
Am I missing something from here? Thanks!!
That's absolutely true. Gemini Flash has got a free Tier (with rate limits) but very good to prototype.
Great news video, my dude, thank you!
Thanks for the update. How many languages does it support? For example does it support turkish, dutch, polish, thai and ...? Good for us, prices will drop to other competitors also like google and anthropic. Hopefully Mistral or Meta will release an open source competitive multimodal llm.
Hahaha agreed, Sam Altman likely did this intentionally! This is indeed cost effective! The funny thing about Gemeni flash context caching is that the content must consist of at least 32,769 tokens. This means that in many real-world cases, like in RAG where the context is fetched from a larger datastore, and it's smaller than 32,769 tokens, you won't experience the caching benefits. This seems more like a sales/marketing constraint rather than a technical one, reminiscent of those pesky terms and conditions with small font sizes that you overlook when purchasing the product.
That is a very interesting detail that I might have overlooked as well. Thanks for sharing
Can chatgpt 4o mini browser the internet?? Thanks for the answer
Should I think of the gpt4 mini as a model in between this, better than the gpt4 but less performing than the gpt4o?
Amazing news 🎉
Super amazing 😍
Could you do a comparison bus between this and Gemini flash? Especially for vision. Take on image recognition and data extraction.
Could you make a video on what kind of apps one can build on top of these and make money? Also, how can one build apps on these? Also, how one can build skills to deliver requirements for genAI and other LLM based jobs on freelance websites like upwork?
I have played tekken. Characters like Haihachi, Yoshimitsu. It also has a movie on primevideo.
Using the gpt-4o-mini API, when I ask, "Which model are you?" I get the reply, "I am based on OpenAI's GPT-3 model, which is designed to..." 🤔
Even in the playground, the same thing happens. However, when I select gpt-4o-mini in the Android app and ask the same question, it says it is based on the gpt-4 architecture.
This is very strange! 😱
gpt-4o-mini is very bad in my tests it was performing worst than LLama 3 8B :D
This isn’t a fair test, OpenAI (for whatever reason) never programs the model to know its own name, which is simply a knowledge issue and has little to do with intelligence.
Tekken 7 now i think. used to pay 2 rs per coin for gon!
Exactly that arcade Machine. I don't remember how much I used to put in.
I tried to switch to mini version. It wouldn't let me!!
RIP togetherAI, DeepInfra, OctoAI and all the inferencing as a service providers.
Wiped!
4o mini is still way slower than models running in Groq. I'll stick to Gemma 2 9b on Groq for now, since it's 500 t/s and works well enough.
They still have unmoderated and fine-tuned models. They do still have cheaper models as well, but not as smart at that price.
Why?
Better you test it first ...
wohuu lets go it would be fun now :) to utilise the API like for poor broke people like me or student currently and who have limited budget!❤
@@MichealScott24 absolutely
- gpt 3.5 retired which means for free users they get data-analysis - vision model/rag & web search which would be cool
Yeah and you better use the free api for now, gpt4o mini doesnt let me to put pdf on it in web or app version.
Despite it says its able to process pdf..... and now u can stop using copilot that love to dodge ur question.
Is it going to be on the site
@@serkhetreo2489 that's what they mentioned. But it's not yet
@@1littlecoder okay 👍
Your turn Anthropic
They need a better and more free fine tuning ability (more user friendly) imo.
Why people are talking about Microsoft Phi model. It's jusr 2 B model.
Are there any use cases thay can be done via such small models?
One use case i found was sentiment analysis & ticket classification.
But are there any other use cases?
Also can we use 2B modle using my normal laptop with 8GB ram?
Yes. It will run slow, but yes. An iGPU on windows allocates RAM as vRAM dynamically.
azure openai is so slow