OpenELM vs Phi-3-mini - Apple vs Microsoft: Who reigns supreme in the battle of small LLMS?

The world of LLMs move really fast, yesterday only we compared Llama 3 8B vs Phi 3 mini and today Apple has launched their LLMs under the name OpenELM.
In this video, we go over how to set up running OpenELM using HuggingFace and ask a series of questions about Phi-3-mini and the OpenELM 1.1B Instruct model to see who comes out on top in this battle of small LLMs.
The link to collab notebook can be found in the article linked to this video -
mlexplained.blog/2024/04/24/a...
Link to OpenELM model cards - huggingface.co/apple/OpenELM

[@GotoRing0]

Why didn't you pick openELM 3B to compare? It would be still smaller than PHI-3. Here we saw how 4 times difference in size looks like :-(

[@ml-explained]

There were some issues in loading 3B model. I'll do some more comparison videos with 3B model as well later.

[@user-en4ek6xt6w]

Good video thanks you for adding for question and a better sound. You would love to see a comparaison with phi-3 and Llama 3 8B on RAG because i already know that althought they claim to be better than Llama 3 that not true I've tested it

[@ml-explained]

Thanks, also please keep up with the feedback, it helps me improve the videos.
Next I'll mostly do OpenELM 3B , but tests on RAG will follow :)

[@user-en4ek6xt6w]

@@ml-explained Ok thanks you, for now the video was more enjoyable the only think that lack is the editing but all good

[@erikjohnson9112]

The bat & ball question results in a range, not a single answer. The bat could be 1.10 and the ball is free (0.0). And the upper range is 1.00 for bat and 0.10 for ball (also valid). Any value in between is also valid, so there is not a single answer without more constraints in the question. UPDATE: I was wrong, I did not maintain the 1.0 difference.

[@ml-explained]

I'll check on it, but it's more to see the approach that LLMs take.

[@13thxenos]

I think you misunderstood the question.
There are two values that we know the sum of, and their difference. You can only have 1 valid answer or no answer at all if you consider real numbers, not a range.
In your examples, if you put the price of the bat 1.10 and then the ball could not be free, because then their difference is not 1, it would be 1.10. If you put 1 for the bat, you can't have 0.10 for the ball because then their difference is not 1, it is 0.90. So both your answers are not valid ones based on the constraints of the question.

[@erikjohnson9112]

@@13thxenos You are correct, I forgot to maintain 1.0 difference. A simple system of 2 equations, b-a=1.0 & b+a=1.10 will produce the correct answer when added together to produce b (then substitute for a). My sloppy natural intelligence failed. :)

[@erikjohnson9112]

I'm leaving my original comment with a correction note at the end. Having comments on a video helps the video author and it shows how even humans can sometimes fail these tests if sampled at the wrong time.

[@ControlProblem]

Good video! But the pronunciation is like "Feye" as in "Fight". Not like "Fee". But overall the content was helpful.

[@KarlLew]

Alpha Phi objects to this comment