It's HF model card says that it is trained on a smallish set of python code files also limited to few modules. Good to see that, it can be used little more extensively.
Hey King, do you hear about project dígits of nvidia, im trying to know if we can connect 3 of them to run locally a 600 b model parameter like deepseek
@@sinayagubi8805 AI models operate using numerical precision, often adjusted through quantization. Higher precision involves using more bits per value, increasing memory usage and computational demands. By quantizing the model to lower precision, such as reducing from 32-bit to 16-bit or 8-bit representations, you can decrease memory and computational requirements. However, this reduction may lead to a slight decrease in model accuracy, offering a trade-off between resource efficiency and performance. It's important to note that the RTX 4090 does not natively support FP4 precision. Therefore, direct comparisons between a system operating at FP4 precision and the RTX 4090 are not straightforward. Lower precision formats like FP4 can significantly increase computational throughput, but they may also lead to reduced accuracy.
I think it's better to wait for now until some people review the product. It might run deepseek v3 with 3 of them, but we don't know about the speed, like how many token/second it could run etc
I have 4gb nvidia gtx 1650 gpu, may be that's why its not running. BTW thanks for the genuine review. Phi-4 looks so cool, hope that they might release the mini version of this too.
Thanks, I'm running phi4 on a 12 GB 3060. No problems so far. I don't think you need 16GB.
More RAM more extra context length, am I wrong?
You're right, more vram = larger contrxt
@@MrVovsn it's 12 gb of VRAM, not RAM
as for RAM, i have 32 gb and same 12 gb 3060, working great for me.
Itll help though to have a 16g vram..the more vram, the better
@@Ren_Zekta I am sorry, I meant VRAM of course.
It's HF model card says that it is trained on a smallish set of python code files also limited to few modules. Good to see that, it can be used little more extensively.
Thanks for the vid, keen to try Phi-4 :)
what was your GPU? what was the model optimization? what tokens per second did you get?
hi i cant run it on cline it always says i need spesicifation as i see you're using template named test phi 4 app can you please share it?
Does it have the function as the same as composer in cursor?
+1 am also curios
I think so because I saw the compose icon on top but never used it it's just my assumption
Hey King, do you hear about project dígits of nvidia, im trying to know if we can connect 3 of them to run locally a 600 b model parameter like deepseek
btw, it has less flops than a 4090... but I am not sure what "peta flop at fp4 precision" means and what the precision of the 4090 was.
@@sinayagubi8805 AI models operate using numerical precision, often adjusted through quantization. Higher precision involves using more bits per value, increasing memory usage and computational demands. By quantizing the model to lower precision, such as reducing from 32-bit to 16-bit or 8-bit representations, you can decrease memory and computational requirements. However, this reduction may lead to a slight decrease in model accuracy, offering a trade-off between resource efficiency and performance.
It's important to note that the RTX 4090 does not natively support FP4 precision. Therefore, direct comparisons between a system operating at FP4 precision and the RTX 4090 are not straightforward. Lower precision formats like FP4 can significantly increase computational throughput, but they may also lead to reduced accuracy.
I think it's better to wait for now until some people review the product. It might run deepseek v3 with 3 of them, but we don't know about the speed, like how many token/second it could run etc
@@vaingaler5001 if can execute a deepseek v3 model, im sure that it will a success but lest wait
Notnsure if you can cobble together three, he mentioned two in the keynote. Remains to be seen if that's the actual limit
I have 4gb nvidia gtx 1650 gpu, may be that's why its not running. BTW thanks for the genuine review. Phi-4 looks so cool, hope that they might release the mini version of this too.
it'll take at least 11.7 GB ram. You'll need to open other web tabs. So 12 is ok and 16 is optimal
You can run any model on any computer. The only problem is speed. If you're memory is not enough then you can increase your swap
That is pretty cool, haha
Thanks
Can you please playlists based on providers like Deepseek playlist, anthropic playlist, microsoft playlist, etc..
what are the pc requirements for it?
What is the best AI Coder?
Still struggles with Strawberry challenge though
how does it compare to deepseekv3 and claude3.5sonnet?
not at all, its not comparable to both
@ is it better or not as Clause at complex codebases?
please do comparison between langraph vs bee framework
Heh
I have 2019 macbook 16 inch full spec
Can I run this locally?
no
try it and let us know