@@H786... You can offload some layers to RAM, but even with fast DDR5 it will be very slow, and when the context increases you will have to wait for responses for 20 minutes or more.
I haven't gotten lazy with my videos. I'm still fully committed to creating quality content while aiming for efficiency where appropriate. If my recent videos seemed lacking, that wasn't my intent. Let me know what specific improvements you'd like to see, and I'll work on incorporating them in future videos.
Please review Magnum-Mini, I sincerely believe it is worth your time and worth it for people who cannot run the larger magnum. 💛
If you cannot run magnum 72b because of your local hardware, know there are lots of hosting providers that can
Thanks for this video, man, very good. 👍
By the way, As for my experience, lumimaid 70B new version is crazy good. Perhaps better than magnum
Excuse me Sir, How many memory is needed for this one?
@@chrishauer5106 i usually run iq3 xxs, it runs well on 30gb vram.
Very excited to try lumimaid once they make quants for it
Im waiting for llama 3.1 4x8 or 8x8... even better if they release MoA...
try llama 3.1 8b
Одобряем)
How do I run this model though?
Thanks
Can you tell how it compares to CosmosRP? It is the one I currently use the most.
same
is it free ?
Yes, it's open source, but you'll need about 24GB VRAM to run it locally.
@@MustacheAI what do yoou mean need. is it slower on 16gb cards or just completely unplayable.
@@H786... You can offload some layers to RAM, but even with fast DDR5 it will be very slow, and when the context increases you will have to wait for responses for 20 minutes or more.
Athene Llama 3 70b?!?!!?!?!??!!?!?
Man, your videos got so lazy. Almost as lazy as me.
Exactly
I haven't gotten lazy with my videos. I'm still fully committed to creating quality content while aiming for efficiency where appropriate. If my recent videos seemed lacking, that wasn't my intent. Let me know what specific improvements you'd like to see, and I'll work on incorporating them in future videos.
@@MustacheAI Nah your videos are good, at least for me.
He could be making a joke tbh @@MustacheAI
@@MustacheAI You can add how much Vram model need, for example.