Відео

The 5090 has 32GB not 24GB.

radeon cards? do I need any additional software installed to make them work? thanks..

@@osman2k Nope. No extra software needed, other than the latest drivers of course. It just works.

@@damienthorn1340 that's great, thank you.

No, it is not the original DSR1 7b. He used the distilled version of Qwen in R1. He's not using Ollama, he's using another app.

YES. LOCALLY. No internet. No remote desktop. Compiled Ollama using GO then downloaded the model in termux.

No, it's not combine without link. I've tried with 4070 & 2070, at best you can only use one card on one task and other on other in ComfyUI, it can free some VRAM on main card.

It is lmstudio

Well, then you might want to skip videos that have 5090 on the title. I think 🤔 even basic AI would be able to come up with that solution 😅

@ Yup! Exactly. I am doing that now.

Same only use around 20gb

Why is that an issue?

5090s are better suited at AI workloads than they are at gaming. Like what game even needs anything close to 32GB of vram? 😂 Most games use between 8GB to 12GB, with only a very select few that even use 16GB, which usually involves full path tracing. The 5090 is literally using a binned GB202 die used in AI workstations.

​@@03chrisvanything beyond RTX 3090 is not needed for gaming i think. But for AI you need multi GPU and at least 32GB vram

This card is better at ComfyUI mostly, video generation in Hunyan all the way. There will be no VRAM in these in any future by temperature reason (more dense=more temp), server LRDIMM the dense RAM can already heat to 90 C.

fp16 should give u nice accurancy... u would need 5 5090 to tun 70b locally just inferenz and around 10 5090 if u want to train it... with fp 32 you would need to double the amount of 5090s.... just my 2 cents

So it's possible since there is no nvlink for 5090

oh it's not quantized

@@agush22 lmao

They are hybrids of R1 and other models (either Llama or Qwen depending on the one you download), their weights containing information from both models they were created from. I don't think it is unreasonable to say something like DeepSeek R1 Qwen Distill is a "version of R1," and equally I would not think it is very unreasonable to say it is a "version of Qwen," both statements are true since it's a hybrid of the two. It is being oddly nitpicky to try and fight against this.

@@amihartz sure but it cannot be compared to the real R1, they are not the same model.

You are correct, but 99.9% just can't grasp that the distilled models are qwen or llama. Heck, it even states the arch in this video a such and people still think it's R1. Notice the other one in this thread yapping about it being a hybrid, etc. Sigh.

@ They are objectively not Qwen or Llama, this is easy to prove just by doing the "diff" command between the models, you will see they are different. The models are R1 Qwen Distill and R1 Llama Distill, not Qwen or Llama, nor are they R1. You are spreading provably false misinformation.

@ they are qwen and llama based, yes the weights have been changed but it does not matter. if you do a distance analysis they are very very close.

He might've downloaded the unquantized version.

@@amihartz aahh, yes did not think about that :)

The more the better honestly. But 16 does me really well. Just can't go any higher than the base sizes.

macs have unified memory so the vram is also your system ram, 14b is around 11gb you would only have 5gb left for macos and whatever else you are working on

@@prof2k which parameter are you running right now?

@@agush22 so 24/ 32 gb would be better for running 14B?

@themohammadsayem Honestly it varies and quality doesn't necessarily go up linearly in size. The best local experience has come from Gemma 2b . I've tried bigger models but for conversation Gemma has been better. It's no where near Claude though and tjis make me think about what I'm missing because I can't run greater than 9gb models.

Even a phone can run 1.5-7 b

Same here, but my Ollama Deepseek did not have any problems with the questions either, so wierd that his did not even could answer the strawberry question correct :)

The 50 series as far as I could understand is shit except for very specific AI tasks to not make it feel like shit. I will not buy it though I have the money. Conclusion: it's probably cheap to make.

@@gabrielesilinic It's more about big tech AI demand. They're (or were) in an arms race to secure top end Blackwell GPUs, so obviously Nvidia could mark them up at insane profit, like the manufacturer playing the scalper. Blackwell consumer GPUs are there to crush anything AMD could come up with at CES 2025, and by the bewildered look of AMD's board partners in CES with AMD not launching their cards, I'm pretty sure Nvidia felt mission accomplished with the 5090 and 5080. Of course having it up for review and potentially curb stomping AMD's next Radeon cards is a great thing, but a paper launch is intentional because big tech pays far more for Blackwell AI, and AMD is unable to take advantage of Nvidia's paper launches with their own, relatively uncompetitive products. Previous Radeon cards weren't exactly value leaders, especially when Nvidia offers much better ray tracing and a more polished DLSS/frame gen experience, making Nvidia cards worth a little bit more. And used Turing/Ampere cards are relatively decent value. In addition far fewer retailers stock Radeon cards especially outside the US, and sometimes they mark them up due to their lack of volume sales, so much of the value proposition gets wiped out by higher retailer margins. That leaves AMD only with volume and availability vs. Nvidia's paper launches. But AMD doesn't have much leeway with the fabs, and CPUs are far more profitable for AMD. So Radeon gets second billing in production as AMD would rather not spend too much on volume, because Jensen might suddenly without warning turn on the tap, and flood the market with Blackwell. Because what Jensen wants more than selling GPUs to big tech at scalper prices, is to completely crush AMD at every turn.

$2,000 for one card? that is absurd

@@Avantime what I was saying is that in the 50 GeForce series in terms of raw performance there is a lot of smoke but not so much meat. The issue is that they technically suck for performance, half the frames they make is generated and this doesn't make them good GPUs.

@ It's still a ~30% rasterization uplift, and there's no competition from AMD. They may suck for you but AMD isn't doing any better, and that's what matters for Jensen Huang. Nvidia doesn't really care about the consumer GPU market right now, and they can afford to because neither AMD or Intel are providing the competition the market needs.

Eh the thought box is not what it actually "thinks". It just answers the prompt a first time, and then summarises it into the "real" answer. There is no thinking going on, we know _how_ it works. We just don't really understand why it works so well.

It's still a black box , it's generating it's reasoning the same way got generates final output

@randomseer well before you weren't sure what biases were in play for your final output, now you get a false window into how it came up with what it said, but even that solution has a new black box, we still don't fully understand it's interpretation and biases because the output of it's deepthink process it's still inexplainable

It'll probably trickle back up. It's investor panic by people who aren't really informed about technology and the implications of certain things. I do think NVIDIA is pretty risky though. I think if it takes AI too long to become profitable, people will pull out.

It's not about nvidea consumer GPUs , it's about the fact that it was trained with a lot fewer nvidea GPUs than people expected. The primary reason Nvidea is valued is for the GPUs they use for training

@@randomseer This. If companies are telling investors they need, say, 2 million GPU's to run ChatGPT but an alternative comes along that shows you only need 1/10th of those, well, then the demand for said GPUs might be a lot less than the 2 million.... That and the fact that smaller models that are just as accurate can run on competitors' products (say AMD gpu's or Apple M series stuff) means that demand for Nvidia might actually be even lower. Lower than forcasted demand means with alternatives might mean that the moat Nvidia has is non-existant. Those could be the reasons Nvidia dropped.

new channel

@DebugWithLewis cool

Okay, dip****

🤬🤬🤬🤬

I don't know why people say this, all the models are "real" models, they're just different. It would make more sense to say that it is not the "original" model, because the Distill models were produced by taking things like Llama or Qwen and readjusting their weights based on synthetic data generated from R1, so the weights are a hybrid of the two models (either a hybrid of Qwen+R1 or Llama+R1 depending on which you download), but they are still "real" models, just not the original R1. I don't know what it would even mean to have a "fake" model.

??? So when you train on the output of o1 model suddenly the model becomes o1?? Naw its just qwen2 finetuned via grpo

@ You literally are changing the weights of the model, it is no longer the same model. To claim that a modified qwen2 is literally identical to qwen2 is easily falsified just by running the "diff" command on the two model files. They are different models. If you adjusted qwen2's weights based on the output of o1, it would neither be qwen2 nor o1, but would be a new model that is hybrid between them and would take on characteristics of both, as this literally causes the model to acquire information and properties from o1.