Відео

Manim

No, actually it is 1680/0.63=2600x

Glad you liked the content. Unfortunately I can't help you out. Thanks a lot for leaving the comment and goodluck with your project 😃

Thanks a lot. My source code will always be open source 😃

Well that`s cause nvidias hardware is the most used for practical applications . You suggest he makes 4h long video using all these products?

@@lizardking640 I suggested to use OpenCL.

Thanks a lot. Glad you liked it 😃

There's a small typo. I forgot to change the A inside the function to d_A. Thanks a lot for catching that, I completely missed that while writing the animation code. However, in the code repo, it's correct.

Thanks a lot 😃

Thanks a lot. Glad you found it useful 😃

Thanks a lot for the comment 😃 I'm in this for the long run.

@@0mean1sigma very glad to hear that! I'll be one of your long-time fans, haha

Thanks a lot 😀…. Next video is on tiling! I’m more excited to share that one. When I first learned tiling (~3 years ago), it was confusing and took me a long time to get the hang of it. I’ve always felt that HPC concepts go well with animations so I’m trying to do that using this channel.

Thanks 😃

Yes you're right in the sense that the sequential program is not optimised. However, I haven't optimised the GPU program as well. Also, what I often say is that we need to use both CPU and GPU. In my last video (as well as my blog posts), I've mentioned several times that GPU is good at very specific tasks and it's our job to figure that out (and use it to our advantage). My comparison between CPU and GPU is intended as a motivation to look beyond traditional CPU programming. Hope this clears things up. Thanks a lot for your comment as you raised some really good points here. 😃

Thanks a lot for your comment 😃. Right now I'm working on GPU programming and will probably use that knowledge to solve the 3D wave equation (or some other PDE) in parallel on a GPU in some future video. I also have a website where I post blogs a little early. If you're interested you can sign up there to get early access to some of the content and give me feedback (that I can use for the UA-cam video). Thanks a lot again 😃

I'm glad you found the video useful 😃

Thanks a lot 😃

Thanks a lot 😃

Glad you're enjoying the content! I've at least 4-5 videos more planned on CUDA C 😃

@@0mean1sigma Excelent! Those are great news! 😌 Tbh, I just started learning about NLA and well I kinda started taking interest in the internals CUDA C library. Seriously your explanations are just amazing. Thank you so much for the videos:)

Thanks a lot 😀

Yup, that's ep 2 and 3. 😃

The next video is on Warps and details related to the GPU memory (shared memory, registers, etc.)! After that video, I'll make another one on tiled matrix multiplication. If you're interested please sign up on my website where I post the video notes. That way you can access the detailed content and post your thoughts in the discussion section. Thanks a lot for the comment 😃

AMD has ROCm (that's their CUDA). However, I'm not sure how well it works. I tried ROCm some 4-5 years ago and it was not a pleasant experience back then. I moved to nvidia after that because with nvidia, things just work. There's also opencl, but again it has installation and performance issues (in some cases). But the concepts of parallel programming that I focus more on are the same everywhere. Only the syntax changes. Hope this answers your questions. 😃

In Python you have modules like CuPy to use the GPU. Tensorflow and PyTorch will also use it under the hood. The performance will be slightly worse as you have some overhead for abstraction. But it should be a point of deminishing returns in almost every case.

@@MrHaggyy In Python, and in C?

@@finmat95 I'm not aware of C/C++ modules that hide HW abstraction on the level that Python does. But those Python modules use a lot of C/C++ themself so you could look into their code. Or use a framework like Chlorine or Kompute. Kompute is nice because it has the same examples in C++ and Python.

Thanks a lot. Glad you liked it 😃

Thanks a lot 😃 I've a few more videos on GPU programming coming up...

In the real world, matrix size is set by the data so for different problems the number of rows/columns will be different (so it's a variable in that sense). However, once the execution begins, the matrix size does not change. The range of values can be anything (there's no limit on that). However, for very large and small matrices, parallelization techniques will change. I'll cover a few of those in my next video (like: using shared memory, memory coalescing, thread coarsening), so please keep an eye on that (you can also sign up on my website where I publish detailed blog posts and you'll get notified when I publish something). As far as the operations are concerned, I have done a detailed analysis of computation cost in the video notes (link in the description), in short, matrix multiplication is of O(N^3) complexity. I appreciate you watching the video, and if you've more questions after reading the notes, I'm happy to answer those in the discussion section of my website. 😃

@@0mean1sigma I would be very happy if you could make a video about H100s sometime.

@@0mean1sigma So I developed a different processor method. It is not transistor based. Therefore, it has features that are faster and consume less energy than you can imagine. But I can't find a way to use it and make money from it. I am no longer sure that companies and states are really searching for this issue.

I'm not sure what I can say about H100s. My focus is on writing fast (enough) + easy to understand code by understanding the general hardware components (not specific to a GPU/CPU model), so that it scales well with new hardware generations. In any case, I'm not influential enough to get access to H100s so I don't think I would be able to code by keeping H100 specs in mind (at least at this point in time).

All good! Hope you're doing well 😃

@@0mean1sigma I too have enrolled myself in a PhD program in Machine Learning like you. Pretty anxious. Hope things turn out well XD

All the best 😃

You're right! But that's only required if the matrix is too large. Otherwise you'll be looping over the elements sequentially, defeating the purpose of parallelization. There are several optimizations yet to be done and I'm working on a video right now where I'll explain how we can use GPU hardware smartly (especially different memory components) to speed up the computations even more. If you're interested in an early discussion on that topic, please sign up to the blog posts on my website and I would love technical discussions in the comment section there (I generally post early there and you'll get notified in your mail). BTW, Thanks a lot for watching the video 😃

Glad you liked it 😃

Thanks. Glad you liked the content 😃

Thanks a lot 😃

Glad you liked it. I would appreciate your feedback on my blog posts as well (link in the description). I started writing early this month and keen on improving there as well. Thanks a lot again 😃

@@0mean1sigma I’ll sure take a look

this is the first part of every chapter 1 of every CUDA programming book.

Manim is open source. All my work is open source as well and I've provided a link to the animation code for my videos. Thanks a lot for watching... 😃

for a moment, i literally thought it was 3blue1brown video 😅

Glad you liked the video. Unfortunately the complete code for the animations got deleted (accidentally) as I didn't have any kind of workflow back then (but the animation of the network learning is available on the GitHub with link in the video description). However I'm improving on that and have uploaded the code for the animation in my latest video. I'm really sorry about this again.

I'm really sorry about this. I'll keep this in mind next time. Thanks a lot for the feedback.

Thanks a lot.

Glad you found it useful.

Thank you! Cheers!