Відео

@@axeDev22 Thanks for the kind words. I put the note in a GitHub gist here - gist.github.com/nihalpasham/ba0a53200c3f870db54734a113401f2a

@@regionaltantrums Thanks you very much. Have a great day, sir.

That was the plan, but unfortunately, I got pulled into a few other things. Maybe someday, when I have more time, I can finish the remaining pieces. Thank you for watching!

@@__noob__coder__ Cool. Would like to check it out. Please feel free to drop a link when you do.

@@Heater-v1.0.0 Thank you for watching! PS: I’m not the author/maintainer of Cranelift.

Thank you! I'm glad you found it helpful.

@@sekarshreyaaspanchaksharam3679 hi there!

@@mrpocock indeed,

@regionaltantrums running some small models locally, I've had situations where I'm pining one cpu to 100% in python doing the data preprocessing and tokenisation. A rust ecosystem would fix that.

Currently, Cranelift supports only 64-bit backends (aarch64, riscv64, s390x, x86-64). I’m interested in contributing a 32-bit backend to Cranelift.

@@regionaltantrums awesome, looking forward to your sharing in this journey 🥰

Yes, I've looked into TRACTOR. While it's an interesting project, in my experience, code translation is a non-trivial problem. For instance, mapping C design patterns to idiomatic Rust is anything but straightforward, especially with Rust's evolving specifications. That said, more projects are tackling this challenge. If you're interested, the C2Rust project by Galois and Immunant is an early attempt to address it.

@@regionaltantrums Thanks I'll look into that project C2Rust I have not read that one! Wouldn't be surprised if you had a PhD, your idea is quite novel and solving it at the IR level sounds promising! I'll keep Cranelift in mind and tell my colleagues about it.

@@perc-ai I don’t have a PhD, just a deep interest in the topic. Thank you for watching, I really appreciate it.

LLVM is a low-level compiler framework that allows multiple languages (such as Rust, Julia, Swift, and C++) to be compiled down to LLVM IR-a form close to assembly language. Once in LLVM IR, we can select an LLVM back-end, such as x86, ARM64, or RISC-V, to generate actual machine code. On the other hand, MLIR is a higher-level framework. It uses dialects (or multiple intermediate representations) to progressively lower a source language (such as Mojo). The goal of multi-level IR lowering is to simplify handling high-level, domain-specific optimizations (like tensor operations in ML/AI). Ultimately, we still need a code generator to produce machine code, so the final step is often lowering to LLVM IR and using an LLVM back-end to generate the executable code.

Even without MLIR, multi-level IR lowering is already present in most modern languages. For example, Rust follows this progression: Rust → HIR → THIR → MIR → LLVM IR. Similarly, languages like Swift and Julia also undergo multiple stages of lowering. MLIR introduces a framework that formalizes and extends this process, making it more flexible and accessible for a variety of domains. However, MLIR is currently primarily used within the AI/ML community, where it excels at handling domain-specific optimizations, such as tensor operations. Its broader adoption in other areas is still a question (I guess).

@@warriorblood92 it’s called stats, an open source project - github.com/exelban/stats

@@SaschaRobitzki I believe it’s called Monokai (light/dark)

A dark Monokai variant.

Yes, here is the link - github.com/nihalpasham/optimus

@@regionaltantrums 👍

Yes, here is the link - github.com/nihalpasham/optimus

@@regionaltantrums thank you !!

Yes, Candle supports three backends: CPU, CUDA, and Metal. In this stream, I provide a quick walkthrough of Candle’s GPU implementation: ua-cam.com/video/Nhb5HIUeiMM/v-deo.htmlsi=Gf2ZsAI3BQXOkqH6

It's been a while, so I can't recollect off the top of my head, but I believe it had something to do with having to refactor the entire implementation. An earlier version of my code used impl traits rather than the current enum-based design. The reason for the switch was the inability to easily Copy the residual layer when iterating through each encoder block in an encoder. So, my issue was the absence of Copyable types in the residual layer and not Dropout itself.

Ok after reading more on this I think I understand now. Drop it here in case it helps someone: - We feed a Tensor (normalized or not) - with a size of e.g. 512 dimensions - as input for the FeedForward layer. - We will chain 3 transformations over this Tensor: 1.- We expand the Tensor to a higher dimensional size of, say, 2048. To do this we need 512 neurons, each of which has 2048 arbitrarily initialized weights. The linear transformation applied to this Tensor is basically a dot product of the arbitrary weight assigned to each neuron and the initial features, resulting in 2048 features per neuron. 2.- Then, we apply a non-linear transformation (relu) to this 1st linear result. This function sets all negative values to 0, breaking linearity. 3.- Lastly, we compress the relu result (2048 dimensions) back to the original embeddings size (512) by following the same linear strategy of the first step, but this time we will have 2048 neurons each of which has 512 arbitrarily initialized weights. This will yield the FeedForward layer output. One thing that still confuses me is how is each neuron able to self-feed its own result back to itself to correct its arbitrarily initialized weights. My guess that comes later on, when the whole model is running and is able to apply a loss function to each prediction, and "auto-correct" itself.

Thanks you! Yes, the tokenizer includes special tokens. However, we're not using them yet. Instead, we're utilizing a hard-coded input sequence for unit testing purposes. We'll utilize these special tokens when we reach the training part of our project. Here's the plan: github.com/nihalpasham/optimus/issues/2

@@regionaltantrums Awesome thank you!

Yes, I have merged the full Transformer implementation into my GitHub. The video explaining it will follow in some time. Note: this only includes the model's implementation; we still need to obtain the dataset, prepare it followed by model training, and validation.

I believe the reason for the GPU's poor performance was due to including the time it takes to load data onto the GPU in our measurements, versus the actual time taken to compute the dot product alone.

In simple terms, the naive approach is straightforward but can be inefficient on GPUs due to excessive memory accesses and potential thread divergence. On the other hand, the tiled approach optimizes for the GPU architecture by breaking the computation into smaller tiles, leveraging shared memory, and improving data locality and thread efficiency. The tiled approach generally provides better performance than the naive approach for matrix multiplication on GPUs, especially for larger matrices, as it takes advantage of the GPU's parallel processing capabilities and memory architecture more effectively.

Yes, I realized that the audio wasn't captured when I switched over to my iPad after I finished recording. Perhaps I'll redo the stream again sometime when time permits.

I presume you're referring to this one - ua-cam.com/users/liveQeiAl35bAIQ?si=_0D5trP5Mzk0LR2b. If yes, this is a livestream, so they wouldn't appear under the video section.

Sorry about that. My audio setup still needs some work. I’m hoping to fix that in the next stream (whenever that is)

I was hoping to continue this as a series but unfortunately am unable to find time. I have way too many parallel engagements. But hopefully, if I do get the time, I’ll drop a note in advance.

Thank you for your comment. My apologies for the delay in responding. I probably lost track of the notifications here. I’ll drop a note in advance when I do the next libp2p stream (hopefully not too far out)

thanks. one of the goals of this project is its focus on 'simplicity'. Project layout is one of them - unlike something like u-boot or any other popular bootloader, where build (i.e make), toolchain and src files are scattered across many directories and sub-directories, rustBoot's uses a per-board folder that contains its entire hardware abstraction layer. This makes debugging and board bring-up a whole lot easier, but the trade-off is code-redundancy (a small price). Ps: I like your suggestion - may just rename it to rboot :)

@@regionaltantrums Code redundancy is actually a huge software engineering mistake that costs a lot in long term maintenance. It makes harder for people to start off with their own board if they need one, but would have to extract CPU specific stuff for their own sake. But that's only what over 20 years on SWE tells me... I'm glad you liked the name suggestion.

github.com/nihalpasham/JWT-based-device-auth

@@regionaltantrums exceptional doc. Thank you!