I'm currently looking for a resource on how to pick `modules_to_save` for LoraConfig (and perhaps the other values too, although those just seem like accuracy trade-offs too). Will update here if I can answer my own question.
Outstanding video, the best one I have seen on LoRA! I have one question about the SVD decomposition procedure: A full fine tune of a large model such as LLaMa would require loading the entire model tensors onto the GPU and adjusting them by delta(phi) for all the parameters. In LoRA, delta(phi) is replaced with 2 smaller SVD matrices that are trained and then multiplied back into the full size and added to the original parameters. My question is this. When you generate the 2 smaller SVD matrices, you still need to load in the full size tensor to then decompose. In PEFT, are the 2 SVD matrices calculated once at the beginning for all the different tensors before fine tuning occurs? Also how is it possible to backpropagate through the 2 smaller matrices without combining them back together on the GPU?
very thoughtful query.. forced me to read the paper :) .. basically you are right about loading the initial full sized matrices but then they are frozen for the rest of the fine tuning process and then the lower ranked / decomposed matrices kick in ..they DO NOT need to be recombined since the idea is NOT to create an exact replica of the information contained in the pre trained model. All we need is a close approximation which is provided by combining these 2 lower ranked johnnies ( U multiplied with top N ranked singular values AND Vt mult with top N ranked sv's)..and since they are both trainable , these are the weights that will then be used. So during inference, you will have the impact of both the pre trained LLM / LM and then the task specific representations of the LoRA ..this is what i have understood ..if there's a fallacy here, please do point out .. i love to be wrong ..helps better learnin
@@pumplove81 Anyways we need to do feed-forward through the original model + BA. Maybe we will just have a gain not to store a lot of gradients during backprop.
According to the study in the video, does Lora really achieve better results than full fine tuning of all parameters of the entire model? Does this mean that it is not only less demanding on computing resources, but also has better performance?
I still do not understand how LORA preserves previous information of the original dataset? Lets say our base model is a bert trained on wikipedia, when using LORA, lets consider the first block of attention, we feed forward the input features to the original weights and to the BA matrix, then we add the two results. In this way the next layer inputs are changed whereas the weights in the next layers expects something else. I still don't get it.
Just my guess, I think you might be able to compress the models for storage but when you actually want to do inference you need to decompress it before use.
BEST video on PEFT and LORA..I was not able to undertsand the concepts from other videos, by searching more on YT I land up on this video and I understood whole concepts. JUST AWESOMEEEEEEE..
Great presentation! A nomenclature question for you, in Figure 1 in the LoRA paper they have "B=0" and "A=Nu(0,sigma^2). I'm not sure how that relates to the description of A and B, can you give any insight?
I'll answer my own question - it looks like the Figure 1 is just showing initialization. I had forgotten that N is the symbol for the Gaussian distribution, so it is just saying they are initializing A with a Gaussian distribution with a mean of 0 and initialization B = 0 since the initial delta for weights is zero at the start of training.
letters and voice is one of the best ways to confuse an audience. There is plenty of studies that prove this the brain can only do one thing at a time.
Great video again sir. I need some guidance from you. Let's say i have a very large text lines those are belongs to doctor's CMS dataset, their publication and clinical trials. I want to train a pretrained model with thses datasets. After request questions would looks like: who is the best urologist at New York? Can you guide to take a correct path to achieve this. Like with language model should I pick to fine tune on my data. My previous experience: 1. Langchain interface, llm=openAI and chromadb. Result are too good but openai is not a open source. 2. Same as above but this time i used llm = Dolly 2.0 12b/7b model. Result's are not accurate. They even can't count the number of paper published by doctor. So now I want to fine tune a pretrained model with my data. Can you suggest me how can achieve this. Or any better approach?
outstanding video. Brilliantly explained complex topics.. I have one question, Can we do Lora to multi modal architectures like Donut which is a combination of Swing transformer + Bard ? Any pointers to do this
If you have access to Bard, to inject additional tensors, and then fine-tune Bard, on its compute infrastructure, you work for Google and know the answer to your question.
Hi boss, I have a question: If I finetune 4 LoRA models based on an LLM (e.g. Llama13b), each LoRA trained on a specific task. Can I deploy all the 5 models on the same GPU (includes the parameters of 4 LoRAs and the base model)? For example, let's suppose each adapter takes 0.5GB vram and the base model takes 26GB vram on the GPU; is it possible to deploy all sets of LoRAs and the base model on the same GPU with the usage of 28GB? (26+0.5*4=28GB)
The deployment framework needs to provide this support. technically saying this should be possible if the architecture supports switching instead of just sequentially calculating Not sure if such support exists in any framework. But then again I don't have knowledge of all frameworks.
hmm, so assuming I have a model X with multiple lora adapters trained on specific tasks. Is it possible to merge all those adapters back to single model? Just curious :)
Quantization is the mapping of a k-bit integer to a real element in Domain D, or (1) Compute a normalization constant N that transforms the input tensor T into the range of the domain D of the target quantization data type Q, (2) for each element of T/N find the closest corresponding value q(i) in the domain D, (3) store the index i corresponding to q(i) in the quantized output tensor T(Q). That is so easy in information theory, that when I compare it to real physics, I smile and think of it as a singular quantification problem. And beliefs manifest themselves in language. Good point!
very clear explaination on low-rank and LLM
Thank you.
Superb video. Excellent presentation of all the concepts and easy to understand. You have a great teaching style sir.
Some hidden gem!
Top notch explanation 💯🔥
讲得非常好，awesome impressive！
Most underrated channel on YT. Deserves a million subs. Thanks.
Appreciate that!
Agreed
Nice Explanation Very Good
I'm currently looking for a resource on how to pick `modules_to_save` for LoraConfig (and perhaps the other values too, although those just seem like accuracy trade-offs too). Will update here if I can answer my own question.
Hi a question, can we use lora to just reduce the size of a model and run inference, or we have to train it always?
First 🦄
Outstanding video, the best one I have seen on LoRA!
I have one question about the SVD decomposition procedure:
A full fine tune of a large model such as LLaMa would require loading the entire model tensors onto the GPU and adjusting them by delta(phi) for all the parameters.
In LoRA, delta(phi) is replaced with 2 smaller SVD matrices that are trained and then multiplied back into the full size and added to the original parameters.
My question is this.
When you generate the 2 smaller SVD matrices, you still need to load in the full size tensor to then decompose. In PEFT, are the 2 SVD matrices calculated once at the beginning for all the different tensors before fine tuning occurs? Also how is it possible to backpropagate through the 2 smaller matrices without combining them back together on the GPU?
very thoughtful query.. forced me to read the paper :) .. basically you are right about loading the initial full sized matrices but then they are frozen for the rest of the fine tuning process and then the lower ranked / decomposed matrices kick in ..they DO NOT need to be recombined since the idea is NOT to create an exact replica of the information contained in the pre trained model. All we need is a close approximation which is provided by combining these 2 lower ranked johnnies ( U multiplied with top N ranked singular values AND Vt mult with top N ranked sv's)..and since they are both trainable , these are the weights that will then be used. So during inference, you will have the impact of both the pre trained LLM / LM and then the task specific representations of the LoRA ..this is what i have understood ..if there's a fallacy here, please do point out .. i love to be wrong ..helps better learnin
@@pumplove81 Anyways we need to do feed-forward through the original model + BA. Maybe we will just have a gain not to store a lot of gradients during backprop.
According to the study in the video, does Lora really achieve better results than full fine tuning of all parameters of the entire model? Does this mean that it is not only less demanding on computing resources, but also has better performance?
I still do not understand how LORA preserves previous information of the original dataset? Lets say our base model is a bert trained on wikipedia, when using LORA, lets consider the first block of attention, we feed forward the input features to the original weights and to the BA matrix, then we add the two results. In this way the next layer inputs are changed whereas the weights in the next layers expects something else. I still don't get it.
You lost me at the linear algebra, I decided I will abstract that part 🤣
Great video that doesn't hand-wave away the mathematical and implementation details. Exactly the kind of content I love. Thank you!
I'm having so many flashbacks from my PCA classes😅, you explain much better than my teacher btw...
Superb video :) Very clear and concise explanation. Thank you.
Thank you for this comment. Appreciate it if people take time for a feedback.
This is so well done. You inspired me to create similar content as well. Hats off!
Every matrix can be represented as a product of matrices BA. Could this be used to compress models?
Just my guess, I think you might be able to compress the models for storage but when you actually want to do inference you need to decompress it before use.
It doesn't get better than this ❤
BEST video on PEFT and LORA..I was not able to undertsand the concepts from other videos, by searching more on YT I land up on this video and I understood whole concepts. JUST AWESOMEEEEEEE..
where can I download the PPTs
Beautiful...Beautiful...
Appreciate your positive feedback! Thank you!
Great presentation! A nomenclature question for you, in Figure 1 in the LoRA paper they have "B=0" and "A=Nu(0,sigma^2). I'm not sure how that relates to the description of A and B, can you give any insight?
I'll answer my own question - it looks like the Figure 1 is just showing initialization. I had forgotten that N is the symbol for the Gaussian distribution, so it is just saying they are initializing A with a Gaussian distribution with a mean of 0 and initialization B = 0 since the initial delta for weights is zero at the start of training.
letters and voice is one of the best ways to confuse an audience. There is plenty of studies that prove this the brain can only do one thing at a time.
Great video again sir.
I need some guidance from you.
Let's say i have a very large text lines those are belongs to doctor's CMS dataset, their publication and clinical trials.
I want to train a pretrained model with thses datasets.
After request questions would looks like: who is the best urologist at New York?
Can you guide to take a correct path to achieve this.
Like with language model should I pick to fine tune on my data.
My previous experience:
1. Langchain interface, llm=openAI and chromadb.
Result are too good but openai is not a open source.
2. Same as above but this time i used llm = Dolly 2.0 12b/7b model. Result's are not accurate. They even can't count the number of paper published by doctor.
So now I want to fine tune a pretrained model with my data.
Can you suggest me how can achieve this.
Or any better approach?
does it work on models i trained using tf? or just the models imported from the hugging face library?
This is really awesome... you nailed it. Such explanation can only come from deep understanding. Thank you very much..
outstanding video. Brilliantly explained complex topics.. I have one question, Can we do Lora to multi modal architectures like Donut which is a combination of Swing transformer + Bard ? Any pointers to do this
If you have access to Bard, to inject additional tensors, and then fine-tune Bard, on its compute infrastructure, you work for Google and know the answer to your question.
Subscribed!
Champion explanation. Thank you
Best video and explanation on LoRA, thank you for your efforts!
NICE !!!
Wunderbar. Vielen Dank.
Hi boss, I have a question: If I finetune 4 LoRA models based on an LLM (e.g. Llama13b), each LoRA trained on a specific task. Can I deploy all the 5 models on the same GPU (includes the parameters of 4 LoRAs and the base model)? For example, let's suppose each adapter takes 0.5GB vram and the base model takes 26GB vram on the GPU; is it possible to deploy all sets of LoRAs and the base model on the same GPU with the usage of 28GB? (26+0.5*4=28GB)
The deployment framework needs to provide this support. technically saying this should be possible if the architecture supports switching instead of just sequentially calculating
Not sure if such support exists in any framework. But then again I don't have knowledge of all frameworks.
Such great content! Is it possible to get the slides? Thanks!
what tool r u using for the presentation i love the smooth transitions
Amazing explaining ❤️
hmm, so assuming I have a model X with multiple lora adapters trained on specific tasks. Is it possible to merge all those adapters back to single model? Just curious :)
Yes.
Thank you for the amazing video!
Just Wow !!!
This is a great video!
You spend a lot of time talk about SVD, how does it related to LoRA, does Lora use SVD in it algorithm?
Thats what he tried to show as far as I understood, the equations for decomposition are the same.
Thank you for explaining. I previously believed that LoRa was a stable diffusion generating beauty,
Dude, you’re like a gift
Amazing explanation!!!!
Glad you think so!
Dude your explanations are 💯👌
Absolutely brilliantly explained. Love this guy's style of teaching and his casual humour. Do we have to drop to int8 for PEFT?
Best video on LORA ever! Simply can't get better than this 🏆
My subscriber are the best!
Thank You
You are welcome.
Comment for the algorithm👏
Thanks for the great video! Is OpenAI using adapters or a similar technology for the fine-tuning they offer on GPT-3 base models?
I have not seen any official documentation by the company about this ...
Open AI are not releasing how they are trained the AI
Wow awesome videos ❤
This was an amazing explanation. Thank you.
Fantastic!!
Absolute gem
Reaaallly amazing explanation👏👏👏
👍🏻
This is Gold. Thanks for this amazing content.
Great feedback. Thanks.
kudos
Quantification and quantization are two very different things.
Quantization is the mapping of a k-bit integer to a real element in Domain D, or
(1) Compute a normalization constant N that transforms the input tensor T into the range of the domain D of the target quantization data type Q, (2) for each element of T/N find the closest corresponding value q(i) in the domain D, (3) store the index i corresponding to q(i) in the quantized output tensor T(Q). That is so easy in information theory, that when I compare it to real physics, I smile and think of it as a singular quantification problem. And beliefs manifest themselves in language. Good point!
@@code4AI Well, FWIW, quantization sort of just happens in real physics, if you go far enough ;)
Wonderful explanation!
Thank you!
WoW everytime, thank you
Thank you for this feedback.
very well explained.
Thank you for your feedback!
Subscribed.
Appreciate it!
Love the speed of explaination!
Clean and accurate!