but i guess here's one: finetuning qwen2.5-32b on various ai related github prjs that are written in python, with a focus on agentic workflows, maybe individual finetunes for each agent python lib / framework, such as autogen, langchain / langgraph, ...
Sadly this may be the first time I disagree with that sentiment. Haha. I loved it but was looking for a course link at the end because it was too much too fast ;)
@@OilersFlash Yea bro (actually uncle, cuz I'm 15 and your dp shows...), I realized that the viewer will need some pre knowledge of LLMs and its workings and also some pre basic knowledge of fine-tuning! But yea the video was good actually!
This is exactly how we are tuning our open source LLMs, the use of unsloth+LORA is key. Validation of the learning is an adventure. We did this over the same tech stack mentioned here. Very good video, of course, lots of details behind each step that cannot be explained in a short video. Great use of lighting and your pace is excellent. Looking forward to more vids.
it depends, this is useful, just if you already know how to do it and you just need a refresh on the steps. In other words, you want to watch it just if you do not need it
I appreciate that this content is geared towards those with prior knowledge, but I think it would be really beneficial to create a beginner-friendly video that breaks down the process into clear, step-by-step instructions. That way, newbies like me can easily follow along and learn from your expertise.
I was already taken by the clear content but the KEY highlight of simply mentioning Conda shows the high quality of your informations. Subscribed and thanks for the tuto.
What a great tutorial, so much info in 5 mins. It took me a while to fill in the gaps because I didn't even know what Jupyter notebook was but I now have this running on my local GPU. You can (just) run this example with 16GB vram.
Great video straight to the point, but could you please elaborate more on feeding custom dataset to the script? What are the steps? You wrote it right on the script or did you load it from a different file? Or did you upload to hugginface and use the token from there? Im confused
Every how-to I've found so far glazes over the training datasets. Like here she goes from here is 100k sql dataset, then here are is how the model expects the prompt, but then doesn't show the format of how the dataset is loaded into the model. What does the sql_context look like, what does the sql_prompt look like, what is the sql, and sql_explaination? Fine tuning is one thing but people also need to know how to build and integrate datasets for training.
@warpdotdev In the interest of establishing the repeatability of your result can you indicate whether the output @ 5:01 is unedited? typical? Mine is amending the prompt and typically produces output in the format of the Alpaca training text generating entries of 'SQL Prompt', 'SQL', 'Explanation', 'Question', 'Instruction', 'Company database' in the output.
hi, can you tell me how I can run a finetuned vision model based on llama-3.2-vision on Ollama. I ask this as unsloth does not support conversion of vision models to gguf .
Thank you for video. I followed your approach for finetuning the model text to SQL. When I work on my own database, its performance is not good. Unable to generate SQL query. I even used agent and examples queries our database. I am requesting suggestion from you. Thank you
Hey, I have a question. I'm currently using gpt api tokens to generate models response thru python and run it over and over. Models are using information from knowledge base folder -- it would be easier to manage it through llama? My ideal scenario is to generate content model, by model and then use that output to generate complete story. Which solutions will you recommend for that purpose? Thanks
Can you train 12B on 24GB or is 12B too big? Another question is if you have multi-turn data (conversations) can you finetune on that? The examples I see are for Q:A pairs.
Thank you for your informative tutorial. I’ve installed Ollama on Windows 10 and currently only use it for interactive conversations through CMD. However, I asked a friend, and they mentioned that Ollama cannot be used for fine-tuning. Why is that? Additionally, Unsloth offers convenient online fine-tuning, but if the training data requires privacy and fine-tuning needs to be done locally or even if the base model is in Safetensors format from others-where should I start? Are there any learning resources or references you would recommend? Thank you!
I'm working on a RAG application. I'm using a pdf file as a text data and I have clean the data as well using NLTK. I already have ollama install in my local system and also llama2 model. I embed the data in to vector form using ollama embedded model mxbai-embed-large and store it on chromabd. but if I give the prompt and get response, it give me the response but not according to my data. and how do I finetune the model? I just need guidance and sequence so I work on the project. It's important to me. I need your guidance. please reply. I'm using windows10. Thank you!
@@ibrahimgamal7603 Could be anything such as you are not using a good doc extractor or your embedding model sucks. Changing chunk and overlap size might help. Don't go for finetuning straight away, do some experimentation and figure out where the problem lies exactly otherwise even finetuning won't work.
if I train a model, will the responses be restricted to just my training data/examples? Or does the model's original training also play a role? For example, say I trained it on model to respond with vehicle specs. I feed it a vehicle name and the model returns the vehicle specs. Say I finetune the model with 50 examples (honda civic, toyota camry, volvo xc90, etc).... however, in reality there are 300,000 possible vehicles. Would finetuning on 50 examples help? Sure, I can put the 50 models in the prompt but then I'd just be wasting tokens.
I would appreciate if this tutorial went more into detail on how to make a dataset comply with the model's promt template. From this video, there is no data processing being done and i think in real cases you would have to work with the data, or write a script, that chnges the data so that it fits with the prompt template (e.g. the Alpaca one she shows in the video)
Where is the data? I did not see any data given to the model to use. I feel like its perfectly possible to have a model "train in realt time" where you tell a model go in to learning mode. Then it will "learn" or "memorize" data you give it. Specifically text based data. And then it has a chance of using it when you ask a question related to that data. Why has this not been done yet? Or has it? I guess what im talking about is somthing like custom GPTs. Which i feel would be eaiser and more worth it. Its just im looking for that but on device.
Great tutorial! Thanks. Really like the definitions and explanations rather than just glossing over the code.. would love to see a full tutorial series 😮 I'd like to try to code it myself, but it would be good if you could share the code too.
It's a great how to but you missed a "before trainng" and "after training" examples of how the model responded. Good video anyway. Couldn't get much out of it though
that subselect that it generated @5:01 makes me instinctively cringe - hoping it would generate the subselect as a join... - for instance why do: "select p.* from posts as p inner join posttags as pt on p.post_id = pt.post_id where tag_id in ( select tag_id from tags where name = 'terminal' )" instead of: select p.* from posts p inner join posttags pt on p.post_i = pt.post_id inner join tags t on t.tag_id = pt.tag_id where t.name = 'terminal' seriously - the 2nd one appears easier to optimize. right? oh well....
why didn't you directly use hugginface trainer to train, why using unsloth? I want to know what was the benefit of using unsloth over hugginface trainer.
Thanks! Although I do have a question why you show so much of yourself... I admit it's nice and pleasant to look at, however I would prefer to look at the code or things you talk about. Especially when I watch your videos at work I would definitely like to avoid someone thinking that I watch some silly videos during work while I actually watch work related stuff. Anyway, really interesting videos so thanks again! 😊✌
What data would you fine tune your LLM on?
what is your hardware ? are you using intel or amd threadripper ?
that depends on the use case... are you asking for what use cases people are aiming to support?
but i guess here's one: finetuning qwen2.5-32b on various ai related github prjs that are written in python, with a focus on agentic workflows, maybe individual finetunes for each agent python lib / framework, such as autogen, langchain / langgraph, ...
youtube need more chanels alike this. great job
I'm using it to fine-tune a Minecraft bot (Mindcraft from Emergent garden)
No over rating, no over talking, streight forward, love it.
Sadly this may be the first time I disagree with that sentiment. Haha. I loved it but was looking for a course link at the end because it was too much too fast ;)
@@OilersFlash Yea bro (actually uncle, cuz I'm 15 and your dp shows...), I realized that the viewer will need some pre knowledge of LLMs and its workings and also some pre basic knowledge of fine-tuning! But yea the video was good actually!
@@siddhubhai2508 it is good ;)
100% this.
This is exactly how we are tuning our open source LLMs, the use of unsloth+LORA is key. Validation of the learning is an adventure. We did this over the same tech stack mentioned here. Very good video, of course, lots of details behind each step that cannot be explained in a short video. Great use of lighting and your pace is excellent. Looking forward to more vids.
Thank you straight to the point
I usually have some ptsd when looking at tutorials
it depends, this is useful, just if you already know how to do it and you just need a refresh on the steps. In other words, you want to watch it just if you do not need it
hi i love you
Great video for people who know coding and local llm but havent finetuned!
amazing quality of editing, sound, video - beyond the programming side!
A pleasure for the eyes and ears to watch!
Knowing such a deep technical subject is one thing, but teaching it well is another! well done young lady.
I appreciate that this content is geared towards those with prior knowledge, but I think it would be really beneficial to create a beginner-friendly video that breaks down the process into clear, step-by-step instructions. That way, newbies like me can easily follow along and learn from your expertise.
seconding this. its hard to find good resources on making/using your own datasets when its not in such a friendly prepackaged format
Keep searching UA-cam.
I was already taken by the clear content but the KEY highlight of simply mentioning Conda shows the high quality of your informations. Subscribed and thanks for the tuto.
What a great tutorial, so much info in 5 mins. It took me a while to fill in the gaps because I didn't even know what Jupyter notebook was but I now have this running on my local GPU. You can (just) run this example with 16GB vram.
Thank you for the simple, straight forward insight. Hope you come out with more bite size knowledge.
Easiest subscribe of my life, just wanted something easy and straight to the point
Five minutes made useful. Thank you for the crisp and neat video. ❤
Insanely good video!! Straight to the point and great presentation
Hi there.
It's wonderful.. Will you Please share the notebook and also Google Collab notebook?
This is what I am thinking to do. Finding this video maybe saved half a day
Great video straight to the point, but could you please elaborate more on feeding custom dataset to the script? What are the steps? You wrote it right on the script or did you load it from a different file? Or did you upload to hugginface and use the token from there? Im confused
Every how-to I've found so far glazes over the training datasets. Like here she goes from here is 100k sql dataset, then here are is how the model expects the prompt, but then doesn't show the format of how the dataset is loaded into the model. What does the sql_context look like, what does the sql_prompt look like, what is the sql, and sql_explaination? Fine tuning is one thing but people also need to know how to build and integrate datasets for training.
Agreed. Every video is skipping over detail to actually build your own functionality.
That’s kind of the point. It’s open source open secret
Did you find a better video that explains this?
great and short video guys incredible!!
@warpdotdev
In the interest of establishing the repeatability of your result can you indicate whether the output @ 5:01 is unedited? typical? Mine is amending the prompt and typically produces output in the format of the Alpaca training text generating entries of 'SQL Prompt', 'SQL', 'Explanation', 'Question', 'Instruction', 'Company database' in the output.
Yes, same. Would be useful if they share the link to the code.
Great explanation and presentation of LLM.
What a clean presentation.
So nice and clear! Tank you a lot!
wouldve been nice if you had shared the full collab code...
Guys, it is a collab from Unsloth. You can search on Google "Llama-3.1 8b unsloth" and you will find out.
Liked and subbed, so helpful, thank you!!
Immediately subscribed
Another great video!! Thanks.
Great video. To the point. Effective
hi, can you tell me how I can run a finetuned vision model based on llama-3.2-vision on Ollama. I ask this as unsloth does not support conversion of vision models to gguf .
Regarding validation loss , is it normal that requires more time?
nice and straight forward approach
Could you please let us know what is hardware configuration needed for pc or laptop to run ollama and train llm models
Great video! Subscribed! 🎉
Well explained. Thanks
this is great! gonna try this...
Thank you for video. I followed your approach for finetuning the model text to SQL. When I work on my own database, its performance is not good. Unable to generate SQL query. I even used agent and examples queries our database. I am requesting suggestion from you. Thank you
Appreciate the brevity. Thank you.
Hey, I have a question. I'm currently using gpt api tokens to generate models response thru python and run it over and over. Models are using information from knowledge base folder -- it would be easier to manage it through llama? My ideal scenario is to generate content model, by model and then use that output to generate complete story. Which solutions will you recommend for that purpose? Thanks
So we are just creating loras here, I can have many and mount them as I need like with Stable Diffusion right?
I would really appreciate a version of this video but using a Mac thank you so much
Excellent tutorial! Doesn’t lowering the bit depth of the model greatly reduce accuracy? What are the pros and cons of doing so? Thanks!
Can you train 12B on 24GB or is 12B too big?
Another question is if you have multi-turn data (conversations) can you finetune on that? The examples I see are for Q:A pairs.
Thank you for your informative tutorial.
I’ve installed Ollama on Windows 10 and currently only use it for interactive conversations through CMD.
However, I asked a friend, and they mentioned that Ollama cannot be used for fine-tuning. Why is that?
Additionally, Unsloth offers convenient online fine-tuning, but if the training data requires privacy and fine-tuning needs to be done locally
or even if the base model is in Safetensors format from others-where should I start?
Are there any learning resources or references you would recommend?
Thank you!
learn from chatgpt
Yoo, what theme are you using in your system?? That's really cool
can this be done all from the wsl terminal? I am using ollama, webui and docker, but would like to train some company pdf's better, can this be done?
It was a great video, but I have a question, is it compatible with the new versions called 3.2, especially versions 1B and 3B?
Awesome explanation in short video….some UA-camrs consuming 2 to 3 hours for this content
Is it possible to do this while not having a NVIDIA graphics card?
Intresting video!!
I'm working on a RAG application. I'm using a pdf file as a text data and I have clean the data as well using NLTK. I already have ollama install in my local system and also llama2 model. I
embed the data in to vector form using ollama embedded model mxbai-embed-large and store it on chromabd. but if I give the prompt and get response, it give me the response but not according to my data. and how do I finetune the model? I just need guidance and sequence so I work on the project. It's important to me. I need your guidance. please reply. I'm using windows10.
Thank you!
It's easy you should write in your prompt template = """Answer the question based ONLY on the following context:
{context}
Question: {question}
"""
@@ibrahimgamal7603 Could be anything such as you are not using a good doc extractor or your embedding model sucks. Changing chunk and overlap size might help. Don't go for finetuning straight away, do some experimentation and figure out where the problem lies exactly otherwise even finetuning won't work.
@@muhammadumarnawaz9200 ok mate thank you for your help
Can you provide me the link for your ubuntu desktop wallpaper 🙃
at 2:00 is 8 Billion not 8 bit.
Do you ever help companies with their own projects? I'm looking for some help on my build
if I train a model, will the responses be restricted to just my training data/examples? Or does the model's original training also play a role?
For example, say I trained it on model to respond with vehicle specs. I feed it a vehicle name and the model returns the vehicle specs. Say I finetune the model with 50 examples (honda civic, toyota camry, volvo xc90, etc).... however, in reality there are 300,000 possible vehicles. Would finetuning on 50 examples help? Sure, I can put the 50 models in the prompt but then I'd just be wasting tokens.
Excellent direct video on fine-tuning, congrats. Could you also share the python source code that you used in the video? Thanks.
I would appreciate if this tutorial went more into detail on how to make a dataset comply with the model's promt template. From this video, there is no data processing being done and i think in real cases you would have to work with the data, or write a script, that chnges the data so that it fits with the prompt template (e.g. the Alpaca one she shows in the video)
How long did it take for the training / fine-tuning on your 4090? Thanks for the video!
Apologies if this is a dumb question, but where is, is there a link to the notebook?
I have 55K classes with an ungodly amount of transcripts. What'll be better and faster? RAG or finetuning?
How do you train on 3Gb pdf and word files locally
is the notebook posted anywhere?
good guide to play by myself
what ubuntu are you using, the terminal looks dope
Does fine-tuning shouldn't generate small models that theoretically run fully local and avoid spending money on OpenAI?
Is unsloath free?
Could you share the link for ipynb file?
Where is the data? I did not see any data given to the model to use. I feel like its perfectly possible to have a model "train in realt time" where you tell a model go in to learning mode. Then it will "learn" or "memorize" data you give it. Specifically text based data. And then it has a chance of using it when you ask a question related to that data. Why has this not been done yet? Or has it? I guess what im talking about is somthing like custom GPTs. Which i feel would be eaiser and more worth it. Its just im looking for that but on device.
what is the estimate cost for training such a model?
I actually liked the music. Great editing also. I found the proyect idea a little bit boring. You could train it for something actually awesome
can you provide the code you used?
Is it possible to make your own AI model with this that speaks to you like Jarvis and understands tone, etc?
Wow! Can this work on MAC?
Do you have github repo of the code?
Video on formatting data sets to follow? 😅
fine tuneing offline ?
Does this support AdaLoRA or Vera?
How to collect dataset pls show it
Isn't this RAG? Doesn't chunking fit anywhere?
Give us the collab link! Or better yet the llamasql gguf link!
Great tutorial! Thanks. Really like the definitions and explanations rather than just glossing over the code.. would love to see a full tutorial series 😮
I'd like to try to code it myself, but it would be good if you could share the code too.
Can we train tinyllama to do something similar? Since was trying to run AI on Raspberry Pi 5 (with Hailo AI Accelerator)
Absolutely!
@@warpdotdev Thanks! ❤️ from India
I think I'll just get an llm to make the dataset and make it llms both sides of the data->training loop
Thanks 🙏
Can you make the code are smaller and remove the background music
Unsloth has a dependency of triton, which doesn't seem to be compatible with windows.
Inam finding difficulty understanding the converting of data set
Is ubuntu on windows ? Or dependent system ?
I need more tiny steps to convert the dataset
Great 👍
It's a great how to but you missed a "before trainng" and "after training" examples of how the model responded. Good video anyway. Couldn't get much out of it though
সাধারনত সুন্দুরী মেয়েদের বুদ্ধিমত্তা হাঁটুর নিচে থাকে। কিন্তু মিলিয়নের মধ্যে তোমার মত হয়তো ১/২ জন থাকে। আমি উপকৃত হয়েছি।।
Nice video, but the music is way too disctracting.
nah, only a little distracting.
that subselect that it generated @5:01 makes me instinctively cringe - hoping it would generate the subselect as a join... - for instance why do:
"select p.* from posts as p inner join posttags as pt on p.post_id = pt.post_id where tag_id in ( select tag_id from tags where name = 'terminal' )"
instead of:
select p.* from posts p inner join posttags pt on p.post_i = pt.post_id inner join tags t on t.tag_id = pt.tag_id where t.name = 'terminal'
seriously - the 2nd one appears easier to optimize. right? oh well....
link
subscribed
Can I do this without Conda? I hate Conda
Why yes, why wouldn’t you?
Why not share the source code???
why didn't you directly use hugginface trainer to train, why using unsloth? I want to know what was the benefit of using unsloth over hugginface trainer.
No benefit My friend :(
Less memory was what she stated
damn, now i want to create my own personal DAN assistant without OpenAI interfere with her
where is the link for google colab?
Nice
Thanks! Although I do have a question why you show so much of yourself... I admit it's nice and pleasant to look at, however I would prefer to look at the code or things you talk about. Especially when I watch your videos at work I would definitely like to avoid someone thinking that I watch some silly videos during work while I actually watch work related stuff. Anyway, really interesting videos so thanks again! 😊✌