Thanks !! Your videos are of great help would really appreciate if you could provide an GGUF model for the same and also a video demonstrating how to quantize a model from huggingface and run on your local ollama
Thx mate. These are awesome. Might you be able to do a similar video for training not just the external LoRA model weights to the cloud but one for training it locally on device and saving the weights locally - potentially to be used with Ollama?
🎯 Key points for quick navigation: 🦙 The video demonstrates how to fine-tune the Llama 3.2 Vision model, a powerful 11 billion parameter AI specifically designed for healthcare image analysis. 📸 The aim is to improve the model's accuracy and specificity in diagnosing from medical images like radiographs by fine-tuning it with specialized data. 🔧 The fine-tuning process is made simpler and faster with the use of the "unso" tool, enabling efficient model adjustments. 🩻 A specific medical dataset, "Radiology mini," containing images and captions, is used to train the model to provide accurate diagnostic descriptions. 💾 The workflow involves several steps: loading the model, dataset preparation, pre-training evaluation, model training, and post-training assessment. 🖥️ The video explains the technical setup required, including software installation, model importing, and use of specific configurations. ⏳ During training, the model's performance is monitored, and after training, the model's accuracy in image description improves notably. ☁️ Once fine-tuning is complete, the adjusted model is saved and uploaded to Hugging Face, integrating changes for wider access and application. Made with HARPA AI
Top result on Google search. "If, on the Llama 3.2 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under this Agreement unless or until Meta otherwise expressly grants you such rights."
Any time I see Fine tuning I click like before watching the video we need more finetuning + CPT cont. pretraining etc.. how to epxlain loss/validation charts & W/B
Do you know of a way to run a fine tunned vision model locally? Looks like saving as gguf is not supported for vision models, so cant run it on ollama.
damn legendary, mate! i was struggling with fine tuning for 3 days and your give was EXACTLY what I was looking for, keep on going!
You are an absolute legend. This is gold.
Thank you fantastic content!
Thanks a lot!! So CooL!
Thanks !! Your videos are of great help would really appreciate if you could provide an GGUF model for the same and also a video demonstrating how to quantize a model from huggingface and run on your local ollama
Thank you Mervin for the video. Would you please tell us the cost of your fine tuning process? Thanks!
Such a useful content, thanks a lot for the video👍
thank you for the démonstration. do you think can we fine tune this model on a videos data?
Super cool.
Hello, I am facing an issue when trying to import FastVisionModel from unsloth:
AttributeError: 'NoneType' object has no attribute 'start'
Did you solve?
@iamakshobhyashukla no not yet
Thx mate. These are awesome. Might you be able to do a similar video for training not just the external LoRA model weights to the cloud but one for training it locally on device and saving the weights locally - potentially to be used with Ollama?
Just to add I presume the Ollama route is probably not possible with your own local version right? Or is it?
How long will the whole process takes?
🎯 Key points for quick navigation:
🦙 The video demonstrates how to fine-tune the Llama 3.2 Vision model, a powerful 11 billion parameter AI specifically designed for healthcare image analysis.
📸 The aim is to improve the model's accuracy and specificity in diagnosing from medical images like radiographs by fine-tuning it with specialized data.
🔧 The fine-tuning process is made simpler and faster with the use of the "unso" tool, enabling efficient model adjustments.
🩻 A specific medical dataset, "Radiology mini," containing images and captions, is used to train the model to provide accurate diagnostic descriptions.
💾 The workflow involves several steps: loading the model, dataset preparation, pre-training evaluation, model training, and post-training assessment.
🖥️ The video explains the technical setup required, including software installation, model importing, and use of specific configurations.
⏳ During training, the model's performance is monitored, and after training, the model's accuracy in image description improves notably.
☁️ Once fine-tuning is complete, the adjusted model is saved and uploaded to Hugging Face, integrating changes for wider access and application.
Made with HARPA AI
Nice.
I am wondering about the ToS of Llama for healthcare.. (clearly, it is non-commercial use…)
Top result on Google search. "If, on the Llama 3.2 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under this Agreement unless or until Meta otherwise expressly grants you such rights."
I was thinking the same thing
Google 'Llama 3.2 licence'. It's fairly open.
Any time I see Fine tuning I click like before watching the video we need more finetuning + CPT cont. pretraining etc..
how to epxlain loss/validation charts & W/B
Very useful information, thanks. Can you please share the code?
Sure: mer.vin/2024/11/finetune-llama-3-2-vision-radiology-images/
Do you know of a way to run a fine tunned vision model locally? Looks like saving as gguf is not supported for vision models, so cant run it on ollama.
You can make a symbolic link between your local llama model and ollama. After that you will be able to run
Thanks for this video. Is there an example or dataset with a more broad usecase? Radiography is a bit niche
Anyone facing issue running unsloth FastvisionModel in Mac?