Very much appreciate this video, fine-tuning seemed like a somewhat amorphous concept to me for sometime, but the diagrams you showed really made it easier to understand how people finetune.
Thanks for sharing, especially about Lit-GPT (I'm always interested in more tutorials as my journey with fine-tuning and LLMs needs all the help it can get). Thanks again.
One of the approaches I have experimented with, which is both manual labor, time and compute expensive but more reliable, is as follows: - Use a LLM to query for outputs. Use RAG and prompt engineering to get the best possible results. - Generate chat logs for each query. The log should include everything - the prompt, the retrieved info if any and the model output. Any special symbol such as to denote the system prompt or anything else should also be left in. This is because LLMs are text generation models with no concept of chat. - Manually update the model outputs to better reflect the expected output. This is a data creation task. - Fine tune a copy of the same LLM using PEFT using the updated chat logs. This can also be done iteratively as long the chat logs are generated initially by a model which hasn't been fine-tuned yet. Like a sort of A/B experiment. Some use cases are served the original model that generates the data for fine-tuning while the other are served the fine-tune model whose outputs are not used for any further fine-tuning. Expensive but over time, your model would work better for realistic inputs.
Glad that it was helpful! HF itself has different tools for finetuning. Similarly, the LitGPT library I help developing supports full finetuning, LoRA, QLoRA, etc.
I really wish people would stop putting their x link and start sharing something like mastadon or threads, as a free user, x is where u go to feel second class citizen.
I hear you. On that note, I do have Threads and Mastodon accounts 😅. Just not using them much, somehow all the AI folks are still on X :(. I think the days of this type of social media are counted ...
@@SebastianRaschka Haha, I get it. I feel in the topic of "All the AI folks are sill ON X" is somewhere the buck starts with you problem, if more people start sharing it will eventually move there I guess.
Thanks
Very much appreciate this video, fine-tuning seemed like a somewhat amorphous concept to me for sometime, but the diagrams you showed really made it easier to understand how people finetune.
Thanks so much, glad these diagram were helpful and helped clarifying!
Thanks for sharing, especially about Lit-GPT (I'm always interested in more tutorials as my journey with fine-tuning and LLMs needs all the help it can get). Thanks again.
Glad you are liking LitGPT!
One of the approaches I have experimented with, which is both manual labor, time and compute expensive but more reliable, is as follows:
- Use a LLM to query for outputs. Use RAG and prompt engineering to get the best possible results.
- Generate chat logs for each query. The log should include everything - the prompt, the retrieved info if any and the model output. Any special symbol such as to denote the system prompt or anything else should also be left in. This is because LLMs are text generation models with no concept of chat.
- Manually update the model outputs to better reflect the expected output. This is a data creation task.
- Fine tune a copy of the same LLM using PEFT using the updated chat logs.
This can also be done iteratively as long the chat logs are generated initially by a model which hasn't been fine-tuned yet. Like a sort of A/B experiment. Some use cases are served the original model that generates the data for fine-tuning while the other are served the fine-tune model whose outputs are not used for any further fine-tuning.
Expensive but over time, your model would work better for realistic inputs.
I recently listened to your latest videos. And now this one was recommended by perplexity for my specific use-case ;-) coincidence?
Haha, looks like LLMs are coming full circle here :D
Thanks for the video, very helpful for me to understand different kinds of finetunning. BTW, what kind of finetunnig is huggingface belong to?
Glad that it was helpful! HF itself has different tools for finetuning. Similarly, the LitGPT library I help developing supports full finetuning, LoRA, QLoRA, etc.
Thanks! Great overview
Awesome, thank you!
Plase Show us please how to create a dataset for finetuning, not just downloading it.
I have some resources for that here: github.com/rasbt/LLMs-from-scratch/tree/main/ch07/05_dataset-generation
Nice Video.
Time saw you here on YT! Hope you remember me.!
Thank you :)
Cool
I really wish people would stop putting their x link and start sharing something like mastadon or threads, as a free user, x is where u go to feel second class citizen.
I hear you. On that note, I do have Threads and Mastodon accounts 😅. Just not using them much, somehow all the AI folks are still on X :(. I think the days of this type of social media are counted ...
@@SebastianRaschka Haha, I get it. I feel in the topic of "All the AI folks are sill ON X" is somewhere the buck starts with you problem, if more people start sharing it will eventually move there I guess.
Amazing, Thanks