Відео

Hi Ronan, Can you guide on the best resources for algorithmic trading using ML, DL and AI. Also, are you planning to offer Black Friday Sale or Christmas Discount on your Trelis Advanced Repo?

Wonderful work.

oH my, another cool content!

I am just going through this... print("Model config:") pprint(model.config.to_dict()) # Specifically get hidden size if hasattr(model.config, 'hidden_size'): print(f" Hidden dimension: {model.config.hidden_size}") elif hasattr(model.config, 'd_model'): print(f" Hidden dimension: {model.config.d_model}") else: # Try to find it in the model architecture for name, param in model.named_parameters(): if 'embed' in name: print(f" Embedding dimension (likely hidden dim): {param.shape[-1]}") break Embedding dimension (likely hidden dim): 1024 So square root of that is about 32, not 8?

Awesome video Ronan as always. This video is very timely as we are optimizing our costs by moving away from runpod serverless. I have a couple of questions. - Can the service you have written scale to 0? It seems that with the minimum TPS being a positive number, this wouldn't work right? Scaling to 0 is very important for us as we have bursty traffic with long idle times and this is the primary motivation for serverless. - Is there any alternative to configuring the TPS scaling limits manually for each GPU/model combination? This seems a bit cumbersome. Would it be possible to scale directly based on the GPU utilization? I am thinking something like ssh'ing into the instance with paramiko and automatically running nvidia-smi (you can output results to a csv with --format=csv and --query-gpu parameters). You can then use the output of these results to determine if the GPUs are at full utilization. Maybe take a sample over your time window as this number can fluxuate a lot. Then you can use this to determine whether you need to add or subtract instances and you could use current TPS to determine if the instances is being used at all (scale to 0). Do you think this approach would work? - Do you only support runpod or can other clouds like vast.ai or shadeform be added as well? Both have apis that allow you to create, delete, and configure specific instances. Runpod has had many GPU shortage issues lately specifically for 48gb GPUs (A40, L4, L40, 6000 Ada etc.) - Is there any configuration here for Secure cloud vs. Community cloud? I think by default if you don't specify in the runpod api, it defaults to "ALL" which means that you will get whatever. Community cloud can be less stable and less secure so many users may want to only opt for Secure cloud. Again, I really appreciate the content you produce. For anyone who hasn't purchased access to the Trelis git repos yet, they are quite the value. Ronan consistently keeps them up-to-date with the latest models and new approaches. It is a great return on your investment and the gift that keeps on giving!

Trelis at it again..

Thanks for the video! I just spoke to RunPod support and they said that you pay for an active worker whether it is running (i.e. processing API endpoint requests) or not. But in your video at 18:07 you mention that you only pay when the worker is actually running (processing requests). I'm a bit confused now as this means for a single worker GPU costing $0.00019 p/s this would be approx. $345 p/m! For a new web app without much traffic or revenue this would be too expensive to run. Would you mind clarifying the severless costs again please? And in-particular if you have run serveless endpoints with a single active worker, and if so, were you charged only when the active worker is processing requests or 24/7 even when it is idle? This is a direct quote from RunPod support: "To clarify, the cost is incurred for each active worker, and it applies while the worker is active, even if it's not actively processing requests. The worker is dedicated to your endpoint and cannot be shared with others, so you incur a cost as long as the worker is running, not just when it is processing requests."

What is main challenge you feel when there is Indian Language. It's about style vector or phonemes?

very helpful and informative video. thank you

Thanks for the awesome video.

So this is the master class, in practical LLM fine-tuning with the latest cutting edge technology. I have searching for such a comprehensive tutorial for so long. Thanks!

whisper faster or whisper turbo?

Amazing

How do we deal with hallucination resulting from our background info?

Isn't referring expression basically like "dog to the right" in a image of, lets say two dogs, it will segment the dog to the right.

wow this is really cool, what's next? um... how about OmniParser by Microsoft?

Can you please make a tutorial on ddp of computer vision or LLMs using docker and Kubernetes for industry scale training. Ray is also slowly picking up pace. Your videos are great but can you please help to scale them to large scale training.

All this feels very exciting , i feel like someone with a great insight on the application of such amazing AI Watching these tutorials feels like watching the under the hood tech of a potential billion or even a trillion dollar empire

You are amazing!!!

Please censor your hair it's so good it's distracting xD

Great tutorial! There's something missing tho'. How to define correctly the examples for any DSPy methods that accept a trainset. You need to define a list of dspy.Example(). Putting it out here maybe it well help someone.

Best tutorial on DSPy I've watched so far.

nice

i see the code ```max_tokens_per_img = 4096``` in the advance demo of vLLM offline inference example, does this basically mean the maximum amount of the patches of 16x16 pixel it would support?

Pros vs cons , faster but uses more tokens? Seems worse than cache, faster and cheaper. Dosent seem efficient to regeneration of everthing for just a small chage.

Great tutorial with no nonsense, thank you so much for creating & sharing.

very clear and thorough. subscribed!

Thanks a lot for this incredibly useful video. I am trying to fine tune Mistral 8x7b using QLORA and PERT. I am using A100 machine. Was very confused about model and adopter combination. Your video has helped a lot.

Great video! What I don't get is how does this actually saves inference compute? I mean to be sure that my "predicted output" is correct do I not need to calculate the output token as I traditionally would? e.g. the cat is on the -> table. I already have table in my predicted outputs, however to be sure that that's correct don't I need to run inference on the llm anyways? thanks a lot!

Nice, is there a similar feature also in transformers?

hi ronan. do you have an email id to connect?

LoRAX has something very similar. The approach is also very interesting, we can Llama 8B and train a Speculative Decoding Lora adapter and speed up the generation in a lot of cases

Please make something on MLX and using not a very high end macbook for fine tuning small LMs locally and more

Very clear and concise video, great work!

On 11:53 After connecting to Jupyter Notebook, I don't see Whisper fine-tuning notebook. Where should I get it and how to put it there? I've purchased Lifetime Access, but didn't receive any access details yet.

Very interesting... will try... tks 🎉

Brilliant Thank you

your video is totally what the world needed, I shared this to my fellow AI engineers newbies.. honestly with low IQ and dyslexia it is really hard to gain skill in this field... I'll never give up because helpful videos like this are really helping me a lot..

Please check datacenters Nvidia supercomputing ,AI Factories even NVIDIA DGX SuperPOD Please and beyond because there will be more just help

I want a python code that loads a model trained by unsloth. The code runs inference with a simple question without training.

Is there a way to finetune LLM to say I don't know? This is very important for domain-specific LLMs. Thanks.

Have you successfully fine-tuned an LLM to incorporate company-specific knowledge? From my experience, fine-tuning helps models learn the preferred output format for information. I'm particularly interested in approaches for fine-tuning on proprietary data. Additionally, is full fine-tuning feasible for this purpose?

Awesome video and very good learning material both for beginners and more advanced! Thank you!

It's a great material! Thanks! I followed this tutorial, used a quantized version of pixtral and then fine-tuned it using lora with a A100 40GB GPU in colab. However, I still got outofmemory error. Here is my colab. Would you mind taking a look and tweaking it a bit to make it all work? colab.research.google.com/drive/1Q6KnuhgJuGvL8AxhalhrVLInkhOyKCqT?usp=sharing I also encountered some minor errors like stuck at installing flash-attention, therefore I had to use the quantized version but even though it still ran out of GPU VRAM...also when I ran the evaluation, images did not show up but it was a minor issue

This is great and thank you. Also looking forward to Colab integration.

Great ideas here. I used this model (from preprint not your example) to estimate US election outcome. I removed betting markets and polling from the search as they are not representative of voters. Surprising results. Look for "2024’s Electoral Math: The Numbers Beyond the Polls"

Hi Ronan, Love your work. Can you please - make this work inside kaggle notebooks. Cant think of typing or switching windows when we are in 2024 - Can you cover full finetune process (base plus instruction) for different libraries like unsloth, axolotl, torchtune etc. Thanks.

18:55 How will the scaling work when temp is 0? Will it not gice a divded by 0 error?

First!

Hi, do you think if its possible to fine-tune LLaVA One Vision model with traffic data for traffic scene understanding in the case of autonomous vehicles? I am working on a similar project and would love your take on this? Also, if you don't mind sharing, is there any cloud based service that provides low-cost GPUs for fine-tuning these models?