Відео

This was quite an amazing (and unexpected) GPU "deep dive" 🤯. Thanks for the care you put into all of what you share with us!

Flash Attention - AIM Event: colab.research.google.com/drive/1-OPCDWnK3sQQncg6gm0bSMr9Nmx3lAil?usp=sharing Event Slides: www.canva.com/design/DAGXB3D4Yd0/ufpIRKB21NdBjzCwNYJOXA/view?DAGXB3D4Yd0&

Dr Greg and the Wiz, what a dream team! Thanks for your wonderful, truly wonderful efforts. I am certainly getting better following you guys. God bless.

AI engineering +1, but eventually will be interested in getting more into detail and understand LLM engineering better.

very useful! Thank you for your sharing

Dr Greg, you are absolutely incredible teacher. Keep it up!

Replit: replit.com/@replit/Anthropic-Computer-Use Event Slides: www.canva.com/design/DAGWX1fpgkA/AYTV-idbZfIoz-0uoosImQ/view?DAGWX1fpgkA&

hey did you called up the large action model live session?

It was a great video and RLHF was explained perfectly, thanks!

Thank you for boldly diving into these highly technical topics in plain English! Your videos are a gift to the community

Thank you for introducing our VPTQ project. VPTQ is an ongoing project and we welcome suggestions from everyone. We have currently open-sourced the inference code and algorithm code. : D

3. Inference Performance: Thank you both for helping me explain this :) Currently, our inference code is just a very naive inference example based on Torch. We haven't done any optimizations specifically for inference yet. I am currently trying to integrate VPTQ into vLLM. I believe this will greatly enhance the inference speed with vLLM. I am currently discussing this in the vLLM Slack channel and with the sig-quant group. Stay tuned!

2. Entropy: I am very interested in using Shannon entropy and information theory to quantitatively analyze the impact of model quantization. In fact, an unresolved academic issue in the quantization field is how to quantitatively describe the effects of quantization on the final loss, or even model accuracy, especially since there are many nonlinear layers involved. The GPTQ/VPTQ series of algorithms, which are based on second-order optimization, actually have very strong assumptions (as analyzed in the paper WoodFisher). These assumptions simplify the impact of quantization on model loss to the impact on proxy error, thus simplifying the algorithm. I believe this is a more fundamental problem. Currently, I am still conceptualizing and do not yet have a clear idea on how to perform quantitative analysis.

I'll try to address the last few questions from the video: 1. The results of VPTQ on smaller models. I have actually tried compressing smaller models like Qwen2.5 1B-3B, and achieved good results around 3-4 bits. I will release some of these results later. Indeed, on smaller models, compressing the model weights is quite challenging. I am currently working on improvements and will continue to provide updates. Thank you.

VPTQ - Vector Post-Training Quanization: colab.research.google.com/drive/1yItAepbYh9HVs3SEBqtZclc5AY4aJIAa?usp=sharing Event Slides: www.canva.com/design/DAGVuEs7C_s/cohyWXMzy7TD_MzR8-MA_g/view?DAGVuEs7C_s&

Hi Greg and Wiz. Great tutorial. I am actually applying it to my own application. I was wondering what would you sugges to do if the whole document size is large more than 700 pages. It want be passed in the contextual chunking function. If I take the 500 pages around the chhunk, the chaching wont work. Please can you advice? Thanks Saurabh

I'm super happy for this video supernova was on my radar last month.

I am trying to understand the unique concepts of this paper. It sounds like this is a workflow of agents and programmatic validators to synthetically generate DPO data. Is the system self learning as well?

Hey this is quite useful.can you help me in how large action model works? The recent Claude computer or ominiparser and lavauge model or how the rabbit mq works.can you help with the codes refference or implementation.thank you

Great information here! Thanks for making it public. I think you're going to get a sizeable community around you because of these live streams. Q: where in the code is prompt caching evoked?

Calibrated reward: github.com/huggingface/trl/pull/2155/files Mixture of judges: github.com/huggingface/trl/pull/2159/files CGPO Trainer (single task single objective): github.com/huggingface/trl/pull/2190/files Event Slides: www.canva.com/design/DAGVDvGDG54/kEflcFEuGxDKMTYb6Rj2vA/view?DAGVDvGDG54&

at 00:24:15 you give a formula for faithfulness, think it is flawed a bit. Should be (#Claims from the answer which exist in the context) / (#claims in answer). Otherwise there could be >1 result.

Great work here Richard and Gil - loved the demo

Love this guys. Great job!

hello!! thank you a lot for the videos! what is the best way to interact in sort of chat engine of chat loop with a workflow?

Great video, but using text that the model was already trained on is a bad test case

Hi! I really appreciated your video. BTW, I wrote an article titled "SWARMing Conversational AI: Integrating No-Code and Code in Agent-Based Workflows," which you can find online. I would love to hear your feedback on my perspective (SWARM emphasis on blending no-code instructions with hardcoded conversational steps. Thanks! Giorgio

OpenAI Swarm - Multi-Agent: colab.research.google.com/drive/1NumpfFNIPxsyjmruJ3jzyxxX2HY8V0MO?usp=sharing Event Slides: www.canva.com/design/DAGUZ0A-Zpc/uctbkE6-rHzlRfjxVFPAlg/view?DAGUZ0A-Zpc&

Are the slides available?

Congratulations dude

Great video! I’m excited to dive into contextual retrieval next week. When it comes to productionizing hybrid retrieval with BM25, I’m considering using Elasticsearch, any other recommendations? My main concern with hybrid retrieval is the added complexity it brings to the production.

🚨MERT ALERT

Great video and demo as always! I learn much from your content. The contextual retrieval paper said if your corpus is less than 200k tokens, just skip rag and dump the entire corpus into the prompt for every question, and they will cache it (but only for a short time) and just use long context Q&A. I didn’t see them publish any metrics comparing long context to rag, so I take it with a grain of salt. They do want customers to spend as many tokens as possible... But I’m very intrigued at the same time. Maybe you could do a video comparing the two methods? That would be amazing research.

thanks :)

Would the results be even better when combined with semantic chunking? Answer: research.trychroma.com/evaluating-chunking

RAG-ception 0:55 - Context of the contextually generated chunks. Got it...got..it.......got it....ok wait what? Need to watch the whole thing.

Re; Would the results be even better when combined with semantic chunking? For more on Semantic Chunking strategies: research.trychroma.com/evaluating-chunking

Contextual Retrieval: colab.research.google.com/drive/1KGVxiwc2zoY9v6f3IQfs8qJIZeGeMKAq?usp=sharing Event Slides: www.canva.com/design/DAGTv5ofV8g/-wFTZpoCu8yYzseTb_kx2g/view?DAGTv5ofV8g&

Very interesting

Where this complete video, id like to understand this loss fuction and the matrix hessian

Top

GPTQ - AIMS: colab.research.google.com/drive/1iZQ_Byo9F-bM6IGywtb3sN9TIEcE_Mqd?usp=sharing Event Slides: www.canva.com/design/DAGTF8vsIIM/9qbXt5T-pvt-KIHDtC_4nA/view?DAGTF8vsIIM&

Thanks for the great video. Subscribed! Question - We saw here that "similar pairs" were trained where the pair implies a (question , context). Is it possible to get good results by fine-tuning on a "similar questions" dataset i.e. (question1, question2) and the difference between those 2 questions is usually one word/phrase. So question1 would have the full-form of an entity; and question2 the acronym of the same entity. Reason I'm doing this is that I'm storing a mix of questions and contexts in my vector Database. If the user's query matches a question then I look up the corresponding answer (static answer that almost never changes - so no LLM required). If the match is a context instead, then LLM generation takes over.

How do I get a scholarship?

Where does ground truth come from? Is this a human annotated property? I understand the ground truth in RAGAS refers to the correct answer to the question. It's typically used for the context_recall metric. But how to we get this? Human in the loop? LLM generated? More documents from the retrieval? Thank you?

LOVE LOVE LOVE the Snatch Dags meme

lovely

Bros DSE awareness is 0