I agree, I am personally really interested in RAG and see that as the main application that will assist people in their workflows before we see anything else
yes please make the next video with RAG and integrate it and also please can you create for us a video tutorial demonstrating how to build a chatbot that inputs in XLS or CSV format, prompts the user for input, and provides charts as output. using OPENAI API
Thank you so much for this... :). I deal with large number of documents. I find dense retrieval is very bad at it. Let me check this approach and comment back.
Please make a video on how to handle dynamic tabular data in pdf to feed in llm and query on tables data, as tables structure gets messed up when creating vectors.
At the moment, the number of vectors store supports are limited, I think only FAISS supports that. You will need a GPU to run this. In THEORY, it should perform better than dense retrieval but probably need better evals.
Super interesting. I want to use dspy with ragatouille/colbert2 for embedding and retrieval. I’d like to use llama index with a different vectordb, e.g. chromadb, pinecone, or qdrant. I want to use ollama with llama 3 to then summarise my retrieved rag data, and combine with some basic analysis of my own dataset. How feasible is that now? I assume that i can use dspy to finetrain on my specific analysis cases if necessary.
As I dive into the world of storytelling and creative expression, VideoGPT emerged as my trusted ally, subtly enhancing the quality of my videos without stealing the spotlight.
Whenever I am doing Rag.search ,I am getting the name of the document in contents rather than answers for the query . how do I solve it ? Please kindly help
Thanks for the video and sharing, I can't seem to pass the loader.load_data("Orca_paper.pdf") line in the colab notebook. The load_data call complains about 'str' has no 'name' attribute.
@engineerprompt Is there a reason why you design your videos so that they must be viewed on a large screen? The font used on the diagram slides is obviously completely unreadable on a phone.
If you are interested in leanring more about Advanced RAG Course, signup here: tally.so/r/3y9bb0
that's definitely a hot topic
by 51 seconds we have the most direct explanation of embedding on youtube.
Thanks for this. There is a lot of obsession over LLMs but I RAG has huge room for innovation that will multiply the performance of ai applications.
I agree, I am personally really interested in RAG and see that as the main application that will assist people in their workflows before we see anything else
nice! Yes another video which uses this in langchain would be cool!
Yes please!
Thanks, would like to see a combination of colbert and langchain optimal chunking method.
me too please
yes please make the next video with RAG and integrate it and also please can you create for us a video tutorial demonstrating how to build a chatbot that inputs in XLS or CSV format, prompts the user for input, and provides charts as output. using OPENAI API
Hii have you figured out solutions for this ??
@@utkarshtripathi9118 Still m working on it
Thank you for the great walkthroughs and insights! RAGatouille interface looks great, can't wait to mess around with it
thanks, have fun :)
Nicely explained! also, wanted to know about time comparision between embedding retrievers and colBERT
From my experience, colBERT is usually faster.
Thanks for the clear and concise explanation.! What metrics can be used to evaluate the output of these models.?
Thank you so much for this... :). I deal with large number of documents. I find dense retrieval is very bad at it. Let me check this approach and comment back.
Please do share your experience. Would love to see what you find.
Please make a video on Rag with a UI where input is a file pdf or csv + Colbert behind the scenes
will do!
Please make a video on how to handle dynamic tabular data in pdf to feed in llm and query on tables data, as tables structure gets messed up when creating vectors.
So what's the disadvantage of using CoBERTv2? Or are you saying it's strictly better?
At the moment, the number of vectors store supports are limited, I think only FAISS supports that. You will need a GPU to run this. In THEORY, it should perform better than dense retrieval but probably need better evals.
Super interesting. I want to use dspy with ragatouille/colbert2 for embedding and retrieval. I’d like to use llama index with a different vectordb, e.g. chromadb, pinecone, or qdrant. I want to use ollama with llama 3 to then summarise my retrieved rag data, and combine with some basic analysis of my own dataset. How feasible is that now? I assume that i can use dspy to finetrain on my specific analysis cases if necessary.
Can you discuss newly pdf handling with tables & docx files parser....
Great content, thanks! Also curious what tool did you use to come up with such beautiful graphs on the "blackboard"
I use excalidraw.com
Go Ahead Sir..... ❤
thank you :)
Wait for the second example you used GPT4 for embeddings instead of ada? Did I miss something?
Its the tokenizer not the LLM. Probably can replace that with tiktoken package to get tokens.
As I dive into the world of storytelling and creative expression, VideoGPT emerged as my trusted ally, subtly enhancing the quality of my videos without stealing the spotlight.
Whenever I am doing Rag.search ,I am getting the name of the document in contents rather than answers for the query . how do I solve it ? Please kindly help
Thanks for the video and sharing, I can't seem to pass the loader.load_data("Orca_paper.pdf") line in the colab notebook. The load_data call complains about 'str' has no 'name' attribute.
fixed, you need documents = loader.load_data(pathlib.Path("Orca_paper.pdf")), the load_data expects a Path object, not str.
BTW, the load_data() method by default parses the pdf page by page into multiple documents, in case you are wondering like I do.
Can you discuss on tables in Pdf files for RAG & other .docx files loader as pdf parser but some os there......
@engineerprompt Is there a reason why you design your videos so that they must be viewed on a large screen? The font used on the diagram slides is obviously completely unreadable on a phone.
How can we use this with Chroma ?
So We can try this with local gpt?
Gread job !!
Please bring next video fast
hi. please help me. how to create custom model from many pdfs in Persian language? tank you.
Nice!