Best tutorial I've ever seen in a long time, maybe ever. Timing, sequence, content, logic, context... everything is right in your video. Thank YOU and congrats, you are smart as hell.
I just wanted to say that this video inspired me to setup my own RAG. I got some help from a friend with some parts, but I've been working on adding more functionality. My current iteration has a menu that has database management and chatbot options. Database management lets me create, update, and delete databases. The chatbot option lets me choose which databases to use before loading the LLM. I also have added graceful interrupt handling. Thanks for the tutorial that provided me with a jumping off point.
Oh man.. by far the best tutorial on the subject.. finally someone using pdf and explaining the entire process! You should do a more in-depth series on this...
One of the questions that I was asking myself with pdf.. do you clean the pdf before doing the embeddings .. or this is something that you can resolve by customizing the prompt? What would be a good way to do semantic search after using pgvector..? I am still struggling with those answers
@@fabsync Yeah I've had a lot of people ask about cleaning the PDFs too. I think if you have PDFs that have certain structural challenges, I'd probably recommend to find a way to clean/augment it for your workflow. And LLM prompt can only go so far, and cleaning noise from the data will always help.
@@fabsync Yep, do your best to make sure the incoming data is as clean as possible. In my projects, some of the PDFs were OCR'd many years ago with inferior tools. I re-OCR them and compare the original text with the newly-OCR'd text and see if there is an improvement with the known word count. If so, then I replace the text. I then do 'smudged-lens' image recognition on the document (ultra-low resolution of the document) and K-means clustering to determine an unsupervised classification of that document, similar to how you see a phone bill or electric bill from 10 feet away. You can't make out the individual letters but the overall image in your head tells you how to classify the document (phone bill vs electric bill) 😀
@@houstonfirefox Do you have a github, video, or anything that goes into how you are cleaning up the pdfs? I need to do some cleaning up of my pdfs and am very interested in the details of your process. Which programs/python libraries/etc did you use to do this?
It's hard to find such high quality videos on China's Beep, but you've done it, thank you so much for your selflessness. Great talk, looking forward to the next video. Thanks again, you did a great job!
Simplifying a complex topic for a diverse set of users requires an amazing level of clarity of thought, knowledge and communication skills, which you have demonstrated in this video. Congratulations! Here are some items on my wish list for you when you can get to it. 1. Ability for users to pick among a selected list of open-source LLMs. A list that users can keep it updated. 2. build a local RAG application for getting insights from personal tabular data, which stored in multiple formats e.g. excel/google sheets, PDF tables
Thanks for your comment, I'm really glad to hear it was helpful. I appreciate you sharing the feedback and suggestions as well, I've added these items to my list of ideas for future videos :)
**Edit:** Just got to 10:25 and realized I was writing an answer to this 😀Yeah, it's just an outstanding tutorial in all aspects. Love the content of this video a lot! It's so well prepared - fantastic job. Just by watching it, I might have spotted a weak point. No biggy, but something that can be improved. The unique chunk ids do not represent their actual content, but rather the position of their content. When the content in an already existing position is changing, the chunk id won't recognise this change. What you can do is adding a checksum of the content to your chunk id. Something like data/monopoly.pdf:1:1:{chunk-content-checksum}. You could even leave out the index, and decide only based on the checksum if you keep / remove / add chunks for a page. In that line of thought, you could even remove all the other id parts, just ending up with something like data:{chunk-content-checksum}. The cleanup process for outdated chunks is a little more complicated doing it like this, but doable nontheless.
Sounds good but I think I'd keep the chunk ID just to be on the safe side. There is the infinitely small chance that the checksums of two different PDFs could potentially compute to the same value. Admittedly it's an extremely rare occurrence but just the fact that it Could happen is enough to make me want to leave the Chunk ID in place 😉
Hey, thanks so much for the kind words! Really glad you found the tutorial helpful. 😊 Using a content-based checksum is a clever idea that could definitely improve change detection. But then without the page (position) of the content, I do think it will be challenging to figure out whether a "new checksum" means you need to add a chunk vs replace a chunk. That's why I feel you'd probably need both a positional index (ID) and a checksum. Great thoughts and ideas. Thank you for sharing them :)
@@pixegami Meanwhile I adapted your tutorial, built a local RAG on my machine + a web-based UI. It all works pretty well. For the document update part (and prob. some other things in the future) I added a command concept. When I type "--refresh" the database gets cleared and all documents are loaded into a fresh database. For me this works well, and it saves all the efforts for implementing more sophisticated updates.
@@pixegami Suggestion for a future tutorial, if you like: With my little python skills I found it impossible to query the collection asynchronously - found some hints how to do this, but it's all too advanced. Would be really interested in learning & understanding how this can be done.
To update the chromadb data for PDF chunks whose data has changed, store the PDF document contents hash in the metadata field. In addition to adding IDs that don't already exist, select records whose metadata.hash has changed and update these records, using collection.update()
That was the most useful video I've seen on the topic (and I watched quite a lot). I didn't realise that the quality of the embedding is so important. I have one working code for local pdf ai, but I wasn't very impressed by the results. That explains why. Thank you for the great content. I'd love to see other uses of local LLMs.
Glad you liked it! Thanks for commenting and for sharing your experience. And absolutely - when building apps with LLM (or any kind of ML/AI technology), the quality of the data and the index is really non-negotiable if you want to have high-quality results.
Thank you very much! I've started my RAG using your vids. Of course, much of your code needed to be updated, but it was simple even given my zero knowledge of Python.
BTW (ByeTheWay): I used the OpenAI Embeddings model="text-embedding-3-large" and obtained very similar results to your demo query about Monopoly. I first used Ollama 'llama3', but then retested with Ollama 'mistra:latest'. Surprisingly, the 'mistral' results were better than the ''llama3' !?!?! All I can say now is "G'Day Mate" and thank you again!
Your content is amazing! Keep it going. I would like to see the continuation of this video in terms of how to upload and automate the workflow in the cloud AWS and how to integrate the chat interface with telegram bot
Glad you liked it, and thanks for the suggestions. My next video will be focused on how to deploy this to the cloud - but I hadn't thought about the Telegram bot idea before, I will look up how to do that.
correct and improve: The issue I have with the Rag is that when I ask about something in a document that I know doesn't exist, the AI still provides a response, even though I would prefer it not to.
@@pixegami Aweeeeeeeesome, Just want to slightly change the knowledge graph based on pdf,txt (own data). Sorry for not elaborating, but too much own data makes it difficult to find connections between many sources.
Thank you so much for the content👍🏼 very well explained! Would be great to see a use case of using autogen multi-agent approach to enhance RAG response.
My laptop is woefully underpowered, and I had to update a couple of the dependencies, but I was able to get mistral to tell me about gears. Hoping I can get these "6 simple machines" pdfs to accomplish similar. Thank you so much for this project. :)
Yeah, that's awesome you got it working! Laptops can definitely struggle with local LLMs, but it's great you found a workaround. Those "6 simple machines" PDFs sound perfect for this kind of project. Have fun exploring that - I'd be curious to hear how it goes! And thanks, glad you're enjoying the project :)
Great video and nicely scripted. Thanks for the excellent effort. I find that nomic 1.5 is pretty good for embedding and lightweight as well. I did not do actual performance metric based analysis of that but actual recall and precision testing is pretty impressive with 768 dimensions only.
He he he... tests are easy. I was wondering how to do those. Prompt: State several facts about the data and construct a question that asks for each fact. Create tests that look for the wrong answer... Give me 50 of each... Give me some examples of boundary conditions... Formatting... In an hour I will have fat stack of tests that would normally take a day a day to create. This is awesome :)
I simply love the hole video! for the next Rag tutorial can you make a deploy on a azure cloud or any other cloud, just to see in depth how this works! thanks so mutch for the content man!
Thanks for sharing and I can have a grasp of the concept. If possible, it will be deeply appreciated if you can show how to do it in the cloud. Thanks.
This is pretty good. I was wondering how I could integrate this with my current python scripts for my AI Calling Agent, so if someone wanted to call the number, they could chat with the PDF.
I think that certainly should be possible, but it's quite complicated (I haven't done anything like that before myself). You'd probably need something to hook up a phone number/service to an app that can transcribe the text in real like (like what Alexa or Siri does), then have an agent to figure out what to do with that interaction. And eventually hook it up to the RAG app. After that, you'll need to seriously think about guard-rails for the agent, otherwise you could end up with it getting your business into trouble. An example of this is when Air Canada's chatbot promised a customer a discount that wasn't available: www.bbc.com/travel/article/20240222-air-canada-chatbot-misinformation-what-travellers-should-know
Thanks for your good work. It was such a crisp and clear video straight to the point. I loved it. Keep doing the good work. I would like also see how can we deploy such models in production. If you can do one such video next time, it would be really informative. Thanks-Ramana
Great video, love the stack you're using also. I wonder if its possible to do a 2 pass on the LLM. Once to help the user formulate a better question for the RAG vector lookup, and secondly running that through the actual RAG and LLM regular. You would need 2 different system prompts, and the first one would need an overview of the contents of the RAG db, so the LLM can read the users question, and then add key words that the RAG system can use to have a better chance at finding the right embeddings.
Yeah, that's a really interesting idea. A two-pass approach could potentially improve the result. I haven't implemented this specific approach myself, but it sounds similar to query expansion techniques used in information retrieval. It could be particularly useful for handling ambiguous or brief user queries. One consideration would be the additional latency from the extra LLM call. But for use cases where accuracy is critical, it might be worth the trade-off.
Thank you for this tutorial! It went into more detail than most. I have questions. In all the tutorial i have watched, there is always a small dataset used (a few games instructions in yours). How big can the dataset be? What if I have 1000s of PDFs? Will a RAG give less accurate answers in this case and are there other things to consider when dealing with larger datasets?
Thank you for the fantastic tutorial! It was incredibly helpful and well-explained. I was wondering if you have any plans to release a video on fine-tuning this project using techniques like RLHF? It would be great to see your insights on that aspect as well.
Thank you! Glad you enjoyed the video. I've noted the suggestion about fine-tuning-I hadn't considered it yet, but thanks for sharing that idea with me.
I would like it not only to be able to read PDFs but also if the final information was 'weak' or missing information, it would do an internet search on the topic provided in the 'query' and complete the final answer with the collected data. Do you think this is feasible to do? I thought about using agents for this, like crewai. I looked but couldn't find any tutorial that used both methods.
Cursory glance says - add a hashing function to the chunk metadata, this way the chunk should have a unique identifier (MD5, SHA, Etc) if anything changes the hash will also change. Then its just simple logic to validate current chunk page.index against an existing one's hash. If its different, overwrite. If its not, dont waste the cycles. In practice, I am not 100% sure that this would be the approach but at least the theory here should be pretty on point for identifying changes with few compute cycles.
Yup! I think that's probably the way I'd do it too. If there's too documents and you need to scale it, then I guess you can hash the entire document as well first to narrow the search space each time.
@@pixegami I just want to add that I would apply the hash to the entire page and not the chunk. A page can be edited in a way where the content is shorter than previous version, thereby causing the number of chunks to be less that what it previously was. And I would also remove all chunks belonging to the said page before adding new chunks, so as not to have an orphaned chunk from the previous lengthier page.
Gracia amigo eres un crack, me ayudo mucho en mi proyecto, lo pude implementar para un chat con django, ademas logre poder agregar las imagenes y tambien me las toma encuenta
Great video! Love the way you simplified the concept. Are you thinking of making videos on the topics fine-tuning, function-calling, Agents? If so, that would be a great series of videos. I am sure all your subscribers including me would greatly benefit from them. Meanwhile, can you share any resources on these topics? (fine-tuning, function-calling, Agents)
Yeah, thanks! Glad you found it helpful. I haven't dived deep into fine-tuning or Agents yet, but function calling is definitely on my radar. For now, here are some solid resources: - Function calling: OpenAI's guide (platform.openai.com/docs/guides/gpt/function-calling) - Agents: Langchain's overview: python.langchain.com/docs/how_to/#agents I'll keep your suggestions in mind for future videos. Appreciate the input!
Thanks for the feedback! That's a great suggestion - I'll definitely keep that in mind for future videos. Always looking to improve the viewing experience.
Thank you for the tutorial and the code! Two questions: 1. How can we improve the chunking part? 2. How can we create a derivative model of llama3 or similar that includes our embeddings? And how do we export and import in multiple machines running ollama? Thanks!
Hey, I have a question. I tried to load a fairly large PDF (100 pages) into the database (approx. 400 documents). However the add_to_chroma function seems to be excruciatingly slow. The output from ollama shows that the embeddings only get requested once every two seconds or so. There is also no CPU or GPU load on my system when this process is running. Is there any way to improve this? Thank's already
This is most definitely because of the time it takes to embed each page (since you mentioned embeddings get requested once every two seconds). Your Ollama model might not be able to fully leverage your hardware, which is potentially why your don't see your CPU/GPU load rise up. You could experiment by switching this to use an online embedding API (like OpenAI or AWS Bedrock) and see if it's faster. Or you could double check to see if Ollama is using your GPU correctly (github.com/ollama/ollama/blob/main/docs/gpu.md)
My first RAG project actually uses OpenAI embeddings: ua-cam.com/video/tcqEUSNCn8I/v-deo.html Here is the documentation and code examples from Langchain: python.langchain.com/docs/integrations/text_embedding/openai/
hi, it give me this problem when I run python populate_database.py: Could not load credentials to authenticate with AWS client. Please check that credentials in the specified profile name are valid. Bedrock error: The config profile (default) could not be found (type=value_error)
You'd have to change all the LLM functions to be cloud based (e.g. AWS Bedrock or OpenAI), wrap the app in an API (like FastAPI) and Docker, and deploy it to the cloud (probably as a Lambda function). I'm working on a video about that now, so stay tuned :)
This is an impressive setup! I'm currently using Weaviate as my Vector DB along with Open AI Models, and it's working really well for handling PDFs, Docs, PPTs, and even Outlook email files. However, I've been struggling to integrate Excel and CSV files into my Knowledge Base. For small Excel files, the vector approach seems fine, but it's challenging for larger ones. I'd love to get your input on how to build a system that incorporates Excel files along with the other formats. I've considered using something like PandasGPT for handling the Excel and CSV files and the traditional RAG approach for the remaining file types (PDFs, Docs, etc.). Perhaps adding an agent as the first layer to determine where to direct the query (to the RAG model or PandasGPT) would be a good idea? What are your thoughts on this?
Thanks for your comment and for sharing your challenges and ideas. I think if you are mixing free-form text (like documents) and something more traditionally queryable (like a DB), it does make sense to engineer some more modality into your app (like what you suggested). I haven't explored that far myself so I can't share anything useful yet. But I'll be sure to keep it in mind for future videos. Good luck with your project!
On model param size, 7B models are enough. Not related to this video, but I'm using Llama3 8B with OpenWebUI's RAG and it works but it sometimes have problems to refer to correct document while giving correct answer (it will hallucinate document name), but its how its RAG implementation are.
Thanks! Glad you liked it. If you just want to **add** new data, the chapter on updating the database should already cover this. You just need to add new files into the folder, and run the `populate_database` command again. Any pages/docs on in the database will be added. But if you meant updating existing pages/segments in the existing data, then yes I'll have to make a video/tutorial about that :)
Yeah, that's a smart approach! Adding a modified date to the metadata could definitely help track changes. You could then use that date to trigger re-embedding for only the updated chunks.
I am just a layman and I want to know if text splitters count the characters, words or sentences for a given chunk size? And if the text splitters can identify sentences or paragraphs in text?
Yup, the splitter should attempt to do that. Here's the documentation: python.langchain.com/v0.1/docs/modules/data_connection/document_transformers/recursive_text_splitter/ "It is parameterized by a list of characters. It tries to split on them in order until the chunks are small enough. The default list is ["
Thank you for the tutorial, I guess it should be a better way to create chunks, not only by character count because it cuts the paragraph in disruptive way. May be another library to split in paragraphs or at least end of the sentence?
The Recursive Text Splitter actually attempts to do what you suggest: python.langchain.com/v0.1/docs/modules/data_connection/document_transformers/recursive_text_splitter/ "It tries to split on them in order until the chunks are small enough. The default list is ["
Great video! Successfully implemented RAG for the first time, so touching. Subscribed to the channel already! In the video, you mentioned handling document updates. Do you have plans to cover this topic in the future? I'm really interested about it! Also, is "ticket_to_ride" and "monopoly" sharing the same database in example code? What if I don't want them to share? Is there a way to handle that?
Awesome! Glad to hear about your successful RAG project, well done! I've had a lot of folks ask about vector database updates, so it's something I definitely want to cover. If you want to store different pieces of data in different databases, then I recommend put another layer of logic on top of the document loading (and querying). Have each folder use a different database (named after each folder), then add another LLM layer to interpret the question, and map it to which database it should query.
You'd probably need a way to store/manage memory and use it as part of the next prompt. I haven't explored this much myself, but it's a topic I'm interested to look into as well, so thanks for the comment :)
Best tutorial I've ever seen in a long time, maybe ever. Timing, sequence, content, logic, context... everything is right in your video. Thank YOU and congrats, you are smart as hell.
agreed.
Wow, thanks for your comment. I really appreciate it, and I'm glad you liked the video.
It's hard to find such high quality videos which is to the point with simplification in all the aspects. Great work !!!
I just wanted to say that this video inspired me to setup my own RAG. I got some help from a friend with some parts, but I've been working on adding more functionality. My current iteration has a menu that has database management and chatbot options. Database management lets me create, update, and delete databases. The chatbot option lets me choose which databases to use before loading the LLM. I also have added graceful interrupt handling. Thanks for the tutorial that provided me with a jumping off point.
And everything is being run locally using HuggingFace and Ollama.
@@jonuldrick yes correct. I am also trying same like you
Oh man.. by far the best tutorial on the subject.. finally someone using pdf and explaining the entire process! You should do a more in-depth series on this...
Thank you for the feedback :) Looks like with the interest this topic has received, I'm definitely keen to dive into it a bit deeper.
One of the questions that I was asking myself with pdf.. do you clean the pdf before doing the embeddings .. or this is something that you can resolve by customizing the prompt?
What would be a good way to do semantic search after using pgvector..? I am still struggling with those answers
@@fabsync Yeah I've had a lot of people ask about cleaning the PDFs too. I think if you have PDFs that have certain structural challenges, I'd probably recommend to find a way to clean/augment it for your workflow.
And LLM prompt can only go so far, and cleaning noise from the data will always help.
@@fabsync Yep, do your best to make sure the incoming data is as clean as possible. In my projects, some of the PDFs were OCR'd many years ago with inferior tools. I re-OCR them and compare the original text with the newly-OCR'd text and see if there is an improvement with the known word count. If so, then I replace the text. I then do 'smudged-lens' image recognition on the document (ultra-low resolution of the document) and K-means clustering to determine an unsupervised classification of that document, similar to how you see a phone bill or electric bill from 10 feet away. You can't make out the individual letters but the overall image in your head tells you how to classify the document (phone bill vs electric bill) 😀
@@houstonfirefox Do you have a github, video, or anything that goes into how you are cleaning up the pdfs? I need to do some cleaning up of my pdfs and am very interested in the details of your process. Which programs/python libraries/etc did you use to do this?
It's hard to find such high quality videos on China's Beep, but you've done it, thank you so much for your selflessness. Great talk, looking forward to the next video. Thanks again, you did a great job!
Thank you! Glad you enjoyed it!
Simplifying a complex topic for a diverse set of users requires an amazing level of clarity of thought, knowledge and communication skills, which you have demonstrated in this video. Congratulations! Here are some items on my wish list for you when you can get to it. 1. Ability for users to pick among a selected list of open-source LLMs. A list that users can keep it updated. 2. build a local RAG application for getting insights from personal tabular data, which stored in multiple formats e.g. excel/google sheets, PDF tables
Thanks for your comment, I'm really glad to hear it was helpful. I appreciate you sharing the feedback and suggestions as well, I've added these items to my list of ideas for future videos :)
One of the best tutorials I’ve watched on UA-cam in a while. The non local RAG video was also great! Well done and thank you for the information!
Thanks! Really glad you found both videos helpful.
Thanks! Really glad you found both videos helpful.
**Edit:** Just got to 10:25 and realized I was writing an answer to this 😀Yeah, it's just an outstanding tutorial in all aspects.
Love the content of this video a lot! It's so well prepared - fantastic job. Just by watching it, I might have spotted a weak point. No biggy, but something that can be improved. The unique chunk ids do not represent their actual content, but rather the position of their content. When the content in an already existing position is changing, the chunk id won't recognise this change. What you can do is adding a checksum of the content to your chunk id. Something like data/monopoly.pdf:1:1:{chunk-content-checksum}. You could even leave out the index, and decide only based on the checksum if you keep / remove / add chunks for a page. In that line of thought, you could even remove all the other id parts, just ending up with something like data:{chunk-content-checksum}. The cleanup process for outdated chunks is a little more complicated doing it like this, but doable nontheless.
Sounds good but I think I'd keep the chunk ID just to be on the safe side. There is the infinitely small chance that the checksums of two different PDFs could potentially compute to the same value. Admittedly it's an extremely rare occurrence but just the fact that it Could happen is enough to make me want to leave the Chunk ID in place 😉
Hey, thanks so much for the kind words! Really glad you found the tutorial helpful. 😊
Using a content-based checksum is a clever idea that could definitely improve change detection. But then without the page (position) of the content, I do think it will be challenging to figure out whether a "new checksum" means you need to add a chunk vs replace a chunk.
That's why I feel you'd probably need both a positional index (ID) and a checksum. Great thoughts and ideas. Thank you for sharing them :)
@@pixegami Meanwhile I adapted your tutorial, built a local RAG on my machine + a web-based UI. It all works pretty well. For the document update part (and prob. some other things in the future) I added a command concept. When I type "--refresh" the database gets cleared and all documents are loaded into a fresh database. For me this works well, and it saves all the efforts for implementing more sophisticated updates.
@@pixegami Suggestion for a future tutorial, if you like: With my little python skills I found it impossible to query the collection asynchronously - found some hints how to do this, but it's all too advanced. Would be really interested in learning & understanding how this can be done.
this is the best RAG tutorial I have come across on youtube, thank you so much man💪
Thank you! I appreciate it!
To update the chromadb data for PDF chunks whose data has changed, store the PDF document contents hash in the metadata field. In addition to adding IDs that don't already exist, select records whose metadata.hash has changed and update these records, using collection.update()
Upvote this guys🎉
That was the most useful video I've seen on the topic (and I watched quite a lot). I didn't realise that the quality of the embedding is so important. I have one working code for local pdf ai, but I wasn't very impressed by the results. That explains why. Thank you for the great content. I'd love to see other uses of local LLMs.
Glad you liked it! Thanks for commenting and for sharing your experience.
And absolutely - when building apps with LLM (or any kind of ML/AI technology), the quality of the data and the index is really non-negotiable if you want to have high-quality results.
which is the best freely available embedder? any help would be really appreciated
Thank you very much! I've started my RAG using your vids. Of course, much of your code needed to be updated, but it was simple even given my zero knowledge of Python.
You may could share the updates to us 😅
Nice work, glad you got it working!
I've watched a few of your videos and I didn't know which one to comment first. And congratulate you. Great content and even better style.
Thanks so much! I'm really glad you're enjoying the content and style. It means a lot to hear that!
BTW (ByeTheWay): I used the OpenAI Embeddings model="text-embedding-3-large" and obtained very similar results to your demo query about Monopoly. I first used Ollama 'llama3', but then retested with Ollama 'mistra:latest'. Surprisingly, the 'mistral' results were better than the ''llama3' !?!?! All I can say now is "G'Day Mate" and thank you again!
llama3 vs mistral? That's a weighty comment there mate...
@@vslabs-za 🤣🤣🤣
This is genuinely a crazy good tutorial, kudos to you man
Thank you! Really glad you found it helpful.
For evaluation, use RAGAs and Langsmith.
There is also an SDK for azure which does same things as RAGAs and Langsmith.
Oh, thanks for the recommendation. I'll have to take a look into that.
The best RAG tutorial so far, you've introduced concepts ave not seen in other similar tutorials
Your content is amazing! Keep it going. I would like to see the continuation of this video in terms of how to upload and automate the workflow in the cloud AWS and how to integrate the chat interface with telegram bot
Glad you liked it, and thanks for the suggestions. My next video will be focused on how to deploy this to the cloud - but I hadn't thought about the Telegram bot idea before, I will look up how to do that.
correct and improve:
The issue I have with the Rag is that when I ask about something in a document that I know doesn't exist, the AI still provides a response, even though I would prefer it not to.
After searching 100s of videos journey ends here. 😍Please would you make a tutorial making a knowledge graph using Ollama?
Thanks, glad your journey came to an end :) Thanks for the suggestion - I've added the idea to my list :)
@@pixegami Aweeeeeeeesome, Just want to slightly change the knowledge graph based on pdf,txt (own data). Sorry for not elaborating, but too much own data makes it difficult to find connections between many sources.
??/
Pure gold! Thanks for putting all the beads and pieces together so well bro!
Thank you so much for the content👍🏼 very well explained! Would be great to see a use case of using autogen multi-agent approach to enhance RAG response.
Glad you liked it, thank you! And thanks for the suggestion and project idea :)
My laptop is woefully underpowered, and I had to update a couple of the dependencies, but I was able to get mistral to tell me about gears. Hoping I can get these "6 simple machines" pdfs to accomplish similar.
Thank you so much for this project. :)
Yeah, that's awesome you got it working! Laptops can definitely struggle with local LLMs, but it's great you found a workaround. Those "6 simple machines" PDFs sound perfect for this kind of project. Have fun exploring that - I'd be curious to hear how it goes! And thanks, glad you're enjoying the project :)
Deploying the model on the cloud would definitely be interesting! thank you for the video :D
Thanks, great video! To avoid duplicate documents and to look for updated documents I use a document hash to check.
Yup! Great idea :)
To run this project we need to install Ollama -> install nomic-embed-text and mistral by ollama pull
Very nice, I wish I had this guide few weeks ago, had to learn it the hard way xD
You got there in the end :)
Great video and nicely scripted. Thanks for the excellent effort.
I find that nomic 1.5 is pretty good for embedding and lightweight as well. I did not do actual performance metric based analysis of that but actual recall and precision testing is pretty impressive with 768 dimensions only.
Thank you! Glad nomic text worked well for your use case :)
He he he... tests are easy. I was wondering how to do those.
Prompt:
State several facts about the data and construct a question that asks for each fact.
Create tests that look for the wrong answer...
Give me 50 of each...
Give me some examples of boundary conditions...
Formatting...
In an hour I will have fat stack of tests that would normally take a day a day to create.
This is awesome :)
I simply love the hole video!
for the next Rag tutorial can you make a deploy on a azure cloud or any other cloud, just to see in depth how this works!
thanks so mutch for the content man!
My upcoming video is actually about how to deploy a RAG app like this to the AWS cloud :) Stay tuned!
Thanks for sharing and I can have a grasp of the concept. If possible, it will be deeply appreciated if you can show how to do it in the cloud. Thanks.
Stay tuned for my next video! ua-cam.com/video/ldFONBo2CR0/v-deo.html
This is pretty good. I was wondering how I could integrate this with my current python scripts for my AI Calling Agent, so if someone wanted to call the number, they could chat with the PDF.
I think that certainly should be possible, but it's quite complicated (I haven't done anything like that before myself).
You'd probably need something to hook up a phone number/service to an app that can transcribe the text in real like (like what Alexa or Siri does), then have an agent to figure out what to do with that interaction. And eventually hook it up to the RAG app.
After that, you'll need to seriously think about guard-rails for the agent, otherwise you could end up with it getting your business into trouble. An example of this is when Air Canada's chatbot promised a customer a discount that wasn't available: www.bbc.com/travel/article/20240222-air-canada-chatbot-misinformation-what-travellers-should-know
Thanks for the whole tutorial, I would suggest adding speed at the queries of the vector database with PostgreSQL with pgvector, pg_embeddings plugin.
For the unit tests you can use tools or grammars to limit the output and that way you can fix some issues that you are showing
Good idea. I haven't actually explored testing LLM output in detail yet, and I think it will be a fascinating topic.
Suuuuuper helpful. I need to test this for a work idea. thank you!
You're welcome!
Thank You for: "Taking it up a notch ..." or 2 or 3 'notches' !
Recently discovered your channel 🎉 , subscribed 😊 keep up the awesome content
Thank you! Welcome to the channel!
This is the best RAG tutorial on youtube, Thanks for the Video, you got a new Subscriber 🎉
Thanks so much! Really glad you found it helpful. Welcome aboard! 😊
This is the best practical tutorial came across llms, RAG, langchain. Also can you make one about agents and their use
Thanks! Really glad you found it helpful. Agents are definitely an interesting topic - I'll keep that in mind for a future video.
Thanks for your good work. It was such a crisp and clear video straight to the point. I loved it. Keep doing the good work. I would like also see how can we deploy such models in production. If you can do one such video next time, it would be really informative. Thanks-Ramana
So it is just an advanced ctrl+f ?
Yes, that's one way to think about it. Still, incredibly powerful.
When a company has 10,000 documents. Trust me this shit is useful and will be the future. Probably Microsoft will sell it as Azure extension option.
Good usecase - alternative for FAQ/ help pages for websites/ applications . Will give it a shot
Semantic now
😂
your tutorials are very simple-to-understand. could you please do a tutorial on reinforcement learning from human feedback?
Excellent one, man! Thanks so much!!!
Very very useful and so much well explained ! Thanks.
Thank you!
Crystal clear. Great video.
Thank you! Glad to hear that :)
This is really good and informative, make a video on deploying application like these to a cloud.
Excellent tutorial. Thanks for uploading such great content!
Great video, love the stack you're using also.
I wonder if its possible to do a 2 pass on the LLM. Once to help the user formulate a better question for the RAG vector lookup, and secondly running that through the actual RAG and LLM regular.
You would need 2 different system prompts, and the first one would need an overview of the contents of the RAG db, so the LLM can read the users question, and then add key words that the RAG system can use to have a better chance at finding the right embeddings.
Yeah, that's a really interesting idea. A two-pass approach could potentially improve the result.
I haven't implemented this specific approach myself, but it sounds similar to query expansion techniques used in information retrieval. It could be particularly useful for handling ambiguous or brief user queries.
One consideration would be the additional latency from the extra LLM call. But for use cases where accuracy is critical, it might be worth the trade-off.
I got this to work with my own data. This was so cool. Thanks!
Awesome! Glad to hear it worked for you :)
I could not get it to work, it won't build a database, neither using bedrock nor ollama, bummer.
@@pmgear yeah, it keeps saying that "from langchain.vectorstores.chroma import Chroma" is depracated
did you use ollama?
재미있는 내용 감사합니다!
네, 감사합니다! 도움이 되었다니 기쁩니다 :)
Great stuff as usual! Could we have a video about how to turn this RAG app into a nice and proper desktop app with a graphic interface? Cheers mate.
Good idea, thanks! I'll note it down as a video idea :)
@@pixegami Thank you for the reply and reactivity! Have a nice day!
@@pixegamiI subbed for the advanced RAG content
Thank you for this tutorial! It went into more detail than most. I have questions. In all the tutorial i have watched, there is always a small dataset used (a few games instructions in yours). How big can the dataset be? What if I have 1000s of PDFs? Will a RAG give less accurate answers in this case and are there other things to consider when dealing with larger datasets?
High quality stuff. Thanks
Glad you liked it!
Smashed the Subscribe button! Awesome content! Looking forward for the next ones.
Thank you! Glad you enjoyed it, and welcome!
Cool. Thank you for the good work.
Thank you for the fantastic tutorial! It was incredibly helpful and well-explained. I was wondering if you have any plans to release a video on fine-tuning this project using techniques like RLHF? It would be great to see your insights on that aspect as well.
Thank you! Glad you enjoyed the video. I've noted the suggestion about fine-tuning-I hadn't considered it yet, but thanks for sharing that idea with me.
I would like it not only to be able to read PDFs but also if the final information was 'weak' or missing information, it would do an internet search on the topic provided in the 'query' and complete the final answer with the collected data. Do you think this is feasible to do? I thought about using agents for this, like crewai. I looked but couldn't find any tutorial that used both methods.
Cursory glance says - add a hashing function to the chunk metadata, this way the chunk should have a unique identifier (MD5, SHA, Etc) if anything changes the hash will also change. Then its just simple logic to validate current chunk page.index against an existing one's hash. If its different, overwrite. If its not, dont waste the cycles.
In practice, I am not 100% sure that this would be the approach but at least the theory here should be pretty on point for identifying changes with few compute cycles.
Yup! I think that's probably the way I'd do it too. If there's too documents and you need to scale it, then I guess you can hash the entire document as well first to narrow the search space each time.
@@pixegami I just want to add that I would apply the hash to the entire page and not the chunk. A page can be edited in a way where the content is shorter than previous version, thereby causing the number of chunks to be less that what it previously was. And I would also remove all chunks belonging to the said page before adding new chunks, so as not to have an orphaned chunk from the previous lengthier page.
Gracia amigo eres un crack, me ayudo mucho en mi proyecto, lo pude implementar para un chat con django, ademas logre poder agregar las imagenes y tambien me las toma encuenta
perfect explanation
Great tutorial! And if you could please do a tutorial on when/how to know data within the documents (pdf or csv etc) has changed?
Thanks for the suggestion :) That's a good idea, I think I'll have to plan it...
Great video! Love the way you simplified the concept. Are you thinking of making videos on the topics fine-tuning, function-calling, Agents? If so, that would be a great series of videos. I am sure all your subscribers including me would greatly benefit from them. Meanwhile, can you share any resources on these topics? (fine-tuning, function-calling, Agents)
Yeah, thanks! Glad you found it helpful. I haven't dived deep into fine-tuning or Agents yet, but function calling is definitely on my radar. For now, here are some solid resources:
- Function calling: OpenAI's guide (platform.openai.com/docs/guides/gpt/function-calling)
- Agents: Langchain's overview: python.langchain.com/docs/how_to/#agents
I'll keep your suggestions in mind for future videos. Appreciate the input!
Great content, thanks for making it!
Glad you enjoyed it!
Hi, thank you for video and info. This video one of the best videos about AI, ML, Rag. Your video was so helpful) thank you again)
Thanks! I'm really glad you found the video helpful and informative!
Using a local embedding model: mxbai-embed-large, got me similar results to your monopoly answer.
Thanks for sharing! I hadn't tried that one yet.
Such a helpful tutorial, thank you!
Glad you enjoyed it!
Good content. Suggestion for the future: zoom in vscode to make test slightly bigger when running full screen.
Thanks for the feedback! That's a great suggestion - I'll definitely keep that in mind for future videos. Always looking to improve the viewing experience.
Yes I would love to know how this can be pushed to the web, as using Ollama would completely change everything.
Do i need to use BedrockEmbeddings? I can't run this because it says I have the wrong AWS credentials. Maybe Im missing something.
Thank you so mcuh, Best tutorial I've ever seen!!
Thanks so much! Really glad you found it helpful! 😊
great tutorial
Great content as always!
Thanks for watching!
Thank you! Could you do a tutorial on how to talk to the data? So based on this create a voice chat with it.
Thx very helpful and good explained ❤
Great tuturial, well explained!!!
Thanks! Glad you found it helpful and easy to follow!
Thank you for the tutorial and the code!
Two questions:
1. How can we improve the chunking part?
2. How can we create a derivative model of llama3 or similar that includes our embeddings? And how do we export and import in multiple machines running ollama?
Thanks!
Hey, I have a question. I tried to load a fairly large PDF (100 pages) into the database (approx. 400 documents). However the add_to_chroma function seems to be excruciatingly slow. The output from ollama shows that the embeddings only get requested once every two seconds or so. There is also no CPU or GPU load on my system when this process is running. Is there any way to improve this? Thank's already
This is most definitely because of the time it takes to embed each page (since you mentioned embeddings get requested once every two seconds). Your Ollama model might not be able to fully leverage your hardware, which is potentially why your don't see your CPU/GPU load rise up.
You could experiment by switching this to use an online embedding API (like OpenAI or AWS Bedrock) and see if it's faster. Or you could double check to see if Ollama is using your GPU correctly (github.com/ollama/ollama/blob/main/docs/gpu.md)
Thanks!
You're welcome! And thank you SO much for the Super Thanks 🤩
Hi Pixe! I was wondering how would you write the get_embedding_function for Chat GPT OPEN AI?
My first RAG project actually uses OpenAI embeddings: ua-cam.com/video/tcqEUSNCn8I/v-deo.html
Here is the documentation and code examples from Langchain: python.langchain.com/docs/integrations/text_embedding/openai/
Very thorough. Best video.
Thanks! Really glad you found it thorough and helpful!
hi, it give me this problem when I run python populate_database.py:
Could not load credentials to authenticate with AWS client. Please check that credentials in the specified profile name are valid. Bedrock error: The config profile (default) could not be found (type=value_error)
same problem, did you solve it and if you did, how?
@@1nd3vthis model requires payment, you need to buy it!
How do you do this NOT running it locally? i.e. using the AWS cloud for pretty much everything (PDF in vector database, Langchain, Bedrock etc...)
You'd have to change all the LLM functions to be cloud based (e.g. AWS Bedrock or OpenAI), wrap the app in an API (like FastAPI) and Docker, and deploy it to the cloud (probably as a Lambda function).
I'm working on a video about that now, so stay tuned :)
@@pixegami Thanks. I'll drop you an email with some ideas
This is an impressive setup! I'm currently using Weaviate as my Vector DB along with Open AI Models, and it's working really well for handling PDFs, Docs, PPTs, and even Outlook email files. However, I've been struggling to integrate Excel and CSV files into my Knowledge Base. For small Excel files, the vector approach seems fine, but it's challenging for larger ones. I'd love to get your input on how to build a system that incorporates Excel files along with the other formats. I've considered using something like PandasGPT for handling the Excel and CSV files and the traditional RAG approach for the remaining file types (PDFs, Docs, etc.). Perhaps adding an agent as the first layer to determine where to direct the query (to the RAG model or PandasGPT) would be a good idea? What are your thoughts on this?
Thanks for your comment and for sharing your challenges and ideas. I think if you are mixing free-form text (like documents) and something more traditionally queryable (like a DB), it does make sense to engineer some more modality into your app (like what you suggested).
I haven't explored that far myself so I can't share anything useful yet. But I'll be sure to keep it in mind for future videos. Good luck with your project!
On model param size, 7B models are enough. Not related to this video, but I'm using Llama3 8B with OpenWebUI's RAG and it works but it sometimes have problems to refer to correct document while giving correct answer (it will hallucinate document name), but its how its RAG implementation are.
Interesting, I haven't tried this with the 7GB models yet. Thanks for sharing!
Hello! Mighty pleased and thank you very much!
Hey there! So glad you enjoyed it! Thanks for the kind words 😊
Amaizing bro! Thank you I request you to tell how to connect the prompt part to ui
Thanks for the suggestion :) This is on my list to work on as well, stay tuned!
Thanks a lot. Great video!!! I want to know how to add new data to the existing database with new unique IDs.
Thanks! Glad you liked it. If you just want to **add** new data, the chapter on updating the database should already cover this. You just need to add new files into the folder, and run the `populate_database` command again. Any pages/docs on in the database will be added.
But if you meant updating existing pages/segments in the existing data, then yes I'll have to make a video/tutorial about that :)
To regenerate modified pdf, just add modified date to meta data and add it to the workflow.
Yeah, that's a smart approach! Adding a modified date to the metadata could definitely help track changes. You could then use that date to trigger re-embedding for only the updated chunks.
Great work! Thank you! Maybe a naive question, what should we change to make it work with LM studio instead?
Exceent video! Thanks. Is it free to use these embedding functions you recommended in the video, like AWS Bedrock or Ollama embedding?
I am just a layman and I want to know if text splitters count the characters, words or sentences for a given chunk size? And if the text splitters can identify sentences or paragraphs in text?
Yup, the splitter should attempt to do that. Here's the documentation: python.langchain.com/v0.1/docs/modules/data_connection/document_transformers/recursive_text_splitter/
"It is parameterized by a list of characters. It tries to split on them in order until the chunks are small enough. The default list is ["
", "
", " ", ""]."
Thank you for the tutorial, I guess it should be a better way to create chunks, not only by character count because it cuts the paragraph in disruptive way. May be another library to split in paragraphs or at least end of the sentence?
The Recursive Text Splitter actually attempts to do what you suggest: python.langchain.com/v0.1/docs/modules/data_connection/document_transformers/recursive_text_splitter/
"It tries to split on them in order until the chunks are small enough. The default list is ["
", "
", " ", ""]."
@@pixegami That's cool.
Excellent 🎉🎉🎉
Thank you! Cheers!
I just subscribed to your channel… very high vids on UA-cam
Thank you! Welcome.
nice video.. can you please share update logic too or make another video if possible.
great video! Thank you! I would be interested how this consent can be deployed to cloud but also scaled.
10:36 updating the content: in kotaemon, there's an option to rebuild the index of a specific file. This is one good option i think.
Oh, that sounds like a great feature.
Great video! Successfully implemented RAG for the first time, so touching. Subscribed to the channel already!
In the video, you mentioned handling document updates. Do you have plans to cover this topic in the future? I'm really interested about it!
Also, is "ticket_to_ride" and "monopoly" sharing the same database in example code? What if I don't want them to share? Is there a way to handle that?
Awesome! Glad to hear about your successful RAG project, well done!
I've had a lot of folks ask about vector database updates, so it's something I definitely want to cover.
If you want to store different pieces of data in different databases, then I recommend put another layer of logic on top of the document loading (and querying). Have each folder use a different database (named after each folder), then add another LLM layer to interpret the question, and map it to which database it should query.
thank you for the video pixegami, question, how will you do to implement follow up questions based on the last answer?
You'd probably need a way to store/manage memory and use it as part of the next prompt. I haven't explored this much myself, but it's a topic I'm interested to look into as well, so thanks for the comment :)
Superb bro 🤩
Thank you!