Just found out your channel and subbed after watching this video. Very good quality video! Keep up the great content creation! I am impressed with your motivation to answer the questions from your comment section. Not every youtube channel cares about answering subscriber questions and doing content around what people comment , so very good job!
Followed your instructions here and installed on Raspberry Pi4. Works but of course painfully slow and chip approach 145 degrees F which slows things down as well. But it works and may try on Pi5. Was using a pdf manual for Viking drill press for document. Have to try something with just text.
Would like to see ollama with openai assistant or similar that can use a file and have assistants do action to the file such as execute code or clean up data etc.
Thanks for doing this: It seems like when one clones the repository, you need to delete everything in the db folder or it will mess up the results of the information you are trying input yourself.
Hi, while setting up Virtual environment I'm getting an error, 7:39 what programs do I need? I have installed Miniconda and it's still no running the line. Thks
Make sure you have installed all the frameworks listed in the requirements.txt file. Rewind the video, he explains how to do it all at once. pip install -r requirements.txt If you have done that, make sure you are working in the correct environment. The same environment you installed the python frameworks in. He explains that as well.
Great video, took me a few tries getting Ubuntu configured (using wsl under windows) and updated before it would build the scripts, but I am very impressed. Thanks for all the effort you put into these videos 😁
Actually using it in linux platform but still I gave the command ollama serve, I'm getting no GPU detected, Please give solution for this ! @@PromptEngineer48
If I delete the cache folder name db , will this effect how the program will ingest the files I supply or will create a new db and then ingest the new files?
Awesome idea. Now use Streamlit or Flask as GUI and the solution is perfect. Thanks for the inspiration. Questions: 1. which OS are you using? 2. python version? 3. do you rather use CPU or GPU? Would an M3 with 128 GB also be sufficient for a quick training / fine tuning of hugging face models up to 20B? Thanks for the answer in advance. Hp
1. Mac OS. Pretty basic. Macbook Air M2, 8 GB 2. python version 3.9 ++ version 3. M3 with 128 GB.. Hmm when you say fine tuning it depends on the model. But here is a rough calculation. If you have a 20B parameter model in 32 bits, then you need 20x32/8 = 80 GB GPU. your system should be able to do the fine tuning. Else please go for 4 bit quantization, then the requirement will reduce by 8 times, now requiring only 80/8 = 10 GB of GPU.
Wonderful, thank you! My dream would be to feed my local model with all my books in epub or pdf format just once and the model will keep a memory of those. Then from there I have so many ideas but not sure if feasible, maybe someone can help? 1) for each book create a mind map of concepts 2) a diagram of how each book is related to the others (citations, same authors, same topic, related concepts) 3) given a question or a topic the system can point me to which book is better to read. Probably impossible at the moment, isn't it?
@@PromptEngineer48 Awesome! Another one could be to automatically find the book category. Since the system has already ingested all the books it already knows the discussed topics in each book and from there it can assign each to an ontology of category and subcategories. Like non-fiction/self-help, fiction/novel, non-fiction/self-help/personal growth, and so on. I have other 100 of these ideas, ping me if you need more :D
I would also be interested in something like this if possible as it would be amazing to do this in a repository of books that you know. If you could do this with a non-trivial number of books, say, 100-150 (from Project Gutenberg), it would be a great application of local LLMs.
About the context window. I have noticed that it cannot exceed over 2k tokens even though Mistral can support up to 8k. From what I have tested so for, it is like the bot identify itself as a GPT-3, is it because of the openai library?
yes. That is because we start everything compatible with OpenAI API. then we shift to opensource APIs. We could instead work for OpenSource APIs for the start. 😁
It would be great if you will create a video where you work on entire project, consisting dozens of files, related to coding adding, editing some feature. Thinking about that, DeepSeek Coder would be a best choice I suppose.
@@PromptEngineer48 Also, will it be necessary to use MEMGPT to store different contexts in separate chat windows? I'm bit frustrated trying to figure out bundle of all required to perform what I have mentioned in initial comment =)
@@PromptEngineer48 Bing came back with following bundle: Yes, you can use **AutoGen**, **MemGPT**, and **DeepSeek Coder** together to chat with an entire project like a Ruby on Rails (RoR) project consisting of dozens of files. But it got concerns regarding token limit: "It might be challenging due to the token limit. Each file would need to be processed separately, and the total number of tokens across all files would need to be within the token limit. If the total exceeds the limit, you might need to truncate or otherwise reduce the size of your input1. Remember, a token in this context is not necessarily a word. It could be as short as one character or as long as one word. For example, “ChatGPT is great!” is encoded into six tokens: [“Chat”, “G”, “PT”, " is", " great", “!”]
Thank you very much, all the videos you have made are great! However, when I was building the local llm, I found that there are multiple frameworks, such as privateGPT, localGPT, langchain, etc. Similarly, I found that there are very many choices for the llm as well as the vector database (e.g., hugging face vs. ollama), which gave me a big headache, and I was wondering if you could make a I was wondering if you could make a video that explains your recommendations for each part of the process of building a rag-based personal local document chat llm?
That would be a very good video but very less relevant as everyday we have so many updates. But this is something I can create. thanks for the idea. Definelty will work on that
@@PromptEngineer48 Thank you for your response! It's true that, as you say, the various programs are moving so fast. Since I'm hoping to be able to build something myself via langchain, I'm starting to work based on your github project, and some of the other out-of-the-box projects (e.g. open-webui, privateGPT, etc.) are just too heavy for me to get into and modify.
My brain is exploding because I was just using your code repository earlier today and then closed everything out. I'm trying to run it again and I'm now getting a bunch of depreciation LangChain warnings. I still get a response but the response is no longer referencing or citing the documents I placed in the folder. Any idea why this would happen?
Thank you a lot for your effort to create an easier way to get up and running. Could you make a video explaining what all of the parts of "requirements.txt" are needed for? I would like to understand how everything works in detail
Great video!! However if i follow the instructions my results are different. I created the source_documents folder an put in another pdf-file. When i then execute the "python3 ingest.py" the ingestion seems to work fine. But when i afterwards exceute the privateGPT.py and start to interact with the llm it still responses to "Think and Grow Rich"-Book.
@@PromptEngineer48 can we not append the pdf files, do we have to keep removing them. when a new file is added and ingested, it should add the document into its cache and should be able to response from both the document isnt it
Thank you for this great content! I was able to get this working on my M1 Mac. I was able to run `python ingest.py` and the `python privtagpt.py` commands. However, when I asked it to summarize the document I had uploaded, it referenced the "Think and Grow Rich" document that you showed in the video, rather than the test document I uploaded to the source_documents folder. How do I clear out the embeddings from the "Think and Grow Rich" document, or clear the chroma db database embeddings completely before running the ingest.py command again?
Thanks, that’s a great video! I’m testing privategpt for some time and I would love to know if you’re experiencing hallucinations from the chat? And yes mistral seems to be a good model but Zephyr and Dolphin seems to give better answers with a little less performance, depending on the context. Therefore, I couldn’t Get rid of some hallucinations, I would say the reliability of information is 45-65%
thank you, sorry but i don't see the old chat interface :-) i have 2 questions how can i remove a ingested document that i don't need? ist it possible to chat with the docker interface ? or is it only for terminal??
1. U can remove the file. Then delete the db folder and __pycache__ folder.. then run python ingest.py again and python privateGPT.py again. 2. Web UI interface is not integrated here right now. But that is some ok my pipe line as well. I am working on that.
Are you using the code from private gpt’s primordial version? What a great work! By the way, now they officially support integration with ollama. You probably wanna make a video about how to set this up.
Does anyone get this warning - UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown (and subsequent crash)
there are research going on. you need to find it from huggingface or ollama .. see the benchmarks and decide which model you want to use. since we have so many options, I cannot name one. but it's a case to case basis.
Hi, thanks for the tutorial. I am able to run the model using Ollama on windows but after everything the project runs and Asks for the Enter a query and if I add the questions it stucks there. In problems I am getting bekow issues such as Import "langchain.chains" could not be resolvedPylance Import "langchain.embeddings" could not be resolvedPylance Import "langchain.callbacks.streaming_stdout" could not be resolvedPylance etc Can you please help me out in understanding this issue and resolving it. Thanks.
How would you do this if you have ollama in docker? Or even cloud gpu like runpod? Or Linode? Seems like a good video for the future as you explain,and teach better than a majority of the creators.
Okay. Will create a video soon on the specific use case that you have mentioned.. I mean by hosting on docker and chatting with docs. Thanks for the comment
I got an error when I typed ollama run mistral. Got a a message saying that ollama command not found. I get this from within the terminal of Visual studio code. But I can run Ollama from the mac's terminal window no problem. Did I forget to do something? Thank you!!!
Hello, i keep getting error ERROR: Could not find a version that satisfies the requirement onnxruntime>=1.14.1 (from chromadb) (from versions: none) ERROR: No matching distribution found for onnxruntime>=1.14.1
I had the error for ChromaDB too. If you Google for them you will probably find an answer. For ChromaDB this solved it for me: - sudo apt install python3-dev - sudo apt-get install build-essential -y
Great Tutorial, Why is it saying loading 235 new documents when it has only one in the source_documents folder. I had 2 in mine and it said 8, wondering why.
Very interesting, easy to follow. I tested this with a music instrument manual, and it wasn't giving accurate answers at all. Any ideas on how to improve this? It's unusable for this type of document. Makes me wonder how accurate it would be with any content. Thank you!
This has been very instructive, thanks! Is there an LLM that's better than Mistral at working with very technical documents, i.e. lengthy API implementation documents? I set up my environment exactly as instructed, got my docs indexes and could get some answers on them. But things get murky when I go very specific, with incorrect or incomplete answers.
Thanks for the great effort. I just have a question. When a file is ingested, then i want to ingest different file, should i delete the db folder first? so not to mix older ingested files with the current one i want to ingest. Also, after ingesting one of my csv files, i asked few questions. Then the answers i was getting were far off and was actually from another source, which i didn't provide, it was from a pdf book about getting rich or something. When i only ingested a csv file with numbers!
If my company has a pre-existing Data Dictionary, is there a way to allow Ollama to integrate it so that it better understands the data I am working with?
I can not update the db, when I ask the agent about the new added document, it still gives answers about the document in this video. It is kind of confusing.
@@PromptEngineer48ok managed to get Conda working. However when I run the python3 ingest.py, I got an error: line6, in from tqdm import tqdm. ModuleNotFoundError: No module named ‘tqdm’
Ok made more progress but now stuck in pymupd. Tried installing it but keep getting message Requirement already satisfied: /opt/anaconda3/envs/privategpt/lib/python3.11/site-packages
i am facing thiis error conda : The term 'conda' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. At line:1 char:1 + conda create -n private1 python=3.11 + ~~~~~ + CategoryInfo : ObjectNotFound: (conda:String) [], CommandNotFoundException + FullyQualifiedErrorId : CommandNotFoundException
Thanks for your tutorial. I am trying this on Windows pc. I am in anaconda prompt. I am stuck at this error "Error: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based project". I tried installing installing visual studio build tools but the error still persists. Any idea?
Hello @PromptEngineer48 , I new to python and want to learn this. I followed your instruction in your great video, but had many errors when installing modules in requirements.txt. Any update?
Thank you for the amazing video! a quick question, are these local llms safe to use with private data? I want to use it for work but idk if the info will be shared with meta.
thank you ❤ can the LLM be given a system prompt to only retrieve answers related to the source pdf or csv and nothing else? also which ui can be used instead of command prompt with this setup?
I am getting the sqlite3 error of using a unspported version, even the link specified does not solve my error can, I get the error when I run the ingest.py file
I'm getting CondaError: Run 'conda init' before 'conda activate' during my installation. I did try conda init but then it says "no action taken". As usual I can't get a step-by-step tutorial to work.
Although I don't get the (base) in front of my path like you did after Preparing transaction: done Verifying transaction: done Executing transaction: done
Hello, I am trying to follow your youtube video about "Private Chat with your Documents with Ollama and PrivateGPT" but when I type in my query after running the 'python privateGPT.py' part. I get the following error message "ValueError: Ollama call failed with status code 404. Details: model '7560' not found, try pulling it first". I'm not sure what this is talking about as I do the 'Ollama pull mistral' command before running the rest of the code. Any idea how to fix this?
It is an another great video, thank you! And I need your further help. I use ollama in docker container, I think it is easier to organize and I use wsl2 ubuntu. I think it is related to the lang-chain and ollama. I cannot find the info in lang-chain docs. Do you want to try that? Though you are using Mac. Thank you !
Hi, I did what you said with some hesitation but it worked as you said . This is one good program to use to have a good look at a documents content and meaning. Thanks. BTW is there way to let the program give a full response with out telling it to continue with its explanation?
Outstanding work you did bro. I was able to get mine running following your instructions with no issues. Does this support llama2 as well? My other question, what are the steps to run privategpt ui? Thanks a lot, this is brilliant!
@promtEngineer, can you host it with basic UI interface, like people could upload their docs and ask questions, without doing all of this hardwork🙈, and also dont keep anyones personal documents with you
great video! it is very cool. I noticed that you implemented the Ollama on your mac, but can you update this to install in a server or the cloud that can provide this service to your friends? that would be helpful for your friends
Interesting idea of integration. But i think that since you have no bugs in this process, it must be automated. Also, ollama is nothing more than click-to-run tool to download and deploy different sorts of AI models, so in fact you don't really need it to run Mistral with PrivateGPT. Or am i wrong?
Great video. Is there a way to know the limitations on the number of ingested documents or their size? or the maximum number of chunks that the model can deal with? Thanks
@@PromptEngineer48 even after ingesting the PDFs,I am getting answers for some questions that are not in pdf.Is it because of the already trained model?
yes. a good prompt. in the begining of the prompt mention that you are a chatbot and answer based on these documents only.... some modifications. but this works for openai api. i have tested in my current project that I am doing on some RAG applications.
Dude … This is AMAZING! I was just looking for pushes in the right direction, but this actually does exactly what I was attempting to do! Thank you!
Glad I could help!
You've saved me from going down my RAG rabbit hole. The code is extremely detailed, clean, and easy to understand too. God bless.
Thanks for tuning in.
I've been looking for this for months. Thank you alot, dude! Subscribed :)
Thanks for the sub!
Just found out your channel and subbed after watching this video.
Very good quality video! Keep up the great content creation!
I am impressed with your motivation to answer the questions from your comment section.
Not every youtube channel cares about answering subscriber questions and doing content around what people comment , so very good job!
Thank you. It's my pleasure to be talking with my viewers..
Followed your instructions here and installed on Raspberry Pi4. Works but of course painfully slow and chip approach 145 degrees F which slows things down as well. But it works and may try on Pi5. Was using a pdf manual for Viking drill press for document. Have to try something with just text.
Hmm. I see.
Awesome effort!! Your code worked right off the bat. Thanks for saving me a LOT of time.
Thanks
I will like to say that this works just as well even today. Much thanks to the prompt engineer, you have made my life and others a lot easier.
Thanks for watching
Would like to see ollama with openai assistant or similar that can use a file and have assistants do action to the file such as execute code or clean up data etc.
Got it. Let me do some research.
this is really high quality content, especially given the effort made in editing. the subtitle is a nice addition also.
Thank You for noticing the efforts... 😍 -- With Love (Prompt Engineer)
However, more than the subtitles, I want the main content to be more engaging.
Thanks for doing this: It seems like when one clones the repository, you need to delete everything in the db folder or it will mess up the results of the information you are trying input yourself.
Hi, while setting up Virtual environment I'm getting an error, 7:39 what programs do I need? I have installed Miniconda and it's still no running the line. Thks
then try with Ananconda
i use "pip install tqdm ",The system prompts me: ModuleNotFoundError: No module named 'tqdm'
Make sure you have installed all the frameworks listed in the requirements.txt file. Rewind the video, he explains how to do it all at once. pip install -r requirements.txt
If you have done that, make sure you are working in the correct environment. The same environment you installed the python frameworks in. He explains that as well.
I get the same error. Everything installed successfully, and I'm in the same environment ... :0)
same here installed all the required frameworks but still got the error , what solved it
Great tutorial, one of the best on the web!! Thanks for your time and effort! Upvoted 👍
Thanks so much.
Great video, took me a few tries getting Ubuntu configured (using wsl under windows) and updated before it would build the scripts, but I am very impressed. Thanks for all the effort you put into these videos 😁
🤗 welcome. Trying to bring the best.
what is "wsl under windows?" I'm running Linux. No Windows here. His instructions in the video don't work for my system at all.
Good work!
I found your video well done!
Can you still use the ollama API to search your documents? or does it need to be via the console?
API can do that
If I give ollama pull mistral
I'm getting error that says could not connect to ollama app, is it running ?
Please give me a solution for it
simple, before typing ollama pull mistral, write ollama serve..
Actually using it in linux platform but still I gave the command ollama serve, I'm getting no GPU detected, Please give solution for this !
@@PromptEngineer48
How do I remove the embedded Think and grow rich pdf file. I say this because when I add other file the query goes back to this pdf file and quotes it
delete the db and cache folders
Thank you, forgive me for sounding slow. You do mean remove or delete the chroma.sqlite3 file?@@PromptEngineer48
If I delete the cache folder name db , will this effect how the program will ingest the files I supply or will create a new db and then ingest the new files?
This is great, can you add or show how we could add gpu integration even better if it is over a GUI with privategpt 2.0?
Alright
Awesome idea. Now use Streamlit or Flask as GUI and the solution is perfect. Thanks for the inspiration. Questions: 1. which OS are you using? 2. python version? 3. do you rather use CPU or GPU? Would an M3 with 128 GB also be sufficient for a quick training / fine tuning of hugging face models up to 20B? Thanks for the answer in advance.
Hp
1. Mac OS. Pretty basic. Macbook Air M2, 8 GB
2. python version 3.9 ++ version
3. M3 with 128 GB.. Hmm when you say fine tuning it depends on the model. But here is a rough calculation. If you have a 20B parameter model in 32 bits, then you need 20x32/8 = 80 GB GPU. your system should be able to do the fine tuning. Else please go for 4 bit quantization, then the requirement will reduce by 8 times, now requiring only 80/8 = 10 GB of GPU.
I'm using streamlit for a great little interface.
Wonderful, thank you! My dream would be to feed my local model with all my books in epub or pdf format just once and the model will keep a memory of those. Then from there I have so many ideas but not sure if feasible, maybe someone can help? 1) for each book create a mind map of concepts 2) a diagram of how each book is related to the others (citations, same authors, same topic, related concepts) 3) given a question or a topic the system can point me to which book is better to read. Probably impossible at the moment, isn't it?
Wonderful idea. I will dedicate time for a POC
@@PromptEngineer48 Awesome! Another one could be to automatically find the book category. Since the system has already ingested all the books it already knows the discussed topics in each book and from there it can assign each to an ontology of category and subcategories. Like non-fiction/self-help, fiction/novel, non-fiction/self-help/personal growth, and so on. I have other 100 of these ideas, ping me if you need more :D
I would also be interested in something like this if possible as it would be amazing to do this in a repository of books that you know. If you could do this with a non-trivial number of books, say, 100-150 (from Project Gutenberg), it would be a great application of local LLMs.
About the context window. I have noticed that it cannot exceed over 2k tokens even though Mistral can support up to 8k. From what I have tested so for, it is like the bot identify itself as a GPT-3, is it because of the openai library?
yes. That is because we start everything compatible with OpenAI API. then we shift to opensource APIs. We could instead work for OpenSource APIs for the start. 😁
@@PromptEngineer48 So for this Ollama, the context windows will not be limited with 2k only right? It will be scale based on the model capability.
Yes.
@@PromptEngineer48 I wished that I could test it right now. Windows user need to wait.
We can try LocalAI. I will come up with a video
It would be great if you will create a video where you work on entire project, consisting dozens of files, related to coding adding, editing some feature. Thinking about that, DeepSeek Coder would be a best choice I suppose.
Noted! Will try to implement. Deepseeker Coder is very good actually. Beat so many benchmarks. 😄
@@PromptEngineer48 Also, will it be necessary to use MEMGPT to store different contexts in separate chat windows? I'm bit frustrated trying to figure out bundle of all required to perform what I have mentioned in initial comment =)
MemGPT working solution.. not yet developed. For a big project like youra need some exploration. I think I have to go for contextual sparsity. ..
@@PromptEngineer48 Bing came back with following bundle: Yes, you can use **AutoGen**, **MemGPT**, and **DeepSeek Coder** together to chat with an entire project like a Ruby on Rails (RoR) project consisting of dozens of files. But it got concerns regarding token limit:
"It might be challenging due to the token limit. Each file would need to be processed separately, and the total number of tokens across all files would need to be within the token limit. If the total exceeds the limit, you might need to truncate or otherwise reduce the size of your input1.
Remember, a token in this context is not necessarily a word. It could be as short as one character or as long as one word. For example, “ChatGPT is great!” is encoded into six tokens: [“Chat”, “G”, “PT”, " is", " great", “!”]
Thanks!
Thank You so much !!
Thank you very much, all the videos you have made are great! However, when I was building the local llm, I found that there are multiple frameworks, such as privateGPT, localGPT, langchain, etc. Similarly, I found that there are very many choices for the llm as well as the vector database (e.g., hugging face vs. ollama), which gave me a big headache, and I was wondering if you could make a I was wondering if you could make a video that explains your recommendations for each part of the process of building a rag-based personal local document chat llm?
That would be a very good video but very less relevant as everyday we have so many updates. But this is something I can create. thanks for the idea. Definelty will work on that
@@PromptEngineer48 Thank you for your response! It's true that, as you say, the various programs are moving so fast. Since I'm hoping to be able to build something myself via langchain, I'm starting to work based on your github project, and some of the other out-of-the-box projects (e.g. open-webui, privateGPT, etc.) are just too heavy for me to get into and modify.
Cool.
hi, can you update this so we can use LLama 3.1 instead, please provide a version that works with Apple silicon as this one fails on my M3 Mac
It would be great to feed local AI with project code or framework, and be able to ask it questions about the code.
nice idea. i will try to implement the same.
Is there any updated guide which is more easy to handle without using VSC and scripts?
My brain is exploding because I was just using your code repository earlier today and then closed everything out. I'm trying to run it again and I'm now getting a bunch of depreciation LangChain warnings. I still get a response but the response is no longer referencing or citing the documents I placed in the folder. Any idea why this would happen?
my bad. i should have freezed the dependencies.
Fear not. I will create another video with fresh codes.
what is the system configuration that you are using and what is the response time for the query
Just a basic 8GB M1
Thank you a lot for your effort to create an easier way to get up and running. Could you make a video explaining what all of the parts of "requirements.txt" are needed for? I would like to understand how everything works in detail
Sure thing!
Great video!! However if i follow the instructions my results are different. I created the source_documents folder an put in another pdf-file. When i then execute the "python3 ingest.py" the ingestion seems to work fine. But when i afterwards exceute the privateGPT.py and start to interact with the llm it still responses to "Think and Grow Rich"-Book.
Delete the db and cache folder
@@PromptEngineer48 can we not append the pdf files, do we have to keep removing them. when a new file is added and ingested, it should add the document into its cache and should be able to response from both the document isnt it
@@arvindelayappan3266 did you ever figure this out?
This is awesome, but still not bale to chnage the base url for ollama, is there any way i can change that?
Thank you for this great content! I was able to get this working on my M1 Mac. I was able to run `python ingest.py` and the `python privtagpt.py` commands. However, when I asked it to summarize the document I had uploaded, it referenced the "Think and Grow Rich" document that you showed in the video, rather than the test document I uploaded to the source_documents folder. How do I clear out the embeddings from the "Think and Grow Rich" document, or clear the chroma db database embeddings completely before running the ingest.py command again?
Delete the db and cache folder
@@PromptEngineer48 Thank you! Got it to work exactly as expected! Thank you for all the great content!
Welcome
Thanks, that’s a great video! I’m testing privategpt for some time and I would love to know if you’re experiencing hallucinations from the chat? And yes mistral seems to be a good model but Zephyr and Dolphin seems to give better answers with a little less performance, depending on the context. Therefore, I couldn’t Get rid of some hallucinations, I would say the reliability of information is 45-65%
Yes. Hallucinations are there. Need a good model in future
This work very well but it has issues ingesting docx, pptx and ods files without a pip python install
Hi I am connecting from another computer. I have my Ollama in AWS cloud. How do I make it where I can train it like what you did here?
there was no training just injest and spit.. if you have ollama in AWS cloud, you need to somehow use that via api calls.
thank you, sorry but i don't see the old chat interface :-) i have 2 questions
how can i remove a ingested document that i don't need?
ist it possible to chat with the docker interface ? or is it only for terminal??
1. U can remove the file. Then delete the db folder and __pycache__ folder.. then run python ingest.py again and python privateGPT.py again.
2. Web UI interface is not integrated here right now. But that is some ok my pipe line as well. I am working on that.
how can i get a good looking ui like a chatbot ? not on the command prompt like ui, instead , would like to see a web based ui with a send button
u can use streamlit, gradio, ollama web ui and so many other options
Well explained. Can you suggest if we want to pass a onenote how we can pass it.
Are you using the code from private gpt’s primordial version? What a great work! By the way, now they officially support integration with ollama. You probably wanna make a video about how to set this up.
You're a life SAVER... many thanks
Welcome
@@PromptEngineer48 can i apply this inside streamlit web app so that users ask and get the answer on ui? Can these models be deployed ?
Can you tell, how to use other models like llama3, without using mistral?
Does anyone get this warning - UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown (and subsequent crash)
I didnt get such
how can we get chatGPT alike local LLm, i want to use it for Research and forecasting and predictions
there are research going on. you need to find it from huggingface or ollama .. see the benchmarks and decide which model you want to use. since we have so many options, I cannot name one. but it's a case to case basis.
Hi, thanks for the tutorial. I am able to run the model using Ollama on windows but after everything the project runs and Asks for the Enter a query and if I add the questions it stucks there. In problems I am getting bekow issues such as
Import "langchain.chains" could not be resolvedPylance
Import "langchain.embeddings" could not be resolvedPylance
Import "langchain.callbacks.streaming_stdout" could not be resolvedPylance
etc
Can you please help me out in understanding this issue and resolving it.
Thanks.
There are bugs in windows.. you can try with Linux on windows.
Hi, can you create a video on how to clean our own data which is in my CSV file which is best to answer our query very effectively?
Yes I can. but to be more clear.. you want to use a data cleaner llm which will give clean you csv file?
@@PromptEngineer48 Okay pls go ahead
How would you do this if you have ollama in docker? Or even cloud gpu like runpod? Or Linode? Seems like a good video for the future as you explain,and teach better than a majority of the creators.
Okay. Will create a video soon on the specific use case that you have mentioned.. I mean by hosting on docker and chatting with docs. Thanks for the comment
same here, I'm using dockerized ollama.. would be great to have privategpt with dockerized ollama..
Okay. Got the requirements
Now it's my turn to create that. 😄
I got an error when I typed ollama run mistral. Got a a message saying that ollama command not found. I get this from within the terminal of Visual studio code. But I can run Ollama from the mac's terminal window no problem. Did I forget to do something? Thank you!!!
try to change the type of terminal that you are using. i mean zsh or command prompt.
Hey! Are you able to show how to use auto-mem-local using Ollama? Thanks!
Noted !!!
ModuleNotFoundError: No module named 'tqdm'
even after installing the library properly. Any fix ?
Were you about to solve?
@@PromptEngineer48 yeah ..updating the package fixed it for me. Thanks again for this awesome tutorial
@@varun_tech7You solved the problem yourself. Congrats.
@@varun_tech7 which package did you update? thx
@@raminderpalsingh123 I don't exactly remember, just make sure all package is up to date
Where the knowledge base is stored? It keeps track of all pdfs already processed?
The are two folders created automatically names db and cache.
hi, ask a question. How can I change the model for example to llama3 with model 8b-instruct-q6_K?? please
go to line 12 of the privateGPT.py file and change the mistral to whatever model your heart desire.
Hello, i keep getting error ERROR: Could not find a version that satisfies the requirement onnxruntime>=1.14.1 (from chromadb) (from versions: none)
ERROR: No matching distribution found for onnxruntime>=1.14.1
I had the error for ChromaDB too. If you Google for them you will probably find an answer. For ChromaDB this solved it for me:
- sudo apt install python3-dev
- sudo apt-get install build-essential -y
Great Tutorial, Why is it saying loading 235 new documents when it has only one in the source_documents folder. I had 2 in mine and it said 8, wondering why.
It's because of the chunking.. I had put only one document but it was chunked into many pieces.
Very interesting, easy to follow. I tested this with a music instrument manual, and it wasn't giving accurate answers at all. Any ideas on how to improve this? It's unusable for this type of document. Makes me wonder how accurate it would be with any content. Thank you!
That was pretty old stuff. Please watch the recent videos on my channel.
Could you provide suggestions on how to enhance the conversational capabilities of this bot?
Prompts and proper chains ⛓️
This has been very instructive, thanks!
Is there an LLM that's better than Mistral at working with very technical documents, i.e. lengthy API implementation documents? I set up my environment exactly as instructed, got my docs indexes and could get some answers on them. But things get murky when I go very specific, with incorrect or incomplete answers.
Try with mixtral 7x8b
Nice video, very informative! What do you use to generate subtitles? thanks
Thanks.. Capcut
can you provide a similar use case setup for windows as well, now that Ollama is available on windows
No difference. Just use the code on windows. Make sure to install Ollama on windows.
great video. now can u do it in streamlit so there's user interface to chat and how can we host it online? like render?
Okay. Streamlit and render integration got it. Will do that.
Thanks for the great effort.
I just have a question. When a file is ingested, then i want to ingest different file, should i delete the db folder first? so not to mix older ingested files with the current one i want to ingest.
Also, after ingesting one of my csv files, i asked few questions. Then the answers i was getting were far off and was actually from another source, which i didn't provide, it was from a pdf book about getting rich or something. When i only ingested a csv file with numbers!
Yes u need to delete the db file first
If my company has a pre-existing Data Dictionary, is there a way to allow Ollama to integrate it so that it better understands the data I am working with?
yes. that could be included technically.
I want to try to give it some ability to troubleshoot for me, like conduct pings and traceroutes.
looks good
do you know how to stop Ollama afterwards? it continues to run in the background even after trying to end the process multiple times.
An icon on the top.. appears. Press and quit
I can not update the db, when I ask the agent about the new added document, it still gives answers about the document in this video. It is kind of confusing.
Delete the db and cache folder.. then upload your own document.
Thank you, you are right, and really helpful!@@PromptEngineer48
I couldn’t get Conda to work after installing Conda . The installed files are in /opt/miniconda3 but whenever I run Conda, it’s says Command Not Found
www.anaconda.com/download
You need to install anaconda. however, you could use the .venv instead of conda. We just need a virtual environment.
@@PromptEngineer48ok managed to get Conda working. However when I run the python3 ingest.py, I got an error: line6, in from tqdm import tqdm. ModuleNotFoundError: No module named ‘tqdm’
Ok made more progress but now stuck in pymupd. Tried installing it but keep getting message Requirement already satisfied: /opt/anaconda3/envs/privategpt/lib/python3.11/site-packages
pip install tqdm should have worked
i am facing thiis error
conda : The term 'conda' is not recognized as the name of a
cmdlet, function, script file, or operable program. Check the
spelling of the name, or if a path was included, verify that the
path is correct and try again.
At line:1 char:1
+ conda create -n private1 python=3.11
+ ~~~~~
+ CategoryInfo : ObjectNotFound: (conda:String) [],
CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
docs.anaconda.com/free/anaconda/install/windows/
install the anaconda
Thanks for your tutorial. I am trying this on Windows pc. I am in anaconda prompt. I am stuck at this error "Error: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based project". I tried installing installing visual studio build tools but the error still persists. Any idea?
I believe we need .NET SDK, .NET Framework and other tools using the build installer
@@PromptEngineer48 all installed but still wont run.
got the same error, did you manage it to run?
Hello @PromptEngineer48 , I new to python and want to learn this. I followed your instruction in your great video, but had many errors when installing modules in requirements.txt. Any update?
why dont i come up with an updated video.. please give me like a week or so.
@@PromptEngineer48 very nice... thank you sooo much....
line 6, in
from tqdm import tqdm
ModuleNotFoundError: No module named 'tqdm'
gets stuck at this
pip install tqdm should solve the issue.
Do you have the pdf document you tried? Would like to try it myself
It's the think and grow rich book. Just search for the book on the internet
Hi, Ollama is running, I input query but nothing receive answer, termial is blank. What should I do?
Try with different LLMs. a smaller version please
Is similar solution possible using 'LM Studio' for users having Windows machines?
Not right now. But if u follow the discord of lmstudio. They have plans
@@PromptEngineer48 i installed wsl in my windows machine and followed your tutorial, everyhting worked fine. Great tutorial btw, thank you :)
Thank you for the amazing video! a quick question, are these local llms safe to use with private data? I want to use it for work but idk if the info will be shared with meta.
U can switch off internet. Safe 🔐 or not, we cannot guarantee, it may so happen that once u connect to internet the data may be transferred.
@@PromptEngineer48 okay, so the advantage of running an llm locally is just to save some $$ from API calls?
It Is possibile to load legacy code (i.e vb6 )and database schema and query how this code does?
I should be possible
Keep getting a error when doing python3 ingest.py says line 8 no module named langchain
pip install langchain langchain_community
Is there a limitation on the max number of personal documents that I can upload to PrivateGPT?
No. but then the vectorstore gets confused and not able to get the relevant chunks.
thank you ❤ can the LLM be given a system prompt to only retrieve answers related to the source pdf or csv and nothing else? also which ui can be used instead of command prompt with this setup?
By default it will retrieve answers related to the source only. you should see the source locations as well.
@@PromptEngineer48I revised my comment to ask about UI
@@RedCloudServices UI?? Streamit, Flash, Gradio etc etc
which is the best vision model to extract entities from complex invoices ?
If you are talking about open source, then I have to go with ollama.com/library/llava
If closed source, then openai
I am getting the sqlite3 error of using a unspported version, even the link specified does not solve my error can, I get the error when I run the ingest.py file
Were u able to solve?
I'm getting CondaError: Run 'conda init' before 'conda activate' during my installation. I did try conda init but then it says "no action taken". As usual I can't get a step-by-step tutorial to work.
So you were able to create a conda environment?? Using conda create -n your-name python=3.11??
Yes. Followed everything up until the activation part
Although I don't get the (base) in front of my path like you did after
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Will it recalculate embeddings everytime I add more pdf documents?
yes
@@PromptEngineer48 I mean it should only compute embeddings for the new documents only, not for entire set.
Can you do a video on using LLM for data science.
let's do it. Next video.. let's start with a good LLM for programming called DeepSeek Code. It's a new LLM and sound promising.
Hello, I am trying to follow your youtube video about "Private Chat with your Documents with Ollama and PrivateGPT" but when I type in my query after running the 'python privateGPT.py' part. I get the following error message "ValueError: Ollama call failed with status code 404. Details: model '7560' not found, try pulling it first". I'm not sure what this is talking about as I do the 'Ollama pull mistral' command before running the rest of the code. Any idea how to fix this?
try this ollama run mistral:latest
@@PromptEngineer48 It let's me run mistral:latest but how will this fix my error above? Do I have to add that to the python code somewhere?
It is an another great video, thank you! And I need your further help. I use ollama in docker container, I think it is easier to organize and I use wsl2 ubuntu. I think it is related to the lang-chain and ollama. I cannot find the info in lang-chain docs. Do you want to try that? Though you are using Mac. Thank you !
yes. I shall start working on that.
Hi, I did what you said with some hesitation but it worked as you said . This is one good program to use to have a good look at a documents content and meaning. Thanks. BTW is there way to let the program give a full response with out telling it to continue with its explanation?
I think there should be a verbose flag, which you can set to False.
how is this approach different from RAG, can you elaborate please?
It is RAG
@@PromptEngineer48 but i'm confused you replied to the message just below for his question which assumes it is not RAG?
I am sorry for any confusion. It is a RAG system. any use input of pdfs etc will be used to reply to the user's questions.
Great video, thanks for sharing it.
Can you create a Video content talking about making work Cheshire cat on ollama ? Must be interesting.
Thanks for the idea! On it.
Outstanding work you did bro. I was able to get mine running following your instructions with no issues. Does this support llama2 as well? My other question, what are the steps to run privategpt ui? Thanks a lot, this is brilliant!
Yes. Llama 2 as well.. PrivateGPT UI I shall have to look.
@promtEngineer, can you host it with basic UI interface, like people could upload their docs and ask questions, without doing all of this hardwork🙈, and also dont keep anyones personal documents with you
Ok
great video! it is very cool. I noticed that you implemented the Ollama on your mac, but can you update this to install in a server or the cloud that can provide this service to your friends? that would be helpful for your friends
Yes you can! I will try to bring in a video
Interesting idea of integration. But i think that since you have no bugs in this process, it must be automated. Also, ollama is nothing more than click-to-run tool to download and deploy different sorts of AI models, so in fact you don't really need it to run Mistral with PrivateGPT. Or am i wrong?
Now with more interesting integrations, we can scrape privateGPT itself and use Ollama to code up our projects natively. Yes you are right
is there a way to view the actual embeddings value from ChromaDB?
Sure it is
@@PromptEngineer48 How exactly can I view them ?
Great video.
Is there a way to know the limitations on the number of ingested documents or their size? or the maximum number of chunks that the model can deal with?
Thanks
There seems to be a limitation of 166 embeddings.
I am getting answers from out of pdf also.How to restrict that to only pdf specific?
Ingest only the pdf.
@@PromptEngineer48 even after ingesting the PDFs,I am getting answers for some questions that are not in pdf.Is it because of the already trained model?
Can private GPT be run for a web type interface similar to your previous video ?
Is there anything which is not possible. 😅😀
is there the posibility to take responses only from your documents?
yes. a good prompt. in the begining of the prompt mention that you are a chatbot and answer based on these documents only.... some modifications. but this works for openai api. i have tested in my current project that I am doing on some RAG applications.
can you make a video for the front end as well?
Front end. okay. which interface would you like.. like gradio, streamlit etc. any specific requirements. my default would be gradio. would that work?
Yes, Gradio will be suitable! Thank you for your quick response. I appreciate your efforts. I look forward to watching your video
Okay. On it
Thank you very much t @PromptEngineer48.Looking forward to it.
Great video! Can I connect Langchain with it? Or is it included. Im a newbie on it :D
Yes u cN