How To Install PrivateGPT - Chat With PDF, TXT, and CSV Files Privately! (Quick Setup Guide)
Вставка
- Опубліковано 21 жов 2024
- In this video, I show you how to install PrivateGPT, which allows you to chat directly with your documents (PDF, TXT, and CSV) completely locally, securely, privately, and open-source. PrivateGPT is the top trending github repo right now and it's super impressive. It uses GPT4All to power the chat.
Enjoy :)
Join My Newsletter for Regular AI Updates 👇🏼
forwardfuture.ai/
My Links 🔗
👉🏻 Subscribe: / @matthew_berman
👉🏻 Twitter: / matthewberman
👉🏻 Discord: / discord
👉🏻 Patreon: / matthewberman
Media/Sponsorship Inquiries 📈
bit.ly/44TC45V
Links:
PrivateGPT - github.com/ima...
GPT4All - github.com/nom...
Google Colab Version (very very slow) - colab.research...
Update: New version of PrivateGPT is apparently 500x faster for ingestion!
Update 2: I don’t believe 2 models are necessary anymore. Just the one.
Was there any quantization applied if it's a 100% "offline" "clone" of a bazillion parameter gpt? I've read where they offer renditions under the "offline" or "private" heading, but they are "4-bit quantized".
Can you please do a tutorial of this but for Google's Colab instead?
@@infini.tesimo The all important testing part may produce an unexpected anomaly or two when building _Private_ GPT on a Google _collab_ platform.
Thank you for the info, Matthew! This tutorial works for the new version?
@@DavidSegura99 Yes, it should!
I think one of the main reasons I love your videos is you get a lot of detail in there whilst being really concise! Keep up the good work :)
Thank you very much
@@matthew_berman i notice such a surge of youtubers doing AI technical content, so far i like your channel, mainly for the same reason... its short and to the point
@@matthew_berman this work with works in spanish and answer in spanish?
@@krellin yep there’s a lot. And thank you.
@@nicolasriveira4262 not sure sorry!
Local AI really is the best for everyone. Thanks for keeping me updated.
You’re welcome, Marshall!
That's what I'm hoping.
Ai will be available as a good, not just as a service as many companies are trying to go for.
It would many companies to develop highly specialized and powerful AI and professional help if they want to offer something else instead, I'm fine to have "basic" AI.
@@AdmiralBison Trouble is it's always going to be a balance between complexity and capability, and the compute power it needs to process the data and get results out of, and the sort of compute power available to the average person, especially with stuff like smartphones and tablets and very low power laptops being the norm for a lot of people. It's always going to balance the more capable tech in favor of those who own the most infrastructure to use it, where they can remain gatekeepers to that.
@@AdmiralBison what do you meant by “as a good”?
@@leopoldomolina1763 “goods” - a synonym for merchandise, products, items, etc. ;)
Thank you for this video, this is exactly what I was looking for. Thanks for getting straight to the point and not being a word buffet making us watch a 30 min video only to give us the info at the end.
You’re very welcome :)
I’m really keen to try PrivateGPT thanks for the update! Really good video and easy to follow 🙏
Thank you, Hendrik!
the process is pretty much changed completely, so request you to make an updated video
Great content I was looking for a way to analyze larger data sets and tailor it to my specific needs this is a great resource. Thank you for the tutorial
You’re very welcome Jayden
GPT4 with Code Interpreter enabled will do that for you ten times better than this local model. I do data regression analysis with huge amounts of data and it saves me literally days.
@@newunderthesun7353 I Would be greatful and thankful to you!
precisely followed your video and after spending two days it worked out very well. Thanks a ton.
Did you train it 9n your own dataset? How did it perform to prompts about the data?
I have been wanting to do this for awhile now, but unfortunately I have ran across problem after problem as your instructions don't include all the stuff you have to have in loaded to make this work, and I still can't get into ingest my documents. I keep getting errors. I had to you chat gpt to figure out all the details, but I am about to give up on being able to make this work. I am installing on PC, maybe you method above works better on a Mac. I don't know.
I wish every tutorial on the internet was like this one.
Thank you. What do you like specifically do I can make sure to do more of that?
@@matthew_berman clear and straight to the point
Hello,
Love your tutorial.. I was trying to implement this. But i dont see requirements.txt file in github.
Or am i missing something. Please guide
It would be easy to run this as a container and would make the setup much simpler.
Create one please :)
Absolutely!
I need to figure out containers😅
@@joe_limon Docker is very easy to use, so many tutorials out there also :)
Yes! Always do this
It would be good to use a per-project virtual environment because different projects can have conflicting requirements, like needing different and incompatible versions of the same package.
Agreed. Should I include that going forward? I feel like the project authors should do it 😂
Just put it into a docker container and you are fine.
@@matthew_berman yes make a video how to pls!
@@croci81 Which shouldn't be surprising. Python wasn't really designed to be used for serious projects. Which shows up in the things that it is just terrible at like iteration, the semantic white space and useless error messages.
The fact that people can write anything sophisticated with it, is a tribute to just how good some people are at programming.
@@SmallSpoonBrigade Every language has Its pros and cons, but it is not that horrific. It is one of the top languages people use mostly today. Really depends on your project and what you are working on. You prob are not programming OS or AAA Games with it but it is so popular there are a huge number of libraries that can speed up your coding a thanks to a big community.
Yesterday I was reviewing a very long video from some good samaritan from India or so, but it seemed like a long process with lots of things aside to master. BUT it is a piece of cake!!, THANKS! this of so much value to me in this moment!
You're welcome!
This is so cool! Thanks for the walk-through!
You’re welcome!
Thank you for your time making this video!
Does it work on GPUs?
This is awesome! Thank you for the clear and easy-to-follow instructions. Only wish the UI for queries and answers was like a website.
You do such a good job with your tutorials
Is alive!!! The private GPT is running (not exactly). It is slow but it is working!!! Thank you for your help! I am having a blast!!!
Thank you so much, that was awesome, I had some hiccups, however, it's working.
Ok great!
Thank you! I didn't know this existed. I have been needing something like this and I thought I was going to have to try to make it myself! I just subbed to you too! :)
Fantastic setup video, made it super easy. Unfortunately the model is not that great, it could barely calculate 25% of my income vs rental costs properly lol.
Thank you! Sorry it didn’t work well.
Large language models are not good at math
Thanks for all the awesome videos! By the way, I loved you on Brooklyn Nine-Nine!
Now this is dope, and this is why corporations are freaking out.
Is it even a leak? like chatgpt leak or they simply replicated this feature using existing stuff?
Not sure I understand?
Hey Matthew, just found your channel and you got a new subscriber. Great content!
As others have mentioned in the last month, there is no 'requirements.txt' in the new file so currently, this video does not reflect the actual process
I used the PrivateGPT link from the description and cloned PrivateGPT on my desktop, but there are not the files "reqiurements.txt" and "example.env" in it. Are you sure that the video is still actual?
Thank you for letting the world (where UA-cam isn't blocked) know how to setup PrivateGPT in May, 2023. Would you recommend it for production use in January, 2024, or do you know a better option for useful, reliable and fast website chatbots?
This sounds really awesome!
You just earned a sub from me , great video clear ,concise , and thorough and most of all I can take action on it and its respectful of my time , keep up the Good work much appreciated
Hi Matthew - this is awesome. Thank you for the work you're doing. You're providing a very valuable bridge between the deep techie stuff and those of us who are adventurous but not so technical
Quick question - can you further train the GPT for all instance? Can I send recent content to it to update itself?
I have the same question.
I'm curious also. Is it live-trained for all users or only on specific threads? For example, I've seen how it learns once you paste a document into the thread, but does that learning then apply to all other users?
You are such a good man. I appreciate you so much.
Sadly when I type ip install -r requirements.txt I get the following message:
ERROR: Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt'
Is this no longer available?
Hi Matthew! Super video!!! I have a question: is it possible to use Italian language to use PrivateGPT and for documents?
That seems awesome as I have been wondering if there is a way of using ChatGPT on your own documents as I spend a lot of time compiling a single version out of dozens of older existing documents. I wish there was a way that ordinary people could have some input into these sorts of technologies coming out. I don't code at all, so could not do it my self, but as an end user, I know the sorts of tools that I would really like to have access to. Personally I watch all of this effort going into products like ChatGPT and I keep thinking, that is almost useful, but not quite, or if only it could do x, y or z, and then I go back to just doing things myself as all of thse AI tools just aren't useful. Same with automated cars. There are so many functions or approaches which would be useful but as an end user you have no way of influencing the products that are being made, and so you just have to watch as others make unhelpful tools or mess around with only making money as an end goal. It is exasperating.
Hey Haydn. I like your proposed usecase. Would you be open to connecting privately? I can build something similar for you at no development cost.
Thats why open source in AI is great! Because just like you, there is hundreds of people asking the same questions, and it only takes one with expertise to adapt the tool. Afterwards, many more join in into the new branch and steers towards utility.
I have never seen someone use VS-Terminal, you legend
this is really interesting, thank you for sharing this!
You're welcome, Adam!
Thanks man! You did the great job 👍
You’re welcome!
easy to follow
straight forward,
fast
thank you man
Thanks for the introduction, may I ask which DB is privateGPT using? Chroma or something else?Maybe I missed some info in the video, excuse me then.
Thanks for the black background... (watching this 2ish am, truly helps with my eyes) lol
thanks for the easy to follow tutorial...keep up the awesome works
Thank you!
This video was super useful, thank you.
Not sure who will benefit from this, but it is worth upgrading your C++ developer tools kit from Visual studio installer before following the steps because it might end up as a blocker halfway. Or maybe it was just me.
Also, to those people using this, did you guys find ways to generate responses faster? Is it solely dependant on the PC specifications?
Yep. Absolutely! I had quite a few blockers as I hadn't used Visual Studio before. Running ChatGPT alongside to debug each error and fix and continue was extremely helpful.
Great video, Thanks for sharing. Quick question does it have API capabilities? I mean is it possible to make API calls to your privateGPT?
sure write an API for it - update the repository
Thanks for the useful tutorial!.
This is exciting and thanks for the great video. I have an archive of MHTML web pages, is there a newer version that would allow me to use those? or is it just a change to the Python code?
That's what I been waiting for.
Now I need that for images so that they can be Auto sorted.
This is amazing. What are the chances you can get the author of the repo onto your channel and chat with them about the why and the next things coming?
Why? This is just one out of a thousand GPTs that have been open-sourced on the internet right now, and it's nowhere near the capabilities of the older GPT-3 that Matthew didn't want to mention.
I'm going to need a second brain based on the books that I have read and incorporate my notes/tasks into a GPT. Thank you for the easy tutorial!
As always great job!!
Thanks, Abdullah!
Dude... you're a freakin' Legend👑for this! ✨👍
This is crazy, I'm going to research and see how to train this beast, it will be nice to have an offline "PrivateGPT" with all my documents like bills and stuff... I could even create a nice financial automation.
Hey, did you find a way to train the model accordingly? Thanks in advance
@xxy1950 Yes and no, I have stopped developing this after a project came, and I don't have the time to spare. Please see huggingface for models that can be fine-tuned. There are many examples there.
Hi Matthew,
Thanks for the great stuff !!
Just curious to know the hardware requirement to run this PrivateGPT. What level of hardware do I need?
Nice video, I'd get another light maybe, the shadows make it look like you have a shiner.
This video appears to be outdated. I would greatly appreciate it if you could either share the GitHub repository corresponding to the version shown in the video, or consider creating an updated video that reflects the latest, more advanced version of PrivateGPT.
the requirements.txt file is no longer available
Subscribed. Is it possible to make a video to implement this on a webpage with a webserver? It would be great to finally get longer responses and send larger peaces of code to Chatgpt
@matthew berman - thanks for the post. Do you know where the files end up when you load documents into privateGPT?
Thank you it worked but needed a little edit on some files because it was MACOS. Thank you so much!
After a lot of effort trying to install hnswlib, I managed to do it by installing it through Visual Code Insiders. Then, I just reopened the project in VS 22 and it worked.
I hit requirements.txt not found error. Seems like this file is now missing in latest download.
Where is the requirements file? It seems like the installation for mac or ubuntu is fast, but for PC … too many steps. Could you update the installation for windows? Or explain how to run it on PC
Awesome! Now I can import my huge pdf library and interact with those files. And it is locally. Big plus.
Yes!
I suspect things have changed in the last 10 months.... trying to follow your steps, but there is no "requirements.txt" file to be found anywhere... Help!
Does this program support languages other than English? If not supported, how can other languages be supported?

Thx for posting!
Whats the size of the whole project?
Do you have any hardware requirements?
The main part is the models, so about 8gb
it keeps saying pip install -r requirements.txt - command not found
Holy snap! That's awesome! I wonder if it can ingest an entire novel manuscript to give feedback or something. Haha
Latest version of the repo doesn't have a requirements.txt file. Anyone have any idea about this?
It's on a Mac. Awesome! I can try this.
hello sir. Now some new changes are made to the repo and the model is not training and saying that model not found,but i placed my downloaded models in a created direcotry of models. Please do a video on this new repo of PrivateGPT. Thanking you and hoping to see a video on this!
can i use llama 2 instead of gpt4all and download and train the llama model locally or in my private compute cloud
Which interface do you recommend to use GPU 12GB/Cuda? Grateful.
Qual interface vc recomenda para usar GPU/Cuda? Grato
What do you mean by interface?
I could not find the requirements.txt file
Where can i get it?
requirements.txt missing from files
thanks for the video Matt.
btw did anyone encounter this issue:
*llama.cpp: can't use mmap because tensors are not aligned; convert to new format to avoid this*
You're welcome. Hmm...never heard of that issue. Jump in my discord and ask for help maybe someone else solved it.
Awesome video. A question though, when you are ingesting the files, I assume that's when Chat GPT get access so your files to do the ingesting. So technically at that point Chat GPT is being trained or getting access to your files.
This isn't ChatGPT, and the LLM (GPT4All in my case) runs completely locally so doesn't 'report back' to anyone over the internet.
Nice. Oh and if my PC took that long to open a terminal, it would be finding itself out of a job.
Thanks for the video, subscribed to your channel
This type of video is my favorite 👍👍👍
I have a question about this model, is it possible to access it using the API?
Thank you!. Yes it is, that's how PrivateGPT was built using the GPT4All model.
what is the API for it?
short and no bs guideline. thank you for this. how can I find you on discord ?
Thanks! Link is in the description :)
Very nice setup guide! I'm wondering is it possible to use privateGPT as api to call in my project?🤔🤔🤔
im not very good at coding, my VSC terminal doesn't seem to understand "git" or "git clone" commands, im sure this is really obvious but im not sure why i cant get it to do that, i can cd into desktop though just fine
edit: i didn't know what 'git' was or that i needed to install it first separately lol
edit 2: so apparently on windows you don't use the VSC terminal, you just use git (which downloads as git bash) instructions seem to be the same though
Mat, Could you show us how to create a nice user interface for this? and if such creation is aided by chatgpt it would be awesome.
I might have an inside scoop on something interesting coming. Video coming soon :)
I got the error of Visual studio c++ 14.0 or greater.
Visual cpp build tools error .
Llama-cpp-python error failed to build.
@@matthew_berman Yes please¬!
Unfortunately not suitable to run on a NAS 😅 It would be great if you were able to run it automatically in the background, for example on specific document folders, and documents like contracts, bank statements, insurance policies are automatically being ingested. It would definitely make it much more human understandable.
Wait, you can't do this?? Kinda defeats the purpose, no?
It would be awesome to have a "home" AI working as part of your the home's central hub and automation.
I recon that would be the way to go, have it installed on all your devices so it can run when you are on the go and when at home it could just run at one central point or work in tandem with the "central AI"
I can't believe what I am saying, it all sounds so Science Fictional but it's becoming a real thing.
"Install one's AI at home"
@@fitz3540 it would probably be possible but most consumer grade NAS systems don't have enough memory nor computing power to make it practically feasible. For example, if it would take a day to ingress a 10 page pdf file and another day to answer the "make it understandable" command, it practically unusable.
@@fitz3540 Almost no NAS would have a good enough CPU
I tried this today and worked like a charm👍. There were few errors around libraries and environment and could solve it with help of google pretty quickly. Thanks Matt for this wonderful tutorial🙏
So it uses langchain, nice. Pretty interesting, although I don't know much about these models!
I also had problems with versions. I updated python to version 3.11 and language server to Pylance in VScode. At the end I ran the command 'pip install --upgrade chromadb' because of the error "KeyError: 'elements'". Thanks!
Hi Mathew, new to working with VScode here, what setting or extension do you use for your terminal suggestions/auto-complete? Thanks!
same, also my git clone function wasn't even working im reeeaally new 😅
Great video. Can this engine support other languages? Hebrew?
Thanks I will take the model 😊
Have you tried our Noteable ChatGPT plugin to leverage files as well?
Yes! Maybe I need to do a full review of it :)
I followed everything perfectly so far, but i keep getting an error called 'no file called dotenv' when I know for a FACT that I DO have that file from the previous step!
I have te same problem :/
Perfect, I plan to train it about my game then let it guide players better than any wiki.. or so I hope
I am running this on Windows 11, and after running the command "pip install -r requirements.txt", I got an error message saying, "ERROR: Could not build wheels for llama-cpp-python, hnswlib, which is required to install pyproject.toml-based projects" Do I need to rectify this or is it okay to ignore it?
Did you get the answer? I have the same error. I did everything that I could do.
Fantastic. In the source documents folder, can we place levels of subfolders?
I have two questions: 1) is it its only function to chat with files or you can task it with any prompt? 2) does this process have any particular system requirements, is it okay to use my laptop (16G ram, i3)
Amazng. thank you for the video. Could this be linked to an external database of documents like box or netdocuments?
3:08 Fatboy Slim would be proud for this reference 😅
Thank you. I tried it. I got following error's:
ERROR: Failed building wheel for hnswlib
ERROR: Could not build wheels for llama-cpp-python, hnswlib, which is required to install pyproject.toml-based project
Under "Environment Setup" is the second file "Embedding: default to ggml-model-q4_0.bin not noticed. Maybe a change on that site.
And EMBEDDINGS_MODEL_NAME=all-MiniLM-L6-v2 didn't have all-MiniLM-L6-v2 in the folder.
I assume there are something updated.
Did you get the answer? I have the same error. I did everything that I could do.
you da man!!