Yeah sure. You don't know if it does what he claims it does. For all I know it probably runs some malware or cryptominer in the background. Build it from the source directly
Note. While this is CPU bound, It also has a processor core limit of 63, so if you have a beefy server or threadripper you might run into issues. You will have to go into your bios and shut some cores off. Even with ~60ish cores, this is currently SUPER SLOW, and gets slower the larger your document database gets. I ingested a directory of ~ 500 MB of PDFs, and it took nearly 30minutes per query.
Yes, it is was too slow to be practical. However I was surprised that it actually worked ;-) Does it make it faster to run on the "lightest" model? And perhaps csv docs?
Personally I see that this project has a lot of value but I think the priority should be on how to scale the infrastructure to speed up ingestion and inference times, rapidly prototype and test a bunch of LLMs and last but not the least if it can have a nice looking UI.
Hey, I'm receiving a "C:\tools\miniconda3\envs\pgpt\python.exe: can't open file 'C:\\TCHT\\privateGPT\\privateGPT.py': [Errno 2] No such file or directory" error because there are no .py files, is anyone else getting this issue?
This doesn't work in the same way as a NN LLM then... it doesn't "train" on the data and use tokens it seems.. it just searches the document and returns an answer... what am I missing here?
No, it handles both the correct Python installed, and even no python installed. If you have an incompatible Python, you'll need to change it. If you want multiple Python versions, Conda is the best solution, and this script handles that as well.
I tried installing this with the Koala model (because I haven't tried that one before) and it had a problem. It was asking me if it was a ggml model, then just dropping out of the installation. It wouldn't start. I eventually reinstalled it with Vicuna 13b and it works. Can't wait for a faster version, though, as even with 34gb of ram it is a little slow.
hi there :) another question, how to install more models and, further, how to switch between the models to compare the results as you did? thank you very much, highly appreciate the video and your responses in advance
Unfortunately - this one doesn't work at all for me (where your others have) - across 18 different Win10 and Win11 machines, including a Win10 fresh, fully updated, and nothing else on it when started.
Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases. How can I solve this guys?
Everyone talks about private gpt being able to talk to your docs, but like I already read my docs, can you make it write some statements of claims, or some written submissions or something like that?
Sir, Good Job! But the interface so boring, hopefully in future, it will be more interactive UI My idea to developer out there... please combine Chatgpt+privateGPT = easily update knowledge base. I really wanted my ChatGPT can be customize + base knowledge
Is there a way to decrease the time it takes for a model to provide an answer? Like it takes 2 to 3.5 minutes for a vic7b, for example, to provide the response, is there anything that can be done on the matter?
No joy. Same issue I've had running it after a manual install. Could not load Llama model from path: models/ggml-vic13b-uncensored-q4_0.bin. Received error (type=value_error) Thanks anyway, man. :/
No way I'm gonna search and put so many individual files in a folder there's gotta be like some kind of datasets for this that has a sht ton of sht for privategpt to just know/learn from Also is there a machine learning version of this? Would be cool to have a version that can learn from datasets as well as reach out to the internet for additional data/info and then be rewarded or punished for getting answers right or wrong etc.
Thank you so much for the one liner! I got an error "ModuleNotFoundError: No module named 'chromadb'" when I tried to run PrivateGPT, any idea how to fix this or download the module?
@@constructiveinterferencech118 there are several module not found errors, when i do all of them i end up getting ERROR: Failed building wheel for llama-cpp-python as the primary error
I tried ingesting a txt file with products and descriptions and the model its awfull with accuracy, everybody its saying it on the github issues page, some ppl just put a simple 3 line table with data into a txt and it fails to answer coherent things about it. Im using vicuna-13b as its mentioned in this video to be one of the most powerfull ones, but still get sensless answers. Anyone with this same problem?
do your one click installers work on linux? or would i be better off doing the manual install? If i replace the model in the models folder manually with the Script pick up on that on its own when i restart it? or would i have modify a setup file of some sort? This is pretty cool. Im setup With Vicuna 7b GPU local on the desktop, AutoGPT on the msi 4gb laptop, This seems like a good targetd short term trainable model. watching your videos have sent me down a rabit hole. Ill be nixed by a home made AI robot in no time! lmfao
I'm a linux guy too. Oobabooga runs great om Linux and makes pretty much all llms from The Bloke easy to run. I doubt the one click wouod work, but you could probably copy the script into chatgpt and ask it to convert it to your Linux version lol. Ive had chatgpt write me setup scripts for all kinds of stuff.
What directory does this install everything in? I noticed a variable called $THCT or something like that (which I know is inherited from a diff script). I don’t use conda, but I want to use venv to activate a virtual environment for whatever directory this script is going to be working from so that all dependencies aren’t installed globally. How can I go about that?
Hello Sir, I've been try this for hours, and found out the limitation of it token. Sometimes, the answer given not complete! How can we modified it so that the answer given complete? Is it related to token? if yes, how to modified the token limit? tq
it doesnt work for me. Error "Invoke-RestMethod: Die angeforderte Adresse ist in diesem Kontext ungültig." in engl. probably "Invoke-RestMethod: The requested address is invalid in this context." PowerShell 7.3.5
The point of using conda is to use a separate python version and environment for the install than what you currently have installed in windows. If you have conda and say Y to use it, it will create an environment using python 3.10.6.
upon ingest I get: PermissionError: [Errno 13] The process cannot access the file because it is being used by another process: 'C:\\Users\\XXXXXXX\\AppData\\Roaming\ ltk_data\\tokenizers\\punkt.zip'
Hello again, with every other try of the loading of any other model, except for the gpt4all, i get the following error Could not load Llama model from path: models/koala-7B.ggmlv3.q4_0.bin. Received error (type=value_error) what shall I do? Thank you
cambiaste el nombre del modelo en el archivo .env ? en el parametro "MODEL_PATH=models/ggml-gpt4all-j-v1.3-groovy.bin" ahi debes cambiar por el nombre del modelo que quieres usar
tc.ht is my website, and it pulls from the open source repo of mine, here: github.com/TCNOco/TcNo-TCHT Anyone can create a PR for new files, or improvements and they should be live on the website as soon as approved.
If you ingest "to kill a mockingbird" into this and ask "how many chapters does to kill a mocking bird has" it will get confused and spit out all sorts of incorrect answer. If you ingest your latest tax return in pdf format, ooof... good luck with the answer, LOL!!!!
In the description: ➡️ My Setup: hub.tcno.co/faq/my-hardware/ 🎨 My Themes & Windows Skins: hub.tcno.co/faq/my-windows/ 👨💻 Software I use: hub.tcno.co/faq/my-software/
its only mildly useful. the model imposes very strong bias onto the source material. I fed it the entire KJV Bible and asked it what the plan for salvation was and it gave me Catholic sacraments .... LOOOL!
Thank you. Very interessting, but it stuck: Progress: 100% - Completed download of C:\Users\Markus\AppData\Local\Temp\chocolatey\visualstudio2022buildtools\117.6.3\vs_BuildTools.exe (3.6 MB). Download of vs_BuildTools.exe (3.6 MB) completed. Hashes match. Installing visualstudio2022buildtools... Nothing after 5 minutes. I have already installed Visual Studio Code V. 1.80.1 installed, under Windows 11. Strange, I had two processes of installing Visual Studio Code and closed one process with task manager. Now it's installing cuda_11.8.0_522.06_windows.exe.
I'm getting the following error: Using embedded DuckDB with persistence: data will be stored in: db llama.cpp: loading model from models/ggml-vic13b-uncensored-q4_0.bin error loading model: unknown (magic, version) combination: 67676a74, 00000003; is this really a GGML file? llama_init_from_file: failed to load model any ideas guys :(
I got an error building wheels for llama-cpp-python and hnswlib. I don't have Visual Studio installed. EDIT: Tried installing Visual Studio 2022, but it made no difference.
Subscribed because you were cool enough to make a 1-click installer. Please continue doing so for future projects!
Yeah sure. You don't know if it does what he claims it does. For all I know it probably runs some malware or cryptominer in the background.
Build it from the source directly
Note. While this is CPU bound, It also has a processor core limit of 63, so if you have a beefy server or threadripper you might run into issues. You will have to go into your bios and shut some cores off. Even with ~60ish cores, this is currently SUPER SLOW, and gets slower the larger your document database gets. I ingested a directory of ~ 500 MB of PDFs, and it took nearly 30minutes per query.
how about 8-30cores?
It really need GPU support.
Are all local vector databases gonna be limited like this or only privategpt?
Yes, it is was too slow to be practical. However I was surprised that it actually worked ;-)
Does it make it faster to run on the "lightest" model? And perhaps csv docs?
Personally I see that this project has a lot of value but I think the priority should be on how to scale the infrastructure to speed up ingestion and inference times, rapidly prototype and test a bunch of LLMs and last but not the least if it can have a nice looking UI.
Amazing project, im sure will help a lot of people like me.
1-click to install, 30 minutes per query, sounds legit ;P
Hey, I'm receiving a "C:\tools\miniconda3\envs\pgpt\python.exe: can't open file 'C:\\TCHT\\privateGPT\\privateGPT.py': [Errno 2] No such file or directory" error because there are no .py files, is anyone else getting this issue?
Same. Have you been able to find it on any sources. The script it the best path otherwise n
@@BetterThanTV888 How did you get it to work / workaround?
I'm running into the same issue here
This doesn't work in the same way as a NN LLM then... it doesn't "train" on the data and use tokens it seems.. it just searches the document and returns an answer... what am I missing here?
Genius. Much respect and thx
So you need previously anaconda installed on your windows right?
Thanks for the clear video and the work.
No, it handles both the correct Python installed, and even no python installed. If you have an incompatible Python, you'll need to change it.
If you want multiple Python versions, Conda is the best solution, and this script handles that as well.
How can I undo the one-click installation? Thank you.
I tried installing this with the Koala model (because I haven't tried that one before) and it had a problem. It was asking me if it was a ggml model, then just dropping out of the installation. It wouldn't start. I eventually reinstalled it with Vicuna 13b and it works. Can't wait for a faster version, though, as even with 34gb of ram it is a little slow.
hi there :) another question, how to install more models and, further, how to switch between the models to compare the results as you did? thank you very much, highly appreciate the video and your responses in advance
Unfortunately - this one doesn't work at all for me (where your others have) - across 18 different Win10 and Win11 machines, including a Win10 fresh, fully updated, and nothing else on it when started.
The installation fails. It keeps saying that the conda environment was removed (I installed in Drive E).
same here
Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases.
How can I solve this guys?
I hope you made a updated video of this, if it ever got GPU-accelerated support of it.
did u get?
I was looking for something like this to use with AutoGPT, Anyone know a fork out there for Local GPT with the Auto like feedback loop?
Everyone talks about private gpt being able to talk to your docs, but like I already read my docs, can you make it write some statements of claims, or some written submissions or something like that?
Sir, Good Job! But the interface so boring, hopefully in future, it will be more interactive UI
My idea to developer out there... please combine Chatgpt+privateGPT = easily update knowledge base. I really wanted my ChatGPT can be customize + base knowledge
The most problem is not its speed. It cannot retrieve the correct answer in the document.
Many thanks
Is there a way to decrease the time it takes for a model to provide an answer? Like it takes 2 to 3.5 minutes for a vic7b, for example, to provide the response, is there anything that can be done on the matter?
I see, it is using GPT4All and Longchain
wow, can you make AMDs rocm for windows available? It would be huge for any AMD graphics card owner for doing AI art but you probably know that.
Can you automate all that ?
Best youtuber and voice :3 ever in my opinion ^^ keep it up
"Index not found, please create an instance before querying " how to reslove this issue, i'm not cod developer
This would be great for java class
Hi, thank you very much for the tutorial. Can I receive answers in other languages as well? How do I add the option to another language? Thank you.
how to use PrivateGPT + langflow pls
Hi, i,m getting this error, Failed building wheel for llama-cpp-python
How about uninstalling everything if you decide not to use it anymore? Has there a PrivateGPT One-Click UNInstaller ?
what's the point if with 10gb of ram I've been waiting for 1 hour almost and still no answer ?
Thank you for the tutorial. is there any way to use a GUI like "Text Generation Web UI".
Got error python exited with an error code.
Then i reinstalled multitool then got a bunch of errors - any suggestions?
Option 7 does not install gpt4all. It installs the same thing as option 6. Is that on purpose?
What about installing it on a Macbook M1?
Tricky, but I have seen it done on youtube
No joy. Same issue I've had running it after a manual install.
Could not load Llama model from path: models/ggml-vic13b-uncensored-q4_0.bin. Received error (type=value_error)
Thanks anyway, man. :/
"Unable to connect optimized C data functions [No module named '_testbuffer'], falling back to pure Python"
???
No way I'm gonna search and put so many individual files in a folder there's gotta be like some kind of datasets for this that has a sht ton of sht for privategpt to just know/learn from
Also is there a machine learning version of this? Would be cool to have a version that can learn from datasets as well as reach out to the internet for additional data/info and then be rewarded or punished for getting answers right or wrong etc.
I have an index not found error
hi thx a lot for ur tuto. Vicuna 13b can generate multilingual response ? i need french answers
Thank you so much for the one liner! I got an error "ModuleNotFoundError: No module named 'chromadb'" when I tried to run PrivateGPT, any idea how to fix this or download the module?
If you’re in vsc, try changing the Python interpreter. If that doesn’t work, I uninstall and try again. Pay attention to the install location.
input "pip install chromadb"
@@constructiveinterferencech118 there are several module not found errors, when i do all of them i end up getting ERROR: Failed building wheel for llama-cpp-python as the primary error
@@tejaskulkarni2907 same here. looks like a CMAKE issue.
Are there already models with different main languages than english available, that could be used with it?
Is there a way to get PrivateGPT to write a longer article? Settings? It seems to generate 3 paragraphs and 1 additional sentence.
How do I use Vicuna 7B GPU with training / ingesting my own pdf files?
A question for noobs. Is it possible to connect PrivateGPT with AutoGPT?
I tried ingesting a txt file with products and descriptions and the model its awfull with accuracy, everybody its saying it on the github issues page, some ppl just put a simple 3 line table with data into a txt and it fails to answer coherent things about it. Im using vicuna-13b as its mentioned in this video to be one of the most powerfull ones, but still get sensless answers. Anyone with this same problem?
tengo el mismo problema, todas las respuestas no tiene sentido es como que si no esta contestando con la información de los documentos proporcionado.
Can the installation be done with Powershell instead of Visual Studio Code?
do your one click installers work on linux? or would i be better off doing the manual install? If i replace the model in the models folder manually with the Script pick up on that on its own when i restart it? or would i have modify a setup file of some sort? This is pretty cool. Im setup With Vicuna 7b GPU local on the desktop, AutoGPT on the msi 4gb laptop, This seems like a good targetd short term trainable model. watching your videos have sent me down a rabit hole. Ill be nixed by a home made AI robot in no time! lmfao
I'm a linux guy too. Oobabooga runs great om Linux and makes pretty much all llms from The Bloke easy to run. I doubt the one click wouod work, but you could probably copy the script into chatgpt and ask it to convert it to your Linux version lol.
Ive had chatgpt write me setup scripts for all kinds of stuff.
@hardbrocklife I have since figured it out an more!
Thank ya sir!
@hardbrocklife I have since figured it out an more!
Thank ya sir
Do I need for this GeForce GPU?
Can Private GPT process embedded Documents like E-Mail attachments?
Hey the link to the GitHub link in the description goes to the issue tracker
hello again, it worked up untill I chose the same model as you did, 7, gpt4all
have you pushed its limits to see the maximum number of pages it can process?
What directory does this install everything in? I noticed a variable called $THCT or something like that (which I know is inherited from a diff script). I don’t use conda, but I want to use venv to activate a virtual environment for whatever directory this script is going to be working from so that all dependencies aren’t installed globally. How can I go about that?
Does it work with documents in Italian Language? Or only english?
Is there a way to use this as customer support on a website?
Hello Sir, I've been try this for hours, and found out the limitation of it token. Sometimes, the answer given not complete! How can we modified it so that the answer given complete? Is it related to token? if yes, how to modified the token limit? tq
Does this work for other languages as well?
It would be possible to also take sql database?
*Can someone make program to "ingest" PDFs, documents for my private AI?*
I don't want to program anything.
"Enter the Python program name (e.g. python3, python310):" what do i do?
it doesnt work for me. Error "Invoke-RestMethod: Die angeforderte Adresse ist in diesem Kontext ungültig." in engl. probably "Invoke-RestMethod: The requested address is invalid in this context."
PowerShell 7.3.5
irm : Unable to connect to the remote server
The link is broken, please look into it
Tried it on my 2018 MacBook... rookie mistake
What if I have python 3.11
The point of using conda is to use a separate python version and environment for the install than what you currently have installed in windows. If you have conda and say Y to use it, it will create an environment using python 3.10.6.
Does this have a gui model?
no, terminal only!
upon ingest I get:
PermissionError: [Errno 13] The process cannot access the file because it is being used by another process: 'C:\\Users\\XXXXXXX\\AppData\\Roaming\
ltk_data\\tokenizers\\punkt.zip'
Try run as administrator, right click the bat file.
Hello again, with every other try of the loading of any other model, except for the gpt4all, i get the following error
Could not load Llama model from path: models/koala-7B.ggmlv3.q4_0.bin. Received error (type=value_error)
what shall I do?
Thank you
cambiaste el nombre del modelo en el archivo .env ? en el parametro "MODEL_PATH=models/ggml-gpt4all-j-v1.3-groovy.bin" ahi debes cambiar por el nombre del modelo que quieres usar
how do u use iex irm is there any site to add ur own script ??
tc.ht is my website, and it pulls from the open source repo of mine, here: github.com/TCNOco/TcNo-TCHT
Anyone can create a PR for new files, or improvements and they should be live on the website as soon as approved.
I have played a bit on my workstation. It is too slow 😂
I need a GPU version, this is so slow!
Yes, too slow for inference to be useful
...but it also hallucinates like crazy
i lost all my desktop files
If you ingest "to kill a mockingbird" into this and ask "how many chapters does to kill a mocking bird has" it will get confused and spit out all sorts of incorrect answer. If you ingest your latest tax return in pdf format, ooof... good luck with the answer, LOL!!!!
128 gigs of ram !? tf you do with your machine bro , also can u tell all the specs 🧐
In the description:
➡️ My Setup: hub.tcno.co/faq/my-hardware/
🎨 My Themes & Windows Skins: hub.tcno.co/faq/my-windows/
👨💻 Software I use: hub.tcno.co/faq/my-software/
@@TroubleChute it says content not found 😭
@@TroubleChute nvm got it 😾🙏
How to enable the GPU?
Afaik ggml models always work with CPU. I'm interested in the same question.
Oh great. Looking for local and not their cloud services or API.
That's what this is. It's using local models. Waiting for the repo to support GPU accelerated models though.
@@rickie_ +1 Waiting for GPU support as well.
How much faster would it run on GPU?
your characters are for ants ! ...
Why would you choose a CPU only version to use, that's just silly
Not everyone can afford GPUs
Aaaand this project doesn't support GPUs just yet - But this is being worked on
@@TroubleChute Can you please make a one-click installer for this? many thanks
its only mildly useful. the model imposes very strong bias onto the source material. I fed it the entire KJV Bible and asked it what the plan for salvation was and it gave me Catholic sacraments .... LOOOL!
Very, very slow
FIRST! pin me sir
Thank you. Very interessting, but it stuck:
Progress: 100% - Completed download of C:\Users\Markus\AppData\Local\Temp\chocolatey\visualstudio2022buildtools\117.6.3\vs_BuildTools.exe (3.6 MB).
Download of vs_BuildTools.exe (3.6 MB) completed.
Hashes match.
Installing visualstudio2022buildtools...
Nothing after 5 minutes. I have already installed Visual Studio Code V. 1.80.1 installed, under Windows 11.
Strange, I had two processes of installing Visual Studio Code and closed one process with task manager.
Now it's installing cuda_11.8.0_522.06_windows.exe.
I'm getting the following error:
Using embedded DuckDB with persistence: data will be stored in: db
llama.cpp: loading model from models/ggml-vic13b-uncensored-q4_0.bin
error loading model: unknown (magic, version) combination: 67676a74, 00000003; is this really a GGML file?
llama_init_from_file: failed to load model
any ideas guys :(
I got an error building wheels for llama-cpp-python and hnswlib. I don't have Visual Studio installed. EDIT: Tried installing Visual Studio 2022, but it made no difference.
did you find a solution to this?
@@tejaskulkarni2907 interested to know if you had any success; I have the same issue
@@TedHawk No.