I got the issue fixed and want to share it here - i type poetry install --extras "ui embeddings-huggingface llms-llama-cpp vector-stores-qdrant" instead and it works.
Nice video! I got it installed, but is there a way to improve the performance of the response? It's often very slow, even if I run it on an RTX 4090 and an i9 13k.
Thank you for the tutorial. Could you give a hint, though, how should fix the ERROR "Current Python version (3.12.3) is not allowed by the project (>=3.11,
I followed the how you instructions until on the part where or at 7:40 min on the video , that to type in ---> poetry install --with ui,local and man i got this error ---> Group(s) not found: local (via --with), ui (via --with). Need help on this part
Legend! Thanks a lot! I spent a ton of time trying to find a set of instructions that actually work but to no avail. Until I stumbled across your video. Thanks again!
Thank you for your videos. As a subscriber to your channel, I look forward to each new one. I would like to make a comment, and perhaps a suggestion for a future video. As a Finance Manager for the last 40+ years, I have come to find these three areas important as it relates to the retrieval of information. What every office need is a way to search: 1) URL's , EXCEL, and PDF on the internet. 2) EXCEL and CSV files locally. (and Securely) 3) PDF and DOC documents locally. (and Securely) And nowadays, you could never get a NON GUI program adopted by the office staff. And of course, all three of the search types would be incorporated into the same GUI. Python based program would be the preferred language. Ollama based. No wrapper programs like streamlit, etc. Thanks for your time, and keep up the good work.
That's a good question. When I did "sudo update-alternatives --config python3" and set the system version to 3.11, Terminal stopped working. When I set it back to 3.12 (the apparent default for 24.04), Terminal worked again.
@Novaspirit Tech I am having issues with the poetry install --with ui command. I keep getting an error stating that Poetry could not find a pyproject.toml file. Any ideas on how to resolve that?
I don´t know what is wrong but I cannot install the segment with: python3 -m pip install pip, it just doesn´t work, even after changing the: export PATH ="~/.local/bin:$PATH" and i have python3.11.8
What an absolute nightmare to install this is impossible unless you are a dev I followed this to the letter over 3 different guides and got the furthest with this one but what a nightmare
Don: thanks for this! Personally I’m deeply into this topic, so I’ll devour everything you produce on this topic. Obviously, don’t please me personally, but I think you’re at the beginning of a big incoming wave. Hack til it hurts, yes!
this is huge and will continue to improve and spread...particularly for commercial apps like lawyer firms and medical practices - expect the LLM ecosystem o expand a ton and things like cxl to kick in and turbocharge the growth in a couple of years - things like models being uncensored, real time and p2p model sharing and datasets will happen and also turbocharge growth - we really aren't there yet - it is going to take 4-5 years to really roll out to entire sme/smb mkt but it should be an interesting ride to say the least. why the limits on tokens? i would like to see things api independent especially with local builds - also it limits the amount of info you can pile into it which is a major limitation
The implications are revolutionary, and I don't think people especially government realize this. Government will clamp down on AI as soon as they realize the economic effects, I can already smell the stench of lobbyists. Corporations want all the data, especially personal data. I can't see them giving up easy. It's hard enough to lobby in favor of right to repair, fixing phone screens is easy for anyone to understand, but AI is like magic and just so easy to use fear to manipulate policymakers.
i am relatively new to this topic and have a question. how can i run this on a server, so that it runs independent from a specific PC? I want to employ it companywide
It took a bit, I had some hurdles that you didnt have in the video with permissions an installs that didnt exist, but I got through and it was running. then i rebooted and its not running anymore. not sure how to start it back up again also how to make it start on boot. can you help?
what hardware are you using for this demo? i see an rpi keyboard but those could be used as an external keyboard. the documentation mentions using llama.cpp for mac os or nvidia gpu support for windows & linux. this looks promising.
i think its updated...some of tutorial in this i have an error and i fixed it one by one... for instance the installation of poetry ui is changed to --extras
Very interesting. I don't have available any kind of tower PC with graphics card though. I have a Dell Lattitude laptop running Ubuntu. Given the complexity of the setup I'd rather do stuff in a Docker container, or LXD container. Any thoughts on that? I see on Docker Hub several privategpt containers but they haven't been updated anytime recently or else do not have documentation.
with poetry run scripts/setup I am getting authentication errors. It says that I am not allowed to access the mistral7b. even with "poetry run huggingface-cli login --token mytoken" I am unable to finish that step. is there a way around it?
figured it out myself: 1. create account at huggingface 2. under settings create a read access token 3. accept the message at mistral7b 4. edit setup file: hf_hub_download( repo_id=settings().llamacpp.llm_hf_repo_id, filename=settings().llamacpp.llm_hf_model_file, cache_dir=models_cache_path, local_dir=models_path, resume_download=resume_download, token="mytoken", # add this ) AutoTokenizer.from_pretrained( pretrained_model_name_or_path=settings().llm.tokenizer, cache_dir=models_cache_path, token="mytoken", # add this ) 5. run scipts/setup again
Realizing upfront this would not work in my environment (Linux VM within ProxMox on Intel N100) I followed your steps. Low and behold it worked up to uploading a document. When I submit a prompt I see that the program in the terminal crashes immediately. Probably because of no NVIDIA CUDA available in my config. If you know of an alternative that doesn't need a GPU, I would love to try.
Hey, huge thanks for all the work. I just have question at 7:30. My result is Group(s) not found: local (via --with), ui (via --with). Can somebody help please?
I followed your steps on Linux Mint 21.3, but now that I'm running it the Browser, I get this Error when trying to use prompts: "NoneType' object has no attribute 'split". What could I have done wrong?
Do you have this installed on a Windows 11 VirtualBox that hasn't been licensed yet. If you can pre-install it and make it available for download. I can get my own Windows license.
Excelente explicación. Gracias a tu aporte lo tengo funcionando en WLS Ubuntu-22.04 Si alguien desea revisar archivos de tipo docx Tan solo debe ejecutar el siguiente comando, dentro del (venv) poetry add docx2txt
it's working, thanks for that. But... how to do this to work after reboot? do i have to start those few commands after every reboot? Maybe someone have this sorted out?
I actually got it working on an old laptop without a GPU. It was slow but it worked. I just did all the other steps except the GPU steps and it worked.
The newest version of what? This is a project on github and not much has changed that I can tell from the documentation. What exactly are you talking about? After reading the documentation, it's not confusing at all.
Hey bud, I appreciate you showing us however, could you explain more about what private GPT is? And what are the benefits? Is it proxy to chatgpt? What are the hardware requirements? NVIDIA?
@@ChargedPulsar Preferably yes. Predipped in milk too. Do you enjoy being an ass online? Is real life hard for you so that you feel the need to act like an ass online to feel big and strong again?
i followed your video. worked well, however after restart i cannot run it anymore... when i execute PGPT_PROFILES=local make run i got following error... poetry run python -m private_gpt make: poetry: No such file or directory make: *** [Makefile:36: run] Error 127 working on OpenSuse. any idea??
Evening Don, This is interesting and I may well follow your lead on this. Also, just so you are aware, I followed your previous PiMox install video to the letter but could not make it work. It came up with two package missing errors and failed to run, even though it wasnt immediately apparent there were problems with the install process. Cheers, Peter.
not sure if the pimox works with current kernel for the pi and might have to ignore the install of the kernel in the install script, i was just looking into that to get prox 8 on pi
@@NovaspiritTech Well done Don, I have a network of currently 5 or so Pi 4B 8 Gb in Pi NAS Server cases and one NUC 12 Pro i7. The NUC is running Proxmox 8 and the Pi's the traditional Docker and Portainer via DietPi. I would really like all servers to be running Proxmox and be controllable from one console. Ta, Peter.
What does this do except make you own computer use more electricity? I mean compared to the free ChatGPT that run on thousands of GPUs on some supercomputer gpu cluster. Why use local resources for something that ChatGPT does and is free (3.5)?
I got the issue fixed and want to share it here - i type poetry install --extras "ui embeddings-huggingface llms-llama-cpp vector-stores-qdrant" instead and it works.
Nice video! I got it installed, but is there a way to improve the performance of the response? It's often very slow, even if I run it on an RTX 4090 and an i9 13k.
Thank you for the tutorial. Could you give a hint, though, how should fix the ERROR "Current Python version (3.12.3) is not allowed by the project (>=3.11,
you need python 11
7:39 Timestamp: poetry install --with ui, local: Group(s) not found: local (via --with), ui (via --with)
Seems that these commands no longer work
try poetry install -E ui
That worked for me. Thank you!@@fierylight2009
Framework updated. Check the official website.
Any luck finding direction with this?
@@fierylight2009 thank you so much!!!!!!!😁
I followed the how you instructions until on the part where or at 7:40 min on the video , that to type in ---> poetry install --with ui,local and man i got this error ---> Group(s) not found: local (via --with), ui (via --with). Need help on this part
do poetry install --extras "ui embeddings-huggingface llms-llama-cpp vector-stores-qdrant"
Legend! Thanks a lot! I spent a ton of time trying to find a set of instructions that actually work but to no avail. Until I stumbled across your video. Thanks again!
Thank you for your videos. As a subscriber to your channel, I look forward to each new one. I would like to make a comment, and perhaps a suggestion for a future video. As a Finance Manager for the last 40+ years, I have come to find these three areas important as it relates to the retrieval of information. What every office need is a way to search:
1) URL's , EXCEL, and PDF on the internet.
2) EXCEL and CSV files locally. (and Securely)
3) PDF and DOC documents locally. (and Securely)
And nowadays, you could never get a NON GUI program adopted by the office staff. And of course, all three of the search types would be incorporated into the same GUI.
Python based program would be the preferred language. Ollama based. No wrapper programs like streamlit, etc. Thanks for your time, and keep up the good work.
Why do you change the python version system wide? Can't you just install 3.11 and create the virtual environment like this "python3.11 -m venv venv"?
That's a good question. When I did "sudo update-alternatives --config python3" and set the system version to 3.11, Terminal stopped working. When I set it back to 3.12 (the apparent default for 24.04), Terminal worked again.
@Novaspirit Tech I am having issues with the poetry install --with ui command. I keep getting an error stating that Poetry could not find a pyproject.toml file. Any ideas on how to resolve that?
3 minutes in we find out this is for a MAC. 😞
I don´t know what is wrong but I cannot install the segment with: python3 -m pip install pip, it just doesn´t work, even after changing the: export PATH ="~/.local/bin:$PATH" and i have python3.11.8
how can we fix a path of folder, so that it automatically captures the files in that folder. Instead of manually uploading all the files ?
why didn't you use a venv for installing python 3.11 ?
this is gold, thank you very much and keep up the brilliant work man
I am a bit wondering why this did not go into a container instead of a venv, since there's dependency twists before this?
Is this available in a container?
What an absolute nightmare to install this is impossible unless you are a dev I followed this to the letter over 3 different guides and got the furthest with this one but what a nightmare
Don: thanks for this! Personally I’m deeply into this topic, so I’ll devour everything you produce on this topic. Obviously, don’t please me personally, but I think you’re at the beginning of a big incoming wave.
Hack til it hurts, yes!
Could you share the hardware and OS specs used in this demo? Or link to a vid where you built it?
i mention it in the video, nvidia 1070, amd 1700x, and 22.04 ubuntu
this is huge and will continue to improve and spread...particularly for commercial apps like lawyer firms and medical practices - expect the LLM ecosystem o expand a ton and things like cxl to kick in and turbocharge the growth in a couple of years - things like models being uncensored, real time and p2p model sharing and datasets will happen and also turbocharge growth - we really aren't there yet - it is going to take 4-5 years to really roll out to entire sme/smb mkt but it should be an interesting ride to say the least. why the limits on tokens? i would like to see things api independent especially with local builds - also it limits the amount of info you can pile into it which is a major limitation
The implications are revolutionary, and I don't think people especially government realize this. Government will clamp down on AI as soon as they realize the economic effects, I can already smell the stench of lobbyists. Corporations want all the data, especially personal data. I can't see them giving up easy. It's hard enough to lobby in favor of right to repair, fixing phone screens is easy for anyone to understand, but AI is like magic and just so easy to use fear to manipulate policymakers.
i am relatively new to this topic and have a question. how can i run this on a server, so that it runs independent from a specific PC? I want to employ it companywide
It took a bit, I had some hurdles that you didnt have in the video with permissions an installs that didnt exist, but I got through and it was running. then i rebooted and its not running anymore. not sure how to start it back up again also how to make it start on boot. can you help?
what hardware are you using for this demo?
i see an rpi keyboard but those could be used as an external keyboard. the documentation mentions using llama.cpp for mac os or nvidia gpu support for windows & linux.
this looks promising.
Actually looks to me like a 400, guessing he's simply using that to access the web gui :).
Care to watch the video? Answer is there
Thanks a lot
When we are asking to summarise it summarising only few pages where as i have uplodede pdf of 20 pages.
How well does it work with datasheets? I'm thinking about uploading datasheets of some electronic component and get help with writing drivers
i think its updated...some of tutorial in this i have an error and i fixed it one by one... for instance the installation of poetry ui is changed to --extras
The 1080 still delivers in 2023
10:15 No arguments expected for "install" command, got "ui"
Are these the world's most crispy fries ? Let's find out 13:01
Sorry but this guide is garbage i followed it step by step and had multiple issules. aftr 3 days i give up
how to eject ingested files
Thank you for sharing.
if you can come back on me on this or anyone for me to fix this i highly appreciate it
this is an alternative to ollama ?
i'm new to this ai thing
Very interesting. I don't have available any kind of tower PC with graphics card though. I have a Dell Lattitude laptop running Ubuntu.
Given the complexity of the setup I'd rather do stuff in a Docker container, or LXD container. Any thoughts on that? I see on Docker Hub several privategpt containers but they haven't been updated anytime recently or else do not have documentation.
That's exactly what I was thinking. Some containers and installation process is done.
why you dont just use pyenv? it is much simpler
with poetry run scripts/setup I am getting authentication errors. It says that I am not allowed to access the mistral7b. even with "poetry run huggingface-cli login --token mytoken" I am unable to finish that step.
is there a way around it?
figured it out myself:
1. create account at huggingface
2. under settings create a read access token
3. accept the message at mistral7b
4. edit setup file:
hf_hub_download(
repo_id=settings().llamacpp.llm_hf_repo_id,
filename=settings().llamacpp.llm_hf_model_file,
cache_dir=models_cache_path,
local_dir=models_path,
resume_download=resume_download,
token="mytoken", # add this
)
AutoTokenizer.from_pretrained(
pretrained_model_name_or_path=settings().llm.tokenizer,
cache_dir=models_cache_path,
token="mytoken", # add this
)
5. run scipts/setup again
Realizing upfront this would not work in my environment (Linux VM within ProxMox on Intel N100) I followed your steps. Low and behold it worked up to uploading a document. When I submit a prompt I see that the program in the terminal crashes immediately. Probably because of no NVIDIA CUDA available in my config. If you know of an alternative that doesn't need a GPU, I would love to try.
you will need to use a llm that is for your CPU and compile the llama.cpp for cpu instead of cuda
Please elaborate on that. How do we compile llama.cpp for CPU instead?
CMAKE_ARGS="-DLLAMA_BLAS=ON -DLLAMA_BLAS_VENDOR=OpenBLAS" pip install llama-cpp-python
@@vipinpg Thank you! I am going to try that!
@@JDSchuitemaker try new version of privategpt. It uses ollama insted of llamacpp it is easy. I alreaddy developed few apps in production.
no amd gpu supported
Hey, huge thanks for all the work. I just have question at 7:30. My result is Group(s) not found: local (via --with), ui (via --with). Can somebody help please?
yes they changed that a bit with is now extras. Look here in the comments or search for the error on google.
@@LatvianGangsta Thanks mate
I followed your steps on Linux Mint 21.3, but now that I'm running it the Browser, I get this Error when trying to use prompts: "NoneType' object has no attribute 'split". What could I have done wrong?
Did this work for anyone who tried? I'm having installation problems with exact steps on a clean install.
Do you have this installed on a Windows 11 VirtualBox that hasn't been licensed yet. If you can pre-install it and make it available for download. I can get my own Windows license.
poetry install -E ui
Excelente explicación. Gracias a tu aporte lo tengo funcionando en WLS Ubuntu-22.04
Si alguien desea revisar archivos de tipo docx
Tan solo debe ejecutar el siguiente comando, dentro del (venv)
poetry add docx2txt
it's working, thanks for that. But... how to do this to work after reboot? do i have to start those few commands after every reboot? Maybe someone have this sorted out?
this just broke my terminal after installing python 3.11
Thank you so much man,,,, this is incredibly useful ... fire content.. a definite subs and follower
Thanks for the video! 5 minutes in and already super helpful. sub'd
Do you have to have a GPU to run the model?
I actually got it working on an old laptop without a GPU. It was slow but it worked. I just did all the other steps except the GPU steps and it worked.
Thank you. Any special when Python is already installed, for other issues?
What i do for alt python is just name it python310 or python311 and i even still even use python2 lol
Not offline?
bro free ah idhu ?
You are confusing everybody, you could have just downloaded the latest version instead of all these confusing mombo jombo
The newest version of what? This is a project on github and not much has changed that I can tell from the documentation. What exactly are you talking about? After reading the documentation, it's not confusing at all.
Hey bud, I appreciate you showing us however, could you explain more about what private GPT is? And what are the benefits? Is it proxy to chatgpt? What are the hardware requirements? NVIDIA?
I ran into this error. Cuda out of memory. how to solve this. Please help
What is your card?
Watching this all confused, I really wish PrivateGPT could just make a 1 click installer that does everything. I am so lost lol
Do you want free cookies delivered to your address as well?
@@ChargedPulsar Preferably yes. Predipped in milk too.
Do you enjoy being an ass online? Is real life hard for you so that you feel the need to act like an ass online to feel big and strong again?
i followed your video. worked well, however after restart i cannot run it anymore...
when i execute PGPT_PROFILES=local make run i got following error...
poetry run python -m private_gpt
make: poetry: No such file or directory
make: *** [Makefile:36: run] Error 127
working on OpenSuse. any idea??
Having the same issue... did you manage to figure it out?
yes i did. i have installed once again poetry . but i am running OpenSuse Thumbleweed and its a rolling distro... @@AndreiPetrescu
@@AndreiPetrescu add poetry to path variable.
Evening Don, This is interesting and I may well follow your lead on this. Also, just so you are aware, I followed your previous PiMox install video to the letter but could not make it work. It came up with two package missing errors and failed to run, even though it wasnt immediately apparent there were problems with the install process. Cheers, Peter.
not sure if the pimox works with current kernel for the pi and might have to ignore the install of the kernel in the install script, i was just looking into that to get prox 8 on pi
@@NovaspiritTech Well done Don, I have a network of currently 5 or so Pi 4B 8 Gb in Pi NAS Server cases and one NUC 12 Pro i7. The NUC is running Proxmox 8 and the Pi's the traditional Docker and Portainer via DietPi. I would really like all servers to be running Proxmox and be controllable from one console. Ta, Peter.
Send me the coffee
What embedding model is it using?
By default version use BAAI/bge-large-en
New one useses nomic-embed-text with ollama
does it work without nvidia card?
Yes
does it support other languages besides english?
yes depends on your model
@@NovaspiritTech thanks...👍
thank you
great video
Thanks!🎉
👍❤️
What does this do except make you own computer use more electricity? I mean compared to the free ChatGPT that run on thousands of GPUs on some supercomputer gpu cluster. Why use local resources for something that ChatGPT does and is free (3.5)?
privacy
So I can draw from my own documents to train output. Like: Write a new novel in the style of my old novels, about the era of …
The same reason you had your own locker at school and not a shared one.
Other than privacy: fun, learning, tinkering, etc.