Llama-CPP-Python: Step-by-step Guide to Run LLMs on Local Machine | Llama-2 | Mistral

Hi, My name is Sunny Solanki, and in this video, I provide a step-by-step guide to running Local LLMs using Python library "llama-cpp-python". I have explained how to use llama-2 and mistral models on local machine.
============================================
CODE - github.com/sunny2309/llama_cp...
==============================================
=======================================================
SUPPORT US - Buy Me Coffee - buymeacoffee.com/coderzcolumn
=======================================================
=======================================================
NEWSLETTER - eepurl.com/gRW2u9
=======================================================
=======================================================
WEBSITE - coderzcolumn.com
=======================================================
Important Chapters:
0:00 - Llama-CPP-Python
1:19 - Generate Text (Query Response)
4:39 - Stream Response of LLMs
5:40 - Pull Models from HuggingFace
7:26 - Chat Completion
10:01 - Generate Embeddings for RAG Apps
#python #datascience #datasciencetutorial #python #pythonprogramming #pythoncode #pythontutorial #llama2 #llama-cpp-python #llama-cpp #llama-2-7b-chat-gguf #llama-2-7b-chat-locally #llama-2-7b-chat-locally-python #mistral-7b-locally #mistral-7b-locally-python #local-llms

[@CoderzColumn]

Check out another simple to use tool to run Open Source LLMs on a local Machine: "Ollama"
Ollama Tool -> ua-cam.com/video/vfm_Pmxe-z8/v-deo.html
Ollama Python Library -> ua-cam.com/video/1kDa3rWIDBk/v-deo.html
Guide to Build RAG App using Ollama -> ua-cam.com/video/et_EREAsIQE/v-deo.html

[@Wasiun3000]

Nice explanation of this topic, thanks!

[@CoderzColumn]

Glad it was helpful!

[@user-db2zx3ox7u]

The perfect walk through with clear explanations and examples. Thank you for sharing this.

[@CoderzColumn]

Thanks for taking time to comment !! Really appreciate it !!!

[@aipy5147]

Nice, clear, and succinct! Thank you for sharing.

[@CoderzColumn]

Thanks for the feedback!!! Really appreciate it !!!

[@hassubalti7814]

great sir..pleas guide me how to give interface like chainlit and to the model and ask querey please do that big fan from pakistan❤❤❤

[@CoderzColumn]

Thanks for feedback !!! Appreciate it !!!
If you want to link llama-cpp-python with Chainlit then you can refer this video
* ua-cam.com/video/FVh32fVFf3c/v-deo.html
Here I have used Ollama to access LLM but you can replace code from this video to use access LLM using llama-cpp-python. I hope this helps.

[@hassubalti7814]

@@CoderzColumn thank you

[@user-wr4yl7tx3w]

How does it compare versus using Ollama?

[@CoderzColumn]

To access Ollama through Pythn, you first need to install Ollama tool and then Ollama Python library as well. For llama-cpp-python, you only need to install python library.
To use any model, you need to download models in both cases. Using ollama, you can simply download using "ollama pull model_name" command and it'll download from their repo. Using llama-cpp-python, you'll have to search for GGUF file of model (quantized version) through you'll mostly found it on huggingface.
To use Ollama python library, you need to keep ollama server running. To use llama-cpp-python, there is no such requirements.
Performance wise, both are almost same as base is coded in CPP. I haven't tested them on different hardware though.