Hey there Dhaval Sir ! Just finished creating this project, Following along and It was really amazing ! Got to learn a lot in the process. Had some issues with installations and versions at some steps, But solving them helped me learn more. Thank you very much 🙏
Can you please elaborate on what issues you faced at time of installations and versions as when I installed all the packages of the requirements.txt it gives this import error : ImportError: Could not import google-generativeai python package. Please install it with `pip install google-generativeai`. Code : from langchain.llms import GooglePalm api_key = "my api key" llm = GooglePalm(google_api_key=api_key,temperature=0.8) I had installed google-generativeai but it still gives this error
@@DARK-fs3rzI am getting error when I try to load it data=loader.load() error: 'utf-8' codec can't decode byte 0x92 in position 1197: invalid start byte.How to fix it?
Sir, Good morning. The way you teach the concepts are phenomenal . Love you from bottom of my heart. Do this kind of video more in coming days. I know that majority just go by the fancy thumbnail but this kind of video need too much effort. Thank You again!
Hey were you able to complete the project ? Since it's been 6 months I'm worrying whether I can start and finish this project without any outdated stuff
sir, im facing lot of issues with packages used in this project. could you please help me out. its been 3 days since im facing this issue. i have tried in both vsocde and pycharm. please help me out
I am very excited when I watch video and title. It's full fill all the needs. And I have a request to you can you add to read pdf which are present in the website that also read and gave answer regarding that . See most of college of University update there notification panel only like they upload pdf. And if we use this it's only scrape the text data present on the website can it also scrape the pdf and creat knowledge base using all the data present on websit. And if it's done there next big hurdle is some are only upload image so can our ai will read that data like if some students of college got placed they not updated the placement page rather than put banner on the homepage. But overall love your content all the time❤❤
@@kishore961 , im facing lot of issues with packages used in this project. could you please help me out. its been 3 days since im facing this issue. i have tried in both vsocde and pycharm. please help me out
How well it works for Tabular data having columns from SQL DB. Should we use SQlAgent or Vector DB can work. We aren't getting good result from SQLAgent
In prompt template there are two variables context and question. When you are using chain("do you have js course"). Is retriever the context? How will it know which is question and which is context?
Hi Dhaval, there are some functions that you show in the video like "response.text" that does not work. Are there a particular versions that you have used and are now outdated?
This is a great learning. Trying out with the samples but does not seem to be working. Facing errors while compiling with Python 3.12. Any suggestions?
Very Nice video Dhavalbhai. I have question what are the privacy related risks connected to using this approach. Any data breach or anything like that ?
if i have csv and excel file consist numerical and categorical now i want to get output in llm even for numerical also how to do it ? i need ur help to start this
why don't you save the chain like you did with vector db? and use that model. Im suffering from re running it, as in colab i had gpu, on my machine I don't, plus on colab there are libraries which make configuration easy. if you can please share a tutorial managing like that it'll be great
Hi, can you also cover end to end Autonomous AI agent creation and deployment. There is plenty of material online but almost no one covers the topic comprehensively enough.
Sir we are encountering ImportError: Dependencies for InstructorEmbedding not found. Please tell how to resolve it . Can we get a list of all depedencies so we do not have to install them one by one . I have already downloaded the files in requirment.txt
Hello Dhaval, great tutorial as usual ! I have a, probably silly, question (sorry about that). I saw that even after the 'Instruction' prompt has been specified to indicate to the model that if not relevant documents have been retrieved it must answer 'I don't know'... there are some retrieved source documents anyway, why is that? It is clear that model is not using them as a context for its answer, how can he knows that it must not use them? There is some similarity threshold behind? Thanks ! :)
sir, im facing lot of issues with packages used in this project. could you please help me out. its been 3 days since im facing this issue. i have tried in both vsocde and pycharm. please help me out
@@kashifahmad9880 yes , make the project in a new virtual environment then install all the packages required , i have used google palm embeddings instead of hugging face instructor embeddedings and some other packages...
@@kashifahmad9880 yes , make the project in a new virtual environment then install all the packages required , i have used google palm embeddings instead of hugging face instructor embeddedings and some other packages...
getting error for line embeddings = HuggingFaceInstructEmbeddings( query_instruction="Represent the query for retrieval: " ) ImportError: Dependencies for InstructorEmbedding not found. any help??
Raise NotImplementedError( NotImplementedError: Need to determine which default deprecation schedule to use. within ?? minor releases....plsss help me to solve these...after 36.13
@@gabbaSLsorry to ask you again were you able to complete the project ? Since it's been 6 months I'm worrying whether I can start and finish this project without any outdated stuff
A doubt - How does the llm get to define the input variables i.e. context and question without being explicitly mentioned by user? We nowhere mentioned that the context should be the vectordb and question should be the user query. Any help would be appreciated! PROMPT = PromptTemplate( template=prompt_template, input_variables=["context", "question"] )
Yes .... Waiting a lottt😍...... Sir in previous Rocky Bot... When I am using the urls which was provided by you...The data from urls is not fetching...( The docs where Empty) But when I tried with other urls i am able to run without any error... Could you please give me a solution why it is happening... Please sirrrr
It can be that it has issues browsing those urls. You can try it again to see it works. Or we need to debug it further to check what the issue is. For me it was working ok without any problems.
@@codebasics Yes sir.... There is issue while browsing the urls.... As you said I have tried multiple times.... More than 20 times... But still not working for the urls you provided... Please help me to slove sir.... I wish I will get proper solution from you.... Soo that i can put this project in my resume.. Because by seeing your videos I have learned all the things related to Langchain+ llms... (Built small small applications to get practical knowledge) ...But not able show them as end to end project... I thought this rockybot application is very good application to showcase in my resume... Please help me sirr
@@chinnibngrm272hello in this question answering project I am getting error when I try to load it data=loader.load() error: 'utf-8' codec can't decode byte 0x92 in position 1197: invalid start byte.How to fix it?
Anyone doing this Project now or has done recently , I need some help actually , Firstly , llm ( ..Prompt..) and InstructEmbeedding these two syntax didnt work , so used HuugingFace embeddings instead , and secondly : I have succesfully completed until the Document chunk retreival step following this code (also created the chain) , But at the last step when With those chunks , it will interact with the LLM , I have tried many times but the Chain("...Question...) syntax is showing error , I have also tried with chain({'query':'question'}) , that also is not working , so stuck at the last stage , anyone may help?
Hi my project was doing well but from few day it is showing Retrying langchain.lIms.google_palm.generate_with_retry.. _generate_with_retry in 60.0 seconds as it raised NotFound: 404 Requested entity was not found.. getting this in logs when i am generating answer.
@Codebasics- Error Message - line 154, in __init__ raise ImportError("Dependencies for InstructorEmbedding not found.") from e ImportError: Dependencies for InstructorEmbedding not found. . Can you pls help me to fix this issue?
Sir can u make videos on how to predict future results with past data in basketball or even football I am trying to create a basketball prediction which predicts the result at any point of time of the match
why vector FAISS db from_document for documents parameter is Docs instead of data from loader.load() in the notebook? I think it's typo but may be Im wrong
sir, im facing lot of issues with packages used in this project. could you please help me out. its been 3 days since im facing this issue. i have tried in both vsocde and pycharm. please help me out
I wish to better understand the issue of context and question prompt_template = """Given the following context and a question, generate an answer based on this context only. In the answer try to provide as much text as possible from "response" section in the source document context without making much changes. If the answer is not found in the context, kindly state "I don't know." Don't try to make up an answer. CONTEXT: {context} QUESTION: {question}""" PROMPT = PromptTemplate( template=prompt_template, input_variables=["context", "question"]
Hey folks 👋 , i have just created this project using updates dependencies and models. Since the video a bit old and there is countinous changes in the libraries we have to change the code a bit if any one want any help you can reply me on this
Hi i getting error now my project is not working Retrying langchain.lIms.google_palm.generate_with_retry._generate_with_retry in 60.0 seconds as it raised NotFound: 404 Requested entity was not found.. getting this in logs when i am generating answer.
Retrying langchain.llms.google_palm.generate_with_retry.._generate_with_retry in 60.0 seconds as it raised NotFound: 404 Requested entity was not found.. gettning this in logs when i am generating answer.
@@abhishekghosh9523 No ,i have not faced any issue ...suppose if you face any issue you can use other embedding techniques like hugging face "sentence-transformers / all-mpnet-base-v2" or "sentence-transformers / all-MiniLM-L6-v2" other transformers to embedding
@@samasrinivasreddy961 Hey thanks for the reply! But already sorted the issue. Presently facing some dependencies issues while trying to run it on vscode, with huggingfaceinstructorembedding. Have you faced something similar?
Check out our premium machine learning course with 2 Industry projects: codebasics.io/courses/machine-learning-for-data-science-beginners-to-advanced
Hey there Dhaval Sir ! Just finished creating this project, Following along and It was really amazing ! Got to learn a lot in the process. Had some issues with installations and versions at some steps, But solving them helped me learn more. Thank you very much 🙏
Great 👍
Can you please elaborate on what issues you faced at time of installations and versions as when I installed all the packages of the requirements.txt it gives this import error :
ImportError: Could not import google-generativeai python package. Please install it with `pip install google-generativeai`.
Code :
from langchain.llms import GooglePalm
api_key = "my api key"
llm = GooglePalm(google_api_key=api_key,temperature=0.8)
I had installed google-generativeai but it still gives this error
@@DARK-fs3rzI am getting error when I try to load it data=loader.load() error: 'utf-8' codec can't decode byte 0x92 in position 1197: invalid start byte.How to fix it?
Excellent 💯
@@amuralikrishna22 Open the csv file, you will find some special characters in prompts. Remove them and try again.
Thanks for the great and very very simple explanation of whole process. The method you explain the concept is extraordinary. Just super !!!
Thank you very very much.I can't explain how important this video was for me.
Sir, Good morning. The way you teach the concepts are phenomenal . Love you from bottom of my heart.
Do this kind of video more in coming days. I know that majority just go by the fancy thumbnail but this kind of video need too much effort.
Thank You again!
I've been looking for a detailed guide on LLM projects, and this video delivers exactly what I needed. Kudos to the presenter.
Hey were you able to complete the project ? Since it's been 6 months I'm worrying whether I can start and finish this project without any outdated stuff
You are in the right pass bro. Keep going and create such amazing content
sir, you are awesome! I wish we we had projects like this during undergrad.
thanks Dhaval sir for another cool end to end project but I think we are missing one thing is that the Deployment part
Thank You So Much dhaval sir. Love from Gujarat ❣
Awsome Dhaval sir...keep educating...Thanks & respect from Bangalore..
Thank you sir . This vedio will very very useful to students. Moreover it is inspiring to those are settling as data scientist
You are most welcome
@@codebasics SIR UDATE DISCORD LINK
This tutorial video helps me to complete my project 'chat with local pdf file'. Thank you 🙏.
So glad it resonated with you! 😊
Hey I have a question, so can we use a Word document instead of a CSV file and make the QA application by following the same.
sir, im facing lot of issues with packages used in this project. could you please help me out. its been 3 days since im facing this issue. i have tried in both vsocde and pycharm. please help me out
Amazing content. Loved it 💥
Thank you for sharing such informative session
Glad it was helpful!
Excellent Sir ... Truly mean it 😍😍😍😍
Awesome explanation
I am very excited when I watch video and title. It's full fill all the needs. And I have a request to you can you add to read pdf which are present in the website that also read and gave answer regarding that . See most of college of University update there notification panel only like they upload pdf. And if we use this it's only scrape the text data present on the website can it also scrape the pdf and creat knowledge base using all the data present on websit. And if it's done there next big hurdle is some are only upload image so can our ai will read that data like if some students of college got placed they not updated the placement page rather than put banner on the homepage. But overall love your content all the time❤❤
at 19:12 you are using "docs" but it is "data" there, how it is working ?
I used docs and got the same behaviour. I pondered for a while and just moved on. :)
@@kishore961 , im facing lot of issues with packages used in this project. could you please help me out. its been 3 days since im facing this issue. i have tried in both vsocde and pycharm. please help me out
thank you bro you re a reference in this domain, might allah guide you and help you .
Thank you! this is a great video!
Hi, i am struggling about Out of context. how to minimize the percentage of out of context response?
Love you sir ❤
How well it works for Tabular data having columns from SQL DB. Should we use SQlAgent or Vector DB can work. We aren't getting good result from SQLAgent
Great content
Thank you for this
Sir please make more project videos with open source hugging face model and more focoused towards fine tuning
Please can you make a video on RAG implementation
Thank you for providing such a valuable videos, In your free time could you please make a video on Concept Normalization .
In prompt template there are two variables context and question. When you are using chain("do you have js course"). Is retriever the context? How will it know which is question and which is context?
Hi Dhaval, there are some functions that you show in the video like "response.text" that does not work. Are there a particular versions that you have used and are now outdated?
In the first part of project before asking the LLM model a question, you compare user query with csv, my question is it concept of RAG?
how does the system associate answers with prompts if you only load and embed the prompt column ?
This is a great learning. Trying out with the samples but does not seem to be working. Facing errors while compiling with Python 3.12. Any suggestions?
Very Nice video Dhavalbhai. I have question what are the privacy related risks connected to using this approach. Any data breach or anything like that ?
Thank you!
where the new question is going to add ? is there any performance matrices to check?
Why are we not splitting the document content in to chunks, in these examples ?
what's the difference between query and retrieve? because both are returning the same result
I have a question. Is it not necessary to clean the data after loading it?
We are expecting blockchain technology and TinyML course also sir
Error when i run program related to import
sir, is there any better way to create a conversational chatbot for a huge database using langchain. I am encountering issues with the payload limit.
Sir how can we integrate to existing website, instead of creating streamlit UI
Great content, thank you and appreciate your effort. Shall we take the same approach to fine tune a LLM to generate python code ?
if i have csv and excel file consist numerical and categorical now i want to get output in llm even for numerical also how to do it ? i need ur help to start this
why don't you save the chain like you did with vector db? and use that model. Im suffering from re running it, as in colab i had gpu, on my machine I don't, plus on colab there are libraries which make configuration easy. if you can please share a tutorial managing like that it'll be great
Hi, can you also cover end to end Autonomous AI agent creation and deployment.
There is plenty of material online but almost no one covers the topic comprehensively enough.
Thank u sir!! Plz make a 6 month roadmap for a fresher data engineer with free resources its a request..
Sir I am having an error while retrieving question's answer. The error says list index out of range.
Sir we are encountering ImportError: Dependencies for InstructorEmbedding not found. Please tell how to resolve it . Can we get a list of all depedencies so we do not have to install them one by one . I have already downloaded the files in requirment.txt
Maza a gya
Hello Dhaval, great tutorial as usual !
I have a, probably silly, question (sorry about that).
I saw that even after the 'Instruction' prompt has been specified to indicate to the model that if not relevant documents have been retrieved it must answer 'I don't know'... there are some retrieved source documents anyway, why is that? It is clear that model is not using them as a context for its answer, how can he knows that it must not use them? There is some similarity threshold behind?
Thanks ! :)
sir i am getting credentials error while running just the poem line itself !!!. can any one help me
were you able to get a solution for this?
sir, im facing lot of issues with packages used in this project. could you please help me out. its been 3 days since im facing this issue. i have tried in both vsocde and pycharm. please help me out
same here...please update if you have found a solution.
Have you solved?
@@kashifahmad9880 yes , make the project in a new virtual environment then install all the packages required , i have used google palm embeddings instead of hugging face instructor embeddedings and some other packages...
@@kashifahmad9880 yes , make the project in a new virtual environment then install all the packages required , i have used google palm embeddings instead of hugging face instructor embeddedings and some other packages...
I have tried to fix this code. It is giving error while running this code.
Why there is no textsplitters/chunks used here ?
18:46 the whole cell is not working😢
yes bro, same.. been stuck there from last 3 hours. If you have found the soln, please share
Same bro....please update if you found a solution.
Bro have you solved this??
getting error for line
embeddings = HuggingFaceInstructEmbeddings(
query_instruction="Represent the query for retrieval: "
)
ImportError: Dependencies for InstructorEmbedding not found.
any help??
i am also getting the same error .please let me know if aapka theek ho gaya hai yeh error toh
use a different embedding like
embeddings = HugginngFaceEmbeddings()
this could be seen on the langchain embeddings like he mentioned
You have to do pip install instructor and then pip install sentence-transformer==2.2.2
don't use the latest sentence-transformer
Raise NotImplementedError(
NotImplementedError: Need to determine which default deprecation schedule to use. within ?? minor releases....plsss help me to solve these...after 36.13
Hi did you resolved the error ? i am having the same issue.
@@gabbaSLwere you able to solve whole project?
@@ayush07310 yes
@@gabbaSLsorry to ask you again were you able to complete the project ? Since it's been 6 months I'm worrying whether I can start and finish this project without any outdated stuff
A doubt - How does the llm get to define the input variables i.e. context and question without being explicitly mentioned by user? We nowhere mentioned that the context should be the vectordb and question should be the user query. Any help would be appreciated!
PROMPT = PromptTemplate(
template=prompt_template, input_variables=["context", "question"]
)
same doubt
Sir please create this project as per new syntax of langchain, I am getting stuck while converting it into newer version.
Yes .... Waiting a lottt😍......
Sir in previous Rocky Bot...
When I am using the urls which was provided by you...The data from urls is not fetching...( The docs where Empty)
But when I tried with other urls i am able to run without any error... Could you please give me a solution why it is happening...
Please sirrrr
It can be that it has issues browsing those urls. You can try it again to see it works. Or we need to debug it further to check what the issue is. For me it was working ok without any problems.
@@codebasics
Yes sir.... There is issue while browsing the urls....
As you said I have tried multiple times.... More than 20 times... But still not working for the urls you provided...
Please help me to slove sir....
I wish I will get proper solution from you.... Soo that i can put this project in my resume.. Because by seeing your videos I have learned all the things related to Langchain+ llms... (Built small small applications to get practical knowledge) ...But not able show them as end to end project... I thought this rockybot application is very good application to showcase in my resume...
Please help me sirr
@@chinnibngrm272hello in this question answering project I am getting error when I try to load it data=loader.load() error: 'utf-8' codec can't decode byte 0x92 in position 1197: invalid start byte.How to fix it?
same problem.@@amuralikrishna22
@dhavalpatel,can we say this as rag chatbot which uses source url for context from vector database?
Yes
Plz bring more langchain projecta
Yes. One more project will be published next week. It will be in retail domain
Anyone doing this Project now or has done recently , I need some help actually , Firstly , llm ( ..Prompt..) and InstructEmbeedding these two syntax didnt work , so used HuugingFace embeddings instead , and secondly : I have succesfully completed until the Document chunk retreival step following this code (also created the chain) , But at the last step when With those chunks , it will interact with the LLM , I have tried many times but the Chain("...Question...) syntax is showing error , I have also tried with chain({'query':'question'}) , that also is not working , so stuck at the last stage , anyone may help?
Hi my project was doing well but from few day it is showing Retrying langchain.lIms.google_palm.generate_with_retry.. _generate_with_retry in 60.0 seconds as it raised NotFound: 404 Requested entity was not found.. getting this in logs when i am generating answer.
Using chain.invoke("")
@@kashifahmad9880 bro google palm is depricated so they say use gemni api
@@kashifahmad9880 same bro, did you get any solution
why can't we use streamlit in jupyterlab why we need pycharm code editor
Is vector database same as embedding matrux?
Hi, महान लोग, I am wondering how you evaluate your model performance
@Codebasics- Error Message - line 154, in __init__
raise ImportError("Dependencies for InstructorEmbedding not found.") from e
ImportError: Dependencies for InstructorEmbedding not found. . Can you pls help me to fix this issue?
yes I got the same error too how do we fix this
installed the required dependencies but still get it
i was getting this error too bro and i used gpt4all instead of instructembedding , its working fine for me
unfortunately gpt4all failed for me too@@ttvgolive3436
not sure if its becoz mine is m1 Mac or sth..
Sir how to integrate it to React.js ??
Sir can u make videos on how to predict future results with past data in basketball or even football I am trying to create a basketball prediction which predicts the result at any point of time of the match
Waiting to hear from you
how to get the OpenAI api key when the existing key has expired , any help or suggestions?
Simply generate a new key no!?
Dear sir how to download all model and embedding concept in my local server and how can run the all process without internet connection.
Hello... guys if any buddy have know so please tell me..
Is it possible to do it on colab?
When can I call myself a data scientist or a machine learning engineer. What all courses in your playlist will help me reach that tag name
in youtube search "codebasics data science roadmap" watch that video to know what all you need to learn
why vector FAISS db from_document for documents parameter is Docs instead of data from loader.load() in the notebook? I think it's typo but may be Im wrong
Up
sir, im facing lot of issues with packages used in this project. could you please help me out. its been 3 days since im facing this issue. i have tried in both vsocde and pycharm. please help me out
waiting...
is it related to data analysis???
No. This is related to data science, machine learning, Gen AI
@ codebasics sir I am facing issue while installing faiss cpu, can anyone please help
Same here have solve this error?
When I try to access to MakerSuite it says "Restricted Access". Why?
did you login using your google account? Anyone with a google account should be able to access this.
@@codebasics yes, still got restrictions
@@codebasics got why, in Italy the API is not available
I was facing the same problem in Finland. Tried with VPN and it works.
@@md.mahadehasan737 yes, in EU is not available
❤❤❤
Hi Dhaval, When we can expect AI+ML course in code basis
I am building math/statistics course right now once that is launched, next one is AI+ML. It will take few months though.
I wish to better understand the issue of context and question
prompt_template = """Given the following context and a question, generate an answer based on this context only.
In the answer try to provide as much text as possible from "response" section in the source document context without making much changes.
If the answer is not found in the context, kindly state "I don't know." Don't try to make up an answer.
CONTEXT: {context}
QUESTION: {question}"""
PROMPT = PromptTemplate(
template=prompt_template, input_variables=["context", "question"]
How can we integrate it to a working website??.....Has anyone tried it??
Hey folks 👋 , i have just created this project using updates dependencies and models. Since the video a bit old and there is countinous changes in the libraries we have to change the code a bit if any one want any help you can reply me on this
Hi can u help me
@@aryanthakur0009 yes tell me
Hi i getting error now my project is not working Retrying langchain.lIms.google_palm.generate_with_retry._generate_with_retry in 60.0 seconds as it raised NotFound: 404 Requested entity was not found.. getting this in logs when i am generating answer.
Hi! Do you have any repo that can share? Have questions on all the dependencies.
@@aryanthakur0009 yes tell me what's the issue
Best country to land a job as a data analyst ???????
Tell sir🤚
yes
Hi Sir , I stay in Bangalore and I had applied at AtliQ ....Sir can we meet?
Not Audi
Does any one tried this recently and ran without any errors?
I tired but Google api key error
Retrying langchain.llms.google_palm.generate_with_retry.._generate_with_retry in 60.0 seconds as it raised NotFound: 404 Requested entity was not found.. gettning this in logs when i am generating answer.
great work
Did you face any issue's while embedding? Please do reply
@@abhishekghosh9523 No ,i have not faced any issue ...suppose if you face any issue you can use other embedding techniques like hugging face "sentence-transformers / all-mpnet-base-v2" or "sentence-transformers / all-MiniLM-L6-v2"
other transformers to embedding
@@samasrinivasreddy961
Hey thanks for the reply! But already sorted the issue.
Presently facing some dependencies issues while trying to run it on vscode, with huggingfaceinstructorembedding.
Have you faced something similar?
@@abhishekghosh9523 hey have you finished the whole project ?
ImportError: Dependencies for InstructorEmbedding not found. This error shows even if I installed all libraries
same problem with me , please update if you have found a solution.
You have to use pip install instructor and then pip instan sentence-transformer==2.2.2 these are the dependency for instructor embeddings
Best country to land a job as a data analyst ???????
Tell sir🤚
Best country to land a job as a data analyst ???????
Tell sir🤚
Best country to land a job as a data analyst ???????
Tell sir🤚
Best country to land a job as a data analyst ???????
Tell sir🤚
Best country to land a job as a data analyst ???????
Tell sir🤚