LangChain GEN AI Tutorial - 6 End-to-End Projects using OpenAI, Google Gemini Pro, LLAMA2
Вставка
- Опубліковано 13 жов 2024
- Learn how to us LangChain with GPT-4, Google Gemini Pro, and LLAMA2 by creating six end-to-end projects. The goal of LangChain is to link powerful LLMs, such as OpenAI's GPT-3.5 and GPT-4, to an array of external data sources to create and reap the benefits of natural language processing (NLP) applications.
💻 Github: github.com/kri...
✏️ Course developed by @krishnaik06
⭐️ Contents ⭐️
⌨️ (0:00:21) Introduction What We will Learn
⌨️ (0:01:36) Langchain Crash Course
⌨️ (1:18:26) Chat With PDF Using Langchain And Astradb
⌨️ (1:41:21) Blog Generation Using LLAMA 2 LLM Models
⌨️ (2:16:21) End To End LLm Projects Using Pinecone VectorDB
⌨️ (2:52:02) Google Gemini Pro Demo
⌨️ (3:20:11) Multi Language Invoice Extractor LLm Project
⌨️ (3:46:02) Conversational Q&A Chatbot Using Gemini Pro API
🎉 Thanks to our Champion and Sponsor supporters:
👾 davthecoder
👾 jedi-or-sith
👾 南宮千影
👾 Agustín Kussrow
👾 Nattira Maneerat
👾 Heather Wcislo
👾 Serhiy Kalinets
👾 Justin Hual
👾 Otis Morgan
👾 Oscar Rahnama
--
Learn to code for free and get a developer job: www.freecodeca...
Read hundreds of articles on programming: freecodecamp.o...
Guys, at 1:12:56 the model da-vinci-003 is deprecated which means it's removed and replaced with another model.
llm=OpenAI(openai_api_key=os.getenv("OPENAI_API_KEY"),model_name="gpt-3.5-turbo-instruct",temperature=0.5)
Make the above changes, so that you dont get an error.
Hi any idea what is the problem with huggingface? I get this error, did you encounter like this "ConnectionError: (ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)), '(Request ID: 92d58948-1e04-40d9-a054-e6e182be6e43)')"
Please can anyone help to use this api freely in start? As everytime I run code it says check your bills
@@islamicvoicesandhistory138 If you run LM Studio on your local machine, it produces an API similar to OpenAI. I will need to override the OpenAI point locally. You could also use Hugging Face.
Exactly… checked their model and changed to this 3.5 instruct
legends dropping legendry tutorials.Thankyou so much gonna watch now
So clean and clear explaination! 😍😍 I love the way he teaches nad makes the complex concepts so easy 😍
Unfortunately the Gemini Pro API is not available in Germany yet and I am too lazy to run via VPN😬, but the other projects were great and already really helpful! 👏👏👏
very good video which shows how important ai is becoming in programming
Gonna watch it tomorrow!! Can someone remind me at 11 am on 26/01/2024. Thank you.
get a calendar maybe?
@@1Iljo1 if you really want to watch it, you can set your alarm or make an event on your mobile...etc. No need to ask someone reminding you. I used to do like your stupid actions many years ago👎🏻👎🏻
😂
@@أحمدالعريقي-ض1خ bro went so hard that I decided to watch it right away !!
No
very useful video, which is very helpful for learning AI related technologies，please make more videos like this.
Thanks for this video :) Unfortunately langchain has already changed a lot of things - even 'predict' is deprecated in favor of 'invoke'
Do you know how to solve the problem?
The run at 1:40:40 results in "This is a chat model and not supported in the v1/completions endpoint" error. To solve this, use model="gpt-3.5-turbo-instruct" when defining the llm.
Love it. a pragmatic approach to learning how to use these libraries.
Nice video, kindly add a list of timestamps and titles.
The tutorial is very good, a few things is now different especially Pinecone related
thanks for the tip with the venv! I did another course on Udemy (which was also great otherwise) but there I had huge issue with breaking changes, dependencies etc.
Right when i needed it , Just blessed ! :)
PhysicsWala legend on another Legendary Channel
Thanks to the association #freecodecamp
Langchain looks like in demand.
This is gold!
Going to complete right now. Thanks
What is the total number of months from January 2014 to December 2019 when NBA is more popular (in terms of hits) than MLB? HOW TO DO THIS ON R STUDIO
Thanks for learning to us
Please make video on Performance Testing (jmeter)
Seems text-davinci-003 model used in Q&A Chatbot is deprecated! I tried with gpt-3.5-turbo-instruct. It worked.
hi boss - can you help me .. i followed the video step by step for the Q*A chatbot.. im unable to launch the streamlit app...its just showing a blank screen. no error. Initially streamlit webpage was giving an error - Import error: can import openai from langchain
running the command: conda create -p venv python -y (worked just fine)
then i try: conda activate venv/
CondaError: Run 'conda init' before 'conda activate'
on mac using brew
Thank you!
Wow Thanks for this tutorial
This is very helpful!! 😀
Can you plz make some content on api tokens management to optimize business price .
Thank you 😊 sir
thank you sir for this informative video
Langchain v0.1.0 is now slightly different that this version. Nonetheless, thanks for this nice tutorial. 👍
running the command: conda create -p venv python -y (worked just fine)
then i try: conda activate venv/
CondaError: Run 'conda init' before 'conda activate'
on mac using brew
@@disrael2101 like the error says, try running `conda init`, and after that it should work...
i tried, i posted it after i tried of course.. it seems to run fine but the same error persist.. looks like it's a brew mac issue with the envs of the ancaonda on the shell@@sprightly106
do you've any idea why he chose 1536 as the dimensions at 2:41:15 he didn't explain that@@sprightly106
Nice tutorial, but just like many others I've seen it sidesteps the use of Meta's raw .pth model in favor of Hugging face's format. I guess that's more comprehensive for the sake of educating but I had to search for a while to understand what switching to Hugging Face's models actually implied.
what does it imply?
Can you share the google colab for the pdfquery tutorial?
Thanks for this❤
Thank you 🎉🎉
After Watching the full video can I able to make my own Application like ChatGPT??
Krish sir guru ji🥰🙏
I think the huggingface llm is making so short and non creative answers because you made its temperature 0 rather than making it 0.6 like you did to openAi llm. Correct me if I'm wrong please.
24:24 reference
He is a great teacher, i m proud to be his student. @krishnaik
Thank you!!
Everybody is just teaching how to make custom assistants...But nobody is guiding how to deploy these on webpages...can someone guide me to implement custom assistant api into webpage?
Thank you
Thank you so much , very clear explanation
Are you all using the paid version for OpenAI part? Any workaround if you can't use paid API for openai?
Yes, i added credit of 5 dollar with tax 5.90 to the open ai account, without that the query was not getting answered.
running the command: conda create -p venv python -y (worked just fine)
then i try: conda activate venv/
CondaError: Run 'conda init' before 'conda activate'
on mac using brew@@mayankjain4379
I did a little bit of my own tinkering and found that you can use mixtrals model for free for quite a bit of the lessons, he explains using Hugging Face LLMs so I reverse engineered the steps.
@@MatthewM2002 can you tell me more? like what's the LLM name exactly - there seems to be more than one by the name of HuggingFace, also what model of the mixtral one do i have to use?
You gott it 2k like
Spring AI is the future or this?
Krish freecodecamp fusion! Nice
Beginning RAG example? 1:19:30
make sure you set up the paid api key, if you've used free, or you won't be able to use Open AI
OpenAI is not free anymore so what to use instead of it
👍👍👍👍👍👍
This is the best 4h resume of a manual you would otherwise read in 1h.
can we use hugging face hub in production? I think not, as license is not open source. please confirm
Guys do i need to get chat gpt plus to proceed with this?? I'm gonna start this on Saturday
No
What is the use of this ?
how can i use this to create a chatgpt specified for my website content q&a
Same Question any one please Ans??
at 2:44:12 why did you use ''doc" that is the entire data intead of the data in chunks.
openai has stopped giving api for free, please suggest some alternative
Reding attentively
This is great material. But please stop saying "probably" so much.
probably stop saying "probably" so much
we need paid api key of open ai 3.5?
The SequentialChain part (00:47:00) is not working for some reason, it tells me I'm missing required input keys, any idea? I've verified that the code is the same..
works for me, please share your code.
@@dietmarbaur1185 Hi, thanks for the rsponse, It was because the versions were different, as it was deprecetad or something, One way or another, the rest of the stuff seems to be working great, so not relevant anymore :D
running the command: conda create -p venv python -y (worked just fine)
then i try: conda activate venv/
CondaError: Run 'conda init' before 'conda activate'
on mac using brew
running the command: conda create -p venv python -y (worked just fine)
then i try: conda activate venv/
CondaError: Run 'conda init' before 'conda activate'
on mac using brew@@dietmarbaur1185
I am unable to create the environment. When type in conda create -p venv python==3.9 -y I get the following message: "'conda' is not recognized as an internal or external command,
operable program or batch file.". Can anybody help? Do I need to download anything prior to theses steps?
hope u are able to solve it
u can try py -m venv 'environment name', if conda is not installed
is the openai key only for paid accounts, coz created the key but I couldn't use it
code is not working for my Personal "OPEN_API_KEY". please help me
I get error because i do not have enough api_keys left for the month
For the Pinecone project I'm receiving PineconeApiAttributeError: QueryResponse has no attribute '0' at ['['received_data']'] error. Error seems to be associated with chain.run(). Can someone suggest how to fix?
Any solution for the RateLimitError? While doing llm.predict(text), it is not giving output and showing RateLimitError, quota exceeded. Do I have to have the paid version for this?
That's because OpenAI has stopped giving its API for free. If you want you can use a paid plan or simply can use a different llm like the one used here by HuggingFace but it says the capital of India to be Calcutta and Chennai lol
Hi, pinecone has migrate to a new version now "init" is no longer a top-level attribute of the pinecone package. I've made lot of reasearch but i don't find any solution to make it working again. Did you have an idea ?
hey man, I had the same problem and I fixed it like this: import openai
import langchain
import os
from langchain_community.document_loaders import PyPDFDirectoryLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain_community.vectorstores import Pinecone as langpinecone
from langchain.llms import OpenAI
from pinecone import Pinecone, ServerlessSpec
from dotenv import load_dotenv
load_dotenv()
# read doc
def read_doc(directory):
file_loader = PyPDFDirectoryLoader(directory)
documents = file_loader.load()
return documents
doc = read_doc('data/')
def chunk_data(docs, chunk_size=800, chunk_overlap=50):
text_splitter= RecursiveCharacterTextSplitter(chunk_size=chunk_size, chunk_overlap=chunk_overlap)
doc = text_splitter.split_documents(docs)
return doc
documents=chunk_data(docs=doc)
len(documents)
## Embedding technique of OpenAI
embeddings = OpenAIEmbeddings(api_key=os.environ['OPENAI_API_KEY'])
embeddings
vectors=embeddings.embed_query("How are you")
len(vectors)
# Create a Pinecone instance
pc = Pinecone(
api_key=os.environ.get('PINECONE_API_KEY'),
environment="gcp-starter"
)
index_name = "langchainvector"
# Now do stuff with 'pc'
# For example, to check if an index exists and create one if it doesn't:
if index_name not in pc.list_indexes().names():
pc.create_index(
name=index_name,
dimension=1536, # Update the dimension as per your requirement
metric='cosine',
spec=ServerlessSpec(
cloud='gcp', # Update this to your cloud provider if different
region='us-west-2' # Update this to your specific region
)
)
index = langpinecone.from_documents(doc, embeddings, index_name=index_name)
This should work and you should see the pinecone database filled with your document vectors!, also with the new version the problem is both pinecone and langchain_community.vectorstores have an object as Pinecone and this leads to conflict when both the libraries are called and used one after the other!, so my work around for one of the library was to call like this "from langchain_community.vectorstores import Pinecone as langpinecone" and now it works. Let me know if it works for you.
@@bafna007 oh thanks a lot I'll look that tonight !
@@bafna007 Thanks, that worked like a charm, having spent looking for error resolution after the version change.
@@bafna007 hey man ,
index = langpinecone.from_documents(doc, embeddings, index_name=index_name)
is still throwing errors , i think the from_documents is also removed now
Does this require a paid subscription to open AI?
you get 5$ worth credit free for your OPENAI API. I think that is sufficient for this tutorial.
i got RateLimitError 429 quota exceeded all the time that i try to run the simple question of "Whats the capital of India" just like in the video, any help?
the api key i use is brand new and its configured to have all permissions.
is everything in this tutorial deprecated? i cant even move post 20mins here.
Indian ice cube
😂😂😂😂
did anyone encounter error for gemini-pro project, at this statement to_markdown(response.text) ?
rate limit error why it is showing my activity is zero
is it outdated? I see different code on the docs?
Guys is open ai key free for everyone?
Probably probably probably x 100000000000000
Conda function not working
First skill required to join iNeuron as a trainer: How bad you are typing and incorrect spellings. 😂😂
I saw krish once upon time
Adjusting for loop indentation for getting output in loop even though there is no error in the loop... He executed thrice 😅😅😅
sidney
Don't use "probably" for everything 😁
The term 'conda' is not recognized as the name of a cmdlet, function,
script file, or operable program. Check the spelling of the name, or if a path
was included, verify that the path is correct and try again.
Get this error when I am running below command in VS Code terminal
conda create -p venv python==3.9 -y
Does it mean I need to install conda first.
It's either that or you need to add conda to path. Check your installation by searching for anaconda prompt. If you have conda installed and still facing the error, add conda to path (Windows)
Do it by this method:
1) python -m venv my_venv
2) cd my_venv/Scripts/activate
Now the virtual environment will get activated.
Note: please run these commands in the command prompt not in the shell etc
And yes you are right, you have to first install Conda for opening it's command prompt
This problem got resolved as below
1. There was path missing in system variables which I added ( I already had conda installed but at different location).
2. conda --no-plugins create -p venv python==3.9 -y
The above line of code helped me solv the issue, by not downloading any plugin. so finally all the files got installed.
@@mayankjain4379 okay that's great 👍
first,
Why python? I'm so fed up with all this python stuff.
It sucks. It's like the most noob friendly programming language made with absolutely no regard to optimization or speed of execution in mind. If you are just starting to learn programming it might be good, but why are you using py to showcase a project like this?
Cause this is for the ones starting out. You could create an advanced, focused on efficiency tutorial with c++ implementation or maybe assembly language.
Broken english
Whose ? Yours ! Are you even literate
Boring youtuber!
Sorry can’t watch this that accent annoying
you'll never learn with this attitude 😆
you're not gonna be missed so don't worry
No problem, you wont die if you don't watch or learn.
No one's gonna miss you😂 get outta here
Hello, do you know what should be used instead of Pinecone.from_document? It is no longer an attribute of the package. "index=Pinecone.from_documents(documents, embeddings, index_name=index_name)"
i have this same problem, i need help.
@@Fredricktbb did you get any solution?
@@kumarshivam1379
"from langchain_pinecone import Pinecone". The code will work when you input this.
4 months old programs - langchain has deprecated several classes, methods
getting this error while running the code to Initialize pinecone.init()? Can someone please help.
Went through pinecone documentation still unable to solve, only got to know that init method as been deprecated.
AttributeError Traceback (most recent call last)
Cell In[69], line 1
----> 1 pinecone.init(
2 api_key="removed the api key for security",
3 environment="gcp-starter"
4 )
5 index_name="langchainvector"
File c:\Users\DELL\anaconda3\Lib\site-packages\pinecone\deprecation_warnings.py:38, in init(*args, **kwargs)
11 example = """
12 import os
13 from pinecone import Pinecone, ServerlessSpec
(...)
29 )
30 """
31 msg = f"""init is no longer a top-level attribute of the pinecone package.
32
33 Please create an instance of the Pinecone class instead.
(...)
36 {example}
37 """
---> 38 raise AttributeError(msg)
AttributeError: init is no longer a top-level attribute of the pinecone package.
...
region='us-west-2'
)
)
May this pdf will help
try this ! pip install langchain-pinecone
from langchain_community.vectorstores import Pinecone as PineconeStore
import os
os.environ['PINECONE_API_KEY'] = pinecone_api_key
index_name = 'vectordb'
vector_database = PineconeStore.from_documents(index_name=index_name,documents=doc,embedding=embeddings) it worked for me
There have been some changes in langchain and pinecone packages.
What works for me right now:
-pip install langchain_pinecone
-from langchain_pinecone import PineconeVectorStore
-store your API key as environment variable PINECONE_API_KEY
- to store your documents to your index and create embeddings do the following:
index_name = "langchainvector"
# this is the new way to upload
vectorstore = PineconeVectorStore.from_documents(
documents=documents,
index_name=index_name,
embedding=embeddings
)
I was checking through the langchain API documentation for pinecone and always followed to get to the latest version...
@@AlexMelemenidis Thanks Alex will try and update here.
In a world of data and code,
Where technology reigns supreme,
There's a new kind of learning mode,
That's taking over the scene.
It's called machine learning,
A powerful tool indeed,
With algorithms and data mining,
It fulfills our every need.
From predicting stock prices,
To recognizing faces,
It can make our lives so precise,
And take us to new places.
With every byte of information,
It grows smarter every day,
It adapts to any situation,
And finds patterns in every way.
But behind its complex workings,
Lies a simple truth,
It's only as good as its teachings,
...
So let us embrace machine learning,
And all the wonders it can bring,
But always remember its true yearning,
To make our world a better thing.
Poem generated by llm.predict("Write a poem on Machine learning")
Everything is good except the Indian accent
Xenova/text-davinci-003 -model depricated