create a AI teacher using swarm agents, like it should personal ai teacher for all subjects with proper accuracy and according to my knowledge modify the content my level of understanding format type
Very informative. Thanks for this. Throttled my network to read the tech bits on your website. Couldn’t find a Community menu, I think that will engage users. More users on website may lead to more conversion.
Great video! I’ve been looking for something like this. I’m currently working on the concept of vector embeddings and building a project around it, but I’ve been struggling to come up with a clear use case for myself or visualize a real-life application-until now. This really helped!
I have one query, for caching what are you essentially caching? Is it only the semantic query relevance if multiple users asks similar query? Would appreciate if you throw some light off
Amazing! I want to do a personal project based on nearly this concept but I want everything to be locally handled including the chatgpt answer generation part. Although it's not related to text but majorly images. Great video anyways.
I guess I didn't understand the problem well. Can't Gpt goes through all of your files (it has already too much data which it's trained on why can't go through some more) ?
Hi, so am I correct in understanding that after feeding ChatGPT all your transcript files, the queries with no context didn’t yield good response; then with context pointing to a specific file yielded better response; and with context to multiple related files it yielded even better response?
I have used ElasticSearch (OpenSearch) earlier: it's very expensive and not as good as a vector DB like PGVector. The problem with text search is that it relies too much on written words instead of the context between them. Maybe I could have used the OpenSearch better, but my experience with Neon (PGVector) is better.
Comment unrelated to the video: Gaurav one request: Coming from someone who have completed >50% of interview ready system design course. I loved your system design but deep knowledge comes beyond 1 hour of the System Design interview for a specific topic. Ex: - you teach us how to design uber in 1 hour for a system design interview and that works till we get a job. - now as a curious engineer I want to do deep dive into Uber's say hexagonal mapping thing. What enhancement I am requesting is when you design a system design topic please divide into 2 parts: 1. for interview(you're already doing) 2. beyond interview(for extreme deep dives like research papers, blogs etc. related to point 1) I know you have a dedicated research paper discussion section, I am yet to go there but it would be great if we can extend it to the system design topics which you have already taught, it'll help us getting a holistic view. Regards From your longtime student
Thank you for the suggestion, I will think about how to make this a reality :) Until then, you can use this page to filter through great deep diving resources: interviewready.io/resources/
When a question is asked, it needs to be sent to Neon DB, where similar context transcripts are retrieved from the database. These transcripts are then sent to OpenAI, and the response is returned and displayed to the user. I want to know if this process happens instantly, or if I am comprehending it incorrectly.
It would be good to give more context on when to implement a vector db. Your explanation is brilliant and I will learn from it. Asking question to OpenAI already has multiple implementations of vector db right? So how does it benefit a certain use case?
Great ecplanation and thanks a lot for this concept!!!I have one doubt. Will the data in neon db be static? How are you storing the data in the db for a particular transcript?
hey gaurav can we also store images in vector database? example can i store youtube thumbnails ? and while prompting gpt can use those as reference and provide me a better outcome love your content 🙌🏽
Thanks! You can store the files. The images can be converted to vectors, yes. But OpenAI is not suitable for this type of embedding, by my limited understanding. A multimodal system will do better.
Hey Gaurav, excellent explanation as always. I have a couple of questions I wanted to ask: 1. How do you create a vector database? Does Neon DB assist in this process? Do we simply pass the transcript to Neon DB, and it returns the vector file for that transcript? 2. Is this what our request to ChatGPT looks like? -> "What is load balancing?" and here are some vector databases for context: transcript-vector.txt. 3. So, does ChatGPT receive the query from the user and the vector database for the transcript provided by Neon DB? 4. I also don't understand how Neon DB selects which transcript vector to send to ChatGPT. Thank you for the video.
Thanks Raymax! 1. Creating a database a one click operation in Neon. No we have to first get an embedding for the transcript using a model. There is LlamaIndex, etc... for this. 2. User prompt: What is chat gpt. System Prompt: You are a system design teacher who answers in 2-3 sentences. Something like the above. 3. ChatGPT talks to our server, the InterviewReady server. Our server finds files most relevant to a query from NeonDB and uses them to add context to the query. 4. It's a vector embedding search. There are various search algorithms for this, like K-nearest and HSNW.
Nice video , i always treat you like peer having discussion in room , you are good techie . One question though, how will you protect the privacy or content of the file that you sent to chatgpt ? will they use that for training or is it very locale to you when you store it ( I assume it will be for your user account )
I think they'll end up using it for training. I have to read their TnC page to be sure. It is local to my account (I am paying for their API access, so it shouldn't be shared anywhere else). Thanks for the comment 😁
1. Is Neon db hosted in your vpc? 2. How do you maintain files/ embedding in vector db (say- For another RAG project where users wants to chat with their documents then how do we query vector db where multiple users have their own documents)? 3. Do you think running llm instance locally (eg. ollama) is better option!!
Small doubt ,At the beginning, you mentioned using ChatGPT to store transcripts in the vector database. I believe you meant to say that you used a text embedding model from OpenAI to generate and store vector embeddings.
No. He stores the transcripts. He uses the "Retrieval" part, i.e. getting similar video files using Neon and tells the GPT model via API to consider files with index [i,j,k ...] to be used for augmentation for answer generation. I intentionally broke down the RAG across the sentence so that you know it is not something fancy.
While working with AWS transcribtion for audio queries its very slow and time consuming is their any way to increase performance for this process specially using audio .mp3 file from s3 bucket
How do you check whether your system is not used other than asking questions related to your videos only as Amazon also got this wrong in its first try where it written python code for the user
So basically, You are storing raw files using openai files api, and storing vector embedding in neon vector db, when a user makes a prompt you fetch the similar vectors from the neondb and inject in prompt sent to openai text generation api. what if user deletes the file, we sure can delete it from openai files, but how do we delete the vector embedding of a particular file ?
@deesiInGermany i found that we have to attach metadata with the vector and when the file is deleted, we have to search embedding that have same name in metadata. I also wondered that if we cannot use uploaded files directly in the prompt like "refer file {file1}", than whats the point of storing it on openai server. We should store it on our server only for better flexibility in future.
@@dhruvwills thanks Dhruv for your reply. I am starting my AI journey with a project. Just confused here if we are storing our files in Vector DB and then will only share the part of the context with the question to gpt, then what's a point of storing files in ChatGpt. Also storing theses files will consume tokens means extra cost
@@deesiInGermany the primary objective of files in gpt is to finetune model, but if you do it then it will generalize for each user. Thats is why we are providing the content in context, so in this case storing files on openai is just acting as a database, this is not necessary, you can store files on your own server and generate embedding using openai api, then store that embeddings in neondb. Here is how it would work, You would use openai api to send the text which was stored on the file and openai will return embedding which you will store on neon vector db. Earlier there used to be answers and query api in openai but that is now deprecated. They suggest to use embedding now. So basically, attach embedding in context, and you are good to go.
Thank you for this fantastic video. Very clear and I like the diagrams. One completely unrelated piece of advice: incorporate exercises that target your shoulders and neck. Not for getting bigger, but to improve your posture and enhance your overall well-being.
people like me who do not have money or do not want to spend money on chatgpt ,can user ollama to run a LLM model on a local instance and run llama3, or mistral for RAG
So which is which? I mean what is the vector database here? Llama3? And ollama is the llm model to which we will give the augmented query with context that we will find from the llama3?
Thank you for watching!
Resource links:
Neon DB - fyi.neon.tech/gs1
AWS Transcribe: aws.amazon.com/pm/transcribe
create a AI teacher using swarm agents, like it should personal ai teacher for all subjects with proper accuracy and according to my knowledge modify the content my level of understanding format type
Please share your GitHub repo link, How to join you class?
Just what I was looking for!!
Building a project with similar specification. This video helped me understand the system design aspect of it.
Cheers!
One of the best videos I saw in recent times. Great details. Thank you and God bless you.
Thank you!
wa..o... superb!! keep your smart work like that, waiting for your next video.
Very nicely explained. Just right amount of information.
Thank you!
Very good conceptual video about a RAG pipeline. well done.
Thank you!
Extremely well explained ,
Loved it ❤
Thank you!
A very insightful video! Thanks Gaurav
Keep UP the GOOD WORK , your videos are very good and of good quality
Thank you!
Very informative. Thanks for this.
Throttled my network to read the tech bits on your website.
Couldn’t find a Community menu, I think that will engage users. More users on website may lead to more conversion.
Hahaha, that's a nice way to read the tech bits 😁
I am looking to add a community section in 2025. Thanks for the feedback!
Really insightful loved it. I heard lot of rag everywhere now I have a good idea on this thanks
Can we get a video of what is agents how to get started on that just a brief video like this would be really helpful
Great video! I’ve been looking for something like this. I’m currently working on the concept of vector embeddings and building a project around it, but I’ve been struggling to come up with a clear use case for myself or visualize a real-life application-until now. This really helped!
That's awesome, good luck with your project!
Very cool and concise video. Was not aware of this and thanks for sharing.
Glad it was helpful!
Great work Gaurav !
Thank you!
I had this challenge previously. Thank you os much for this.
Cheers!
I have one query, for caching what are you essentially caching? Is it only the semantic query relevance if multiple users asks similar query?
Would appreciate if you throw some light off
I am using digital ocean droplet to host Postgres,
can i use Neon?
how do you rate Neon??
I found Neon fast and good.
@ i want overall rating sir
Like
Can we trust for production ready app, which may cater lakh around user?
@@BloggerVikash Read the fine manual
@@PrabhakarKumar97 yes i have gone through
Currently avoiding neon
Because of the region.
Indian region is not available
Amazing! I want to do a personal project based on nearly this concept but I want everything to be locally handled including the chatgpt answer generation part. Although it's not related to text but majorly images. Great video anyways.
Go for it!
The generation is also possible if you have your own transformers.
Thanks, nicely explained.
Looking fwd to more such AI system designs.
Thanks, more are coming!
Sorry, May i kindly know what steps are involved in augmentation (a in rag)
Bro thankyou for this concept😊😊😊
I guess I didn't understand the problem well. Can't Gpt goes through all of your files (it has already too much data which it's trained on why can't go through some more) ?
Hi, so am I correct in understanding that after feeding ChatGPT all your transcript files, the queries with no context didn’t yield good response; then with context pointing to a specific file yielded better response; and with context to multiple related files it yielded even better response?
That's right 😁
Hi Gaurav, did you consider elasticsearch as well? I mean just for benchmarks
I have used ElasticSearch (OpenSearch) earlier: it's very expensive and not as good as a vector DB like PGVector.
The problem with text search is that it relies too much on written words instead of the context between them. Maybe I could have used the OpenSearch better, but my experience with Neon (PGVector) is better.
Great video, Gaurav. Also, from where do I get that cool tshirt?
Thanks!
I got it as a gift, so I am not sure where. I think it's from a creator shop in Instagram though.
Comment unrelated to the video:
Gaurav one request:
Coming from someone who have completed >50% of interview ready system design course.
I loved your system design but deep knowledge comes beyond 1 hour of the System Design interview for a specific topic.
Ex:
- you teach us how to design uber in 1 hour for a system design interview and that works till we get a job.
- now as a curious engineer I want to do deep dive into Uber's say hexagonal mapping thing.
What enhancement I am requesting is when you design a system design topic please divide into 2 parts:
1. for interview(you're already doing)
2. beyond interview(for extreme deep dives like research papers, blogs etc. related to point 1)
I know you have a dedicated research paper discussion section, I am yet to go there but it would be great if we can extend it to the system design topics which you have already taught, it'll help us getting a holistic view.
Regards
From your longtime student
Thank you for the suggestion, I will think about how to make this a reality :)
Until then, you can use this page to filter through great deep diving resources: interviewready.io/resources/
When a question is asked, it needs to be sent to Neon DB, where similar context transcripts are retrieved from the database. These transcripts are then sent to OpenAI, and the response is returned and displayed to the user. I want to know if this process happens instantly, or if I am comprehending it incorrectly.
It doesn't happen instantly, it takes about 6 seconds for OpenAI to respond.
We are working on showing a loader while this happens, so the UX is good.
It would be good to give more context on when to implement a vector db. Your explanation is brilliant and I will learn from it. Asking question to OpenAI already has multiple implementations of vector db right? So how does it benefit a certain use case?
Can we utilised not diamond ai tool which provides free apis for different ai models, so does it works the same?
Great video. Your humour "depending on your salary you can set up the instance you want ...xD"
😁
May I get your AI Teacher demo? How it actually looks like.
Did you use Langchain?
Great ecplanation and thanks a lot for this concept!!!I have one doubt. Will the data in neon db be static? How are you storing the data in the db for a particular transcript?
Thanks! Yes the data is static for a transcript. I can update the vector embeddings by clearing the data and replacing all the vectors again.
@@gkcs thanks a lot for the reply 😇
a muchhh needed video, thankss a lottt sirrr😭😭😭😭😭😭😭😭😭😭😭😭
Cheers!
Thank you Gaurav 😁
Thank you!
@gkcs would the outcome of this be the same as creating a custom GPT in OpenAI using your transcript files and calling that GPT via the API?
hey gaurav
can we also store images in vector database?
example can i store youtube thumbnails ? and while prompting gpt can use those as reference and provide me a better outcome
love your content 🙌🏽
Thanks!
You can store the files. The images can be converted to vectors, yes. But OpenAI is not suitable for this type of embedding, by my limited understanding.
A multimodal system will do better.
@@gkcs thanks 👍
I really really like your videos
Thank you!
helpful, thank you.
Thank you :D
bro, how to make embeddings in vector database, like use openai embeddings api for that or neon did that?
You can use openAI embeddings, yes.
You can also use other open source embedding algorithms. We used OpenAI because it's something we have heard of :p
Are you using any Guardrails?
Hey Gaurav, excellent explanation as always. I have a couple of questions I wanted to ask:
1. How do you create a vector database? Does Neon DB assist in this process? Do we simply pass the transcript to Neon DB, and it returns the vector file for that transcript?
2. Is this what our request to ChatGPT looks like? -> "What is load balancing?" and here are some vector databases for context: transcript-vector.txt.
3. So, does ChatGPT receive the query from the user and the vector database for the transcript provided by Neon DB?
4. I also don't understand how Neon DB selects which transcript vector to send to ChatGPT.
Thank you for the video.
Thanks Raymax!
1. Creating a database a one click operation in Neon. No we have to first get an embedding for the transcript using a model. There is LlamaIndex, etc... for this.
2. User prompt: What is chat gpt. System Prompt: You are a system design teacher who answers in 2-3 sentences.
Something like the above.
3. ChatGPT talks to our server, the InterviewReady server. Our server finds files most relevant to a query from NeonDB and uses them to add context to the query.
4. It's a vector embedding search. There are various search algorithms for this, like K-nearest and HSNW.
@@gkcs thankyou sir !! 🫡
Are you suggesting we pass all the matching files for getting queries answered? Would it not be very expensive?
Nice video , i always treat you like peer having discussion in room , you are good techie . One question though, how will you protect the privacy or content of the file that you sent to chatgpt ? will they use that for training or is it very locale to you when you store it ( I assume it will be for your user account )
I think they'll end up using it for training. I have to read their TnC page to be sure.
It is local to my account (I am paying for their API access, so it shouldn't be shared anywhere else).
Thanks for the comment 😁
@@Parul-u6e Thanks for letting me know :)
Thanks Gaurav
Cheers 😁
Thanks
awesome vid.. i am your fan..
Waao very useful
Ammmaazing boi... thanks
Cheers :D
would you be able to open source the entire solution? Like a git repo?
great application :)
can you please share the source code so that the extensions can be made out of it.
1. Is Neon db hosted in your vpc?
2. How do you maintain files/ embedding in vector db (say- For another RAG project where users wants to chat with their documents then how do we query vector db where multiple users have their own documents)?
3. Do you think running llm instance locally (eg. ollama) is better option!!
Small doubt ,At the beginning, you mentioned using ChatGPT to store transcripts in the vector database. I believe you meant to say that you used a text embedding model from OpenAI to generate and store vector embeddings.
That's right. We got the embeddings with OpenAI:
platform.openai.com/docs/api-reference/embeddings
No. He stores the transcripts. He uses the "Retrieval" part, i.e. getting similar video files using Neon and tells the GPT model via API to consider files with index [i,j,k ...] to be used for augmentation for answer generation. I intentionally broke down the RAG across the sentence so that you know it is not something fancy.
While working with AWS transcribtion for audio queries its very slow and time consuming is their any way to increase performance for this process specially using audio .mp3 file from s3 bucket
Adobe takes time too and so does Vimeo. I found AWS easier since it's an API and runs async.
Another way for transcripts is to use youtube as well. It generates transcripts whenever a file is uploaded.
That's a great idea, thank you!
How do you check whether your system is not used other than asking questions related to your videos only as Amazon also got this wrong in its first try where it written python code for the user
So basically,
You are storing raw files using openai files api,
and storing vector embedding in neon vector db,
when a user makes a prompt you fetch the similar vectors from the neondb and inject in prompt sent to openai text generation api.
what if user deletes the file, we sure can delete it from openai files, but how do we delete the vector embedding of a particular file ?
Too have the same doubt
@deesiInGermany i found that we have to attach metadata with the vector and when the file is deleted, we have to search embedding that have same name in metadata.
I also wondered that if we cannot use uploaded files directly in the prompt like "refer file {file1}", than whats the point of storing it on openai server. We should store it on our server only for better flexibility in future.
@@dhruvwills thanks Dhruv for your reply.
I am starting my AI journey with a project. Just confused here if we are storing our files in Vector DB and then will only share the part of the context with the question to gpt, then what's a point of storing files in ChatGpt. Also storing theses files will consume tokens means extra cost
@@deesiInGermany the primary objective of files in gpt is to finetune model, but if you do it then it will generalize for each user.
Thats is why we are providing the content in context, so in this case storing files on openai is just acting as a database, this is not necessary, you can store files on your own server and generate embedding using openai api, then store that embeddings in neondb.
Here is how it would work,
You would use openai api to send the text which was stored on the file and openai will return embedding which you will store on neon vector db.
Earlier there used to be answers and query api in openai but that is now deprecated. They suggest to use embedding now.
So basically, attach embedding in context, and you are good to go.
@dhruvwills perfect Dhruv. I'll give it a try. Thank you
Thank you for this fantastic video. Very clear and I like the diagrams.
One completely unrelated piece of advice: incorporate exercises that target your shoulders and neck. Not for getting bigger, but to improve your posture and enhance your overall well-being.
Thank you!
Agreed, I'll work on it :D
people like me who do not have money or do not want to spend money on chatgpt ,can user ollama to run a LLM model on a local instance and run llama3, or mistral for RAG
Yes that's right.
So which is which? I mean what is the vector database here? Llama3? And ollama is the llm model to which we will give the augmented query with context that we will find from the llama3?
Awesome 👍
Thank you!
Depending on your salary got me 🤣🤣🤣
I got this video after commenting on a post
4:18 uff the humour
:p
Hey can anyone tell me about approx cost of this project or a working prototype?
What does load balancing mean?
Brilliant video btw. Just get a hang on the stuff that am looking for 🤝
Hahaha!
🎉🎉 Amazing
Cheers!
❤ super
like marvel, the post credits 🤣🤣🤣
:D
pg vector = PostgreSQL ?
@@Hercules159 Yes it's a vector database extension with the Postgres DB.
@gkcs thx a lot
Woke up to this
Good morning!
This an use case and real implementation of an AI agent correct?
@gkcs
💖
Osm
Thank you!
hahaha ! that laugh for 100$ was epic
Bootstrapped budget constraints :P
PG-vector 💀
Why this '💀' though
Waooo
Thank you!
Awesome 👌
Thanks!