I am blown away at how much information is densely packed into this. You got yourself a new subscriber, sir. It’s staggering to think about how these technologies will shape the landscape for data and analytics. This is just the beginning.
The clarity of this video while maintaining detailed granularity of the subject is very impressive and very appreciated. Thank you for making this video.
Damn, I spent like a week researching this shit on my own, and have been working on almost exactly the same thing. Processing MDX files into embeddings etc. It’s really cool to see somebody doing almost the same exact thing. Makes me think I am really on the right track!
@@automioai In this project no PDF files were used - all documentation had been written directly in MDX. You'll have to do some research on ways to extract text from PDF files. Once you have that, I wouldn't bother with MDX at all - just generate embeddings directly on that content.
Prediction: This is gonna get a million views. Just saw fireship video about vector databases and wanted to understand embeddings. Before I could even search, this video was in the page. Though I wasn’t interested in a 40 min video (had a feeling I’ll just stop after 5 mins like I usually do) I ended up watching it all. The rabbit hole 🐇 🕳️ format is so naturally elegant. Clear end to end use case. I secretly don’t want to share it with anyone but I am forced to fulfill my prediction.
Found your channel while learning React Three Fiber, subbed with notifications immediately. Today I get a notification for a well-explained ChatGPT tutorial, right as I embark on building a similar thing. Fantastic continued work, thank you very much!
Ive been looking for this information for months. Such an excellent tutorial and I love that Supabase's code is all open source so i can actually clone it and read how it works in detail later. Thank you so much for the walk through. Super talented dude too - love the blender stuff.
@@RabbitHoleSyndrome why is the generate embeddings file so different in the video then what is in the repo now? I can't find anything you talk about in minutes 10-13.
Hey @@fraternitas5117! Supabase moves pretty quick - the code I references has been refactored now to support multiple knowledge sources (ie. more than just markdown). You can find the markdown specific code here: github.com/supabase/supabase/blob/1b2361c099c2573afa1fe59d3187343bb8f1bcab/apps/docs/scripts/search/sources/markdown.ts
Wow. I'm so thrilled to know that you were one of the ones behind that great feature. I've been using Supabase for 6 months, and have been pretty happy with it. Except for the docs and the transition to 2.0. I was blown away when I saw that it generated the code for me when I started writing its documentation
@@RabbitHoleSyndrome So far so good! I think one challenge is to know how we can check if a user's email address exists. (Or other specific user's metadata) I couldn't find it in the docs. There was a GitHub issue which said to store the user's data in a separate table as the auth table was private. I ended up doing that and haven't had any problems. Btw, thanks again for all the awesomeness!
What a fantastic video and content. I’ve gone through multiple videos trying to better understand embedding and how to work with ChatGPT in the best way for querying large amount of content and producing an analyzed response. I’m not a developer, have a background in computer science but I’m a software sales person that is curious about technology and I was able to completely understand your video and content. Subscribed, liked and will be watching more of your videos. Thank you!
The most amazing think is that I basically made a chatbot app in less than a week with only the help of GPT4, I had no knowledge of AWS services, PostgreSQL or python. Everthing you told in the video is what GPT4 told me. All of the serves and database are setup, it has memory, STT, TTS and Cognito login/register.
Fireship guy? Either way, its been pretty useful in terms of learning API's and how to connect them to my nocode builder. Spent hours trying to get things working and the Assistant basically told me what i was doing wrong and how to fix it. So well done with the implementation.
Insane, I love how you are able to do so many things. My laptop atm is unable to power every wish of mine (getting into 3D) but I hope I will soon be able to do so.
This is a glorious illustration. Thank you very much! I've been trying to find an example of doing this, and yours has put it all together for me! Subscribed!
Dude, that's exactly what I was looking for. No more bullshit articles with clickbait titles, just DIY in the essence. Is there a way to support you through patreon or smth. ?
28:37 Whenever I see examples of decoder (GPT) prompts starting with "You are a helpful finance advisor" or "You are an enthusiastic support rep", I can almost see the AI clearing its throat and sitting up straight and saying "right, ok". Gimme that can-do attitude, GPT!
One thing that might help is if the question result shows the links to the documents that it acquired the information from. Since you are currently fetching which document to run chatgpt on based on similarity of features, maybe you can change the prompt so that it also returns the link of the document that was deemed as a "similar document"
First time here. This is so well done. Subscribed. Your viewer number will explode! I like how you approached the topic in a very calm way without jumping on the "LLMs will take over the world" train :) You don't happen to have the clippy blender asset somewhere?
This is a fantastic video! Thank you very much for sharing :D Quick question - Currently if the info is not in the documentation it responds "Sorry I don't know how to help with that". But how can we make it respond like this: "Sorry I don't have relevant info in the documentation but you can do something like this". For e.g. "I don't have any info about how to make banana pancakes in the documentation, but here is how you can make one...." Idea here is to make it act like chatgpt on top of the information provided. Keen to know more on this and thank you so much for making this video :D
I have not seen such a great video in a while! How wonderful have you explained the whole process 👍👍! Could you explain a bit more about how did you choose 0.78 as threshold for embeddings comparision? have you statisticized that wether the most relevant sections can be found with it?
Glad you liked it! 0.78 was a first-stab threshold that worked best based on a limited sample of test queries. I wouldn’t claim that this number is universal - almost certainly this could change by domain.
Great content. I wonder if it can generate code as well. Lets say you are doing this for cloudflare products, kv, workers, durable objects etc all documents injected. Then give prompt like "generate a worker for x" and it will specifically generate with given docs etc..
@@RabbitHoleSyndrome Thanks you for making this content for us for free! I very much appreciate it and you motivates me to share knowledge with my friends :)
I've got an internal api that returns train times for given stations. Could I use user prompts to ask the GPT LLM to find train times for a particular station and then fetch using an api get request an array of trains that are set to depart and then get GPT to format that back to the user in a tailored natural language type of way?
Definitely possible, will just take some work on the prompt engineering side. Your best bet today is probably LangChain - I recommend you check out their documentation on creating custom tools: python.langchain.com/en/latest/modules/agents/tools.html
are there alternatives to openai to create these vectors? Dont really feel comfortable building something around a closed source api that is controlled by one vendor.
Really great question. You’ll want to look into sentence embeddings. There has been a lot of work on the OSS side with Sentence-BERT (SBERT) you can check out. You might also want to look into Universal Sentence Encoder (USE) and InferSent.
LlamaIndex actually uses OpenAI (text-embedding-ada-002) by default for embeddings today. They're more of a toolkit layer to assist with the workflow. There are many other alternatives though (which LlamaIndex supports via LangChain) that are worth checking out: langchain.readthedocs.io/en/latest/reference/modules/embeddings.html
Loved it, came for the embeddings and left with that and whole lot more! Too bad that the clippy did not made to the web on supabase :). Copyright problems?
Thanks for watching! Clippy did make it - but things move fast at Supabase 😅. This feature has evolved into a unified cmd+k menu and just renamed as “Supabase AI”.
Amazing video! This is exactly what I was looking for a long time. You basically explains everything I wanted to know about how to create a search engine using open ai. But I have a few questions: How much did you spend on open ai embending API building this? How much supabase spends monthly with searchs using the open ai api? It is possible to use an open source embedding API instead of calling the open ai api ? Wouldn't it be less expensive than the approach you took?
Glad it was useful! As for costs, you may be surprised how inexpensive OpenAI embeddings are (at least I was). To put it in perspective, for the Supabase guides we currently have around 1500 page sections which total just over 220000 tokens. At OpenAI's current embedding price ($0.0004/1k tokens), that brought us to just less than $0.10 for the entire guide knowledge base (~one-time pre-processing). After that the average query is likely
So if I understood correctly: The embeddings were only used to check for similarity between the user's input and the doc's content, in order to provide the prompt with relevant (text) context, right? Is there a way to provide the GPT model with the embeddings instead?
That’s correct. You could have used an alternate search method, but embeddings have a nice alignment with LLMs since they also use language model themselves. Unfortunately no there is not currently a way to inject embeddings directly into GPT today. Maybe this will change in the future or become available in open source models like LLaMa in the same way we’ve seen it happen with Stable Diffusion.
Great video! How closely do the .mdx files have to match this structure before they can be processed into embeddings? Do they need to export the meta const, for example?
Hey great content! I was looking for exactly this. Do you have a discord for asking queries? I was making the same thing but for ".md" files not mdx and faced some issues.
Hey - CSV files are definitely doable. It will mostly come down to how you plan to pre-process them. Perhaps you can import your CSV file into a table itself and generate embeddings on the content within it.
This is amazing. Super clear and short explanation of embeddings. Great walkthrough the code touching on the relevant parts. Subbed the channel👍 I see the code is still in the Supabase repo - but the Clippy is not there? what happened?
Glad it was helpful! Edit: Just realized you might have been talking about the Clippy graphic on the site, not the code. Search and Clippy have been combined into the same interface - you can find Clippy by clicking search, then switch to “Ask Clippy”. Original: Things move quick at Supabase 😆 The Clippy frontend code got moved when search was upgraded to also use embeddings. After the refactor everything is just under “Search”: github.com/supabase/supabase/blob/0ecc238ad6d81202bb2301f7919b166a98929697/apps/docs/components/Search/SearchModal.tsx The backend Clippy logic is still in the same edge function: github.com/supabase/supabase/blob/0ecc238ad6d81202bb2301f7919b166a98929697/supabase/functions/clippy-search/index.ts
@@RabbitHoleSyndrome cool. You can feedback to Supabase, that a) this video significantly increased my interest in adoption of Supabase b) clipy icon is tooo small , it should be large and obstructive. Serious question Prompt Engineering: With Chat Completion API - do you do system persona and pack all prompt into user? Or would you break prompt like here into example 3 user entries? E.g.: Context, question, use markdown. Does it make a difference?
Good feedback 🙌 and great question about prompt engineering in the chat API. We are actually experimenting with this right now and trying to understand what produces the best results. At the moment we do a bit of both (system message and user message with a bit of prompt overlap between both). OpenAI says that the model doesn’t pay strong attention to the system message, so it may be better to use system message strictly to set identity and provide instructions & context in a user message.
16:10 would love an explanation on how embedding work with a document structure? I.e query is “summarize chapter 3”. The embedding sans retrieval don’t seem to capture the structure of the chunks that are contained in title chunk “chapter 3 “. All explanations on embedding I’ve seen all rely on the text content within a chunk.
Great explanation! I was wondering how to perform similarity searches in my files, there you have it! I'm also wondering if there would be a way (or need??) to speed up the similarity check step? For this application it might be fast enough, but imagine having a billion or trillion entries which you need to check you query against.. That would be a nightmare. Something like clustering ,say, N similar entries and in the first step only checking your query against their average value. If they are similar enough (above a certain threshold) you could then proceed to check against each of them and you have made only 1 extra check. If they're not at all similar, then you don't have to check against each and you have saved yourself N-1 unnecessary checks!! What do you guys think?
You may find this post helpful: mattboegner.com/knowledge-retrieval-architecture-for-llms/ The community is starting to settle with "retrieval-augmented generation" as the terminology for this now.
amazing! how does clippy update the vector db and process the text as embedding when NEW documentation is added? is it automatic? maybe i missed it in the video
Great question! The `generate-embeddings` script was designed to be diff-based. So next time you run it, it will pull in only the documents that have changed and re-create embeddings on just those. It currently works using checksums: 1. Generate a checksum for the content and store in the DB 2. Next time the script runs, compare the checksums. If they don't match, the content has changed and embeddings should be re-generated. The script runs on CI, so anytime documents change a GitHub Action will trigger the script. See this PR for details: github.com/supabase/supabase/pull/13936
Is there a version of this that can apply to any type of content? Not just specifically made for Supabase documentation, but documentation of any type, or insert a book, of textbook, and the search outputs relevant gpt responses?
Hey! I don’t have a pre-built suggestion off the top of my head, but solutions to this seem to be popping up everywhere (just hang out on Twitter for a few hours 😅). If you don’t mind coding, it should be relatively straightforward to swap out the MDX docs with really any kind of content source, and the remaining steps should be identical.
Yo, thank you for putting out your knowledge :). But I have question regarding the data search for the context. So basically your creating a graphdatabse with the gpt models and then you input the users question into the database to find the relevant articles/information?
I am blown away at how much information is densely packed into this. You got yourself a new subscriber, sir.
It’s staggering to think about how these technologies will shape the landscape for data and analytics. This is just the beginning.
Thanks for the sub! I am continuously blown away by the possibilities of large language models 😀
The clarity of this video while maintaining detailed granularity of the subject is very impressive and very appreciated. Thank you for making this video.
Damn, I spent like a week researching this shit on my own, and have been working on almost exactly the same thing. Processing MDX files into embeddings etc. It’s really cool to see somebody doing almost the same exact thing. Makes me think I am really on the right track!
Nice! Glad to help give some validation 😄 are you also building for docs?
Hey! how had you turn your pdf files into a propper mdx format ? tnx
@@automioai In this project no PDF files were used - all documentation had been written directly in MDX. You'll have to do some research on ways to extract text from PDF files. Once you have that, I wouldn't bother with MDX at all - just generate embeddings directly on that content.
Thank you for sharing . I was struggling on how to get started . This was very well presented . From this side of the world asante sana !
Glad it was helpful 😃
This was so valuable to just settle the thoughts into a clear action plan as to how to implement in production. Thank you!
Glad it helped!
Wow, this is incredibile... I can see a future where every docs site does the same thing. This is truly powerful stuff! Well done 👏
I’m am both excited and blown away by the possibilities. Thanks for watching!
Prediction: This is gonna get a million views. Just saw fireship video about vector databases and wanted to understand embeddings. Before I could even search, this video was in the page. Though I wasn’t interested in a 40 min video (had a feeling I’ll just stop after 5 mins like I usually do) I ended up watching it all. The rabbit hole 🐇 🕳️ format is so naturally elegant. Clear end to end use case. I secretly don’t want to share it with anyone but I am forced to fulfill my prediction.
Thanks for the comment! Glad to hear the format is working 😃
wrong!!
Found your channel while learning React Three Fiber, subbed with notifications immediately. Today I get a notification for a well-explained ChatGPT tutorial, right as I embark on building a similar thing. Fantastic continued work, thank you very much!
Awesome! Thank you!
So you're the one who made this beautiful thing. Pretty nice.
Seems to have found one great coding channel among the noise of today; back to good old days when coding was boring & nerdy. Great job!
Ive been looking for this information for months. Such an excellent tutorial and I love that Supabase's code is all open source so i can actually clone it and read how it works in detail later. Thank you so much for the walk through. Super talented dude too - love the blender stuff.
Glad to hear it helped! Agreed - open source is amazing! Let me know if you hit any road blocks along the way, happy to help 😃
@@RabbitHoleSyndrome why is the generate embeddings file so different in the video then what is in the repo now? I can't find anything you talk about in minutes 10-13.
Hey @@fraternitas5117! Supabase moves pretty quick - the code I references has been refactored now to support multiple knowledge sources (ie. more than just markdown). You can find the markdown specific code here:
github.com/supabase/supabase/blob/1b2361c099c2573afa1fe59d3187343bb8f1bcab/apps/docs/scripts/search/sources/markdown.ts
This 40-minute video looks like 10 minutes. I have been researching related engineering topics recently and they have been very inspiring to me.
The potentials with LLMs seem to be endless 🤯
Wow. I'm so thrilled to know that you were one of the ones behind that great feature.
I've been using Supabase for 6 months, and have been pretty happy with it.
Except for the docs and the transition to 2.0.
I was blown away when I saw that it generated the code for me when I started writing its documentation
Glad to hear it has helped you! Any feedback on the docs that you think should improve?
@@RabbitHoleSyndrome So far so good!
I think one challenge is to know how we can check if a user's email address exists. (Or other specific user's metadata)
I couldn't find it in the docs.
There was a GitHub issue which said to store the user's data in a separate table as the auth table was private.
I ended up doing that and haven't had any problems.
Btw, thanks again for all the awesomeness!
This content is really top notch. I appreciate the clear and detailed explanations of everything!
Glad you found it helpful!
What a fantastic video and content. I’ve gone through multiple videos trying to better understand embedding and how to work with ChatGPT in the best way for querying large amount of content and producing an analyzed response. I’m not a developer, have a background in computer science but I’m a software sales person that is curious about technology and I was able to completely understand your video and content. Subscribed, liked and will be watching more of your videos. Thank you!
Glad it was helpful!
I am grateful to this video. This open my eyes to Embeddings
Glad it helped 😃
First time on this channel. The way you structured the video, the pace and the explanations are all on point. Keep up the good work. +1 subscriber.
Glad to hear that, thanks for the sub!
This channel is awesome! Love the rabbit holes you take us one! keep them coming please!
The most amazing think is that I basically made a chatbot app in less than a week with only the help of GPT4, I had no knowledge of AWS services, PostgreSQL or python. Everthing you told in the video is what GPT4 told me.
All of the serves and database are setup, it has memory, STT, TTS and Cognito login/register.
It is quite amazing - and I’m sure it will only get better!
I'm doing something even cooler with the vector embeds I cant wait till I can share!
whaaaattt tell us??
Fireship guy?
Either way, its been pretty useful in terms of learning API's and how to connect them to my nocode builder. Spent hours trying to get things working and the Assistant basically told me what i was doing wrong and how to fix it. So well done with the implementation.
Glad it helped!
fantastic content, didn't expect it will be so informative in just 40 mins. Looking forwards to the next one!
Thanks for watching 😀
Insane, I love how you are able to do so many things. My laptop atm is unable to power every wish of mine (getting into 3D) but I hope I will soon be able to do so.
You are awesome! Now I can understand how these things work together.
Happy to hear it! 😀 thanks for supporting the channel!
excellent . you are hands-on & practical
This really is/was an epic video clearly and well laid out!
Amazed by your content. Fantastic work here .
Thanks, glad it helps!
Wonderful primer on prompt engineering.
You do a such great job bro ! I love what youhave built and your video, keep build great things bro.
incredible end to end tutorial, nicely done
Thank you! 😃 Have you explored embeddings or pgvector?
This is amazing, you’ve created Clippy just as enthusiastic and helpful you are!
Thanks a lot
Thanks for watching 😄
@@RabbitHoleSyndromehow did you generate the mdx files?
The MDX files weren’t generated - the Supabase team wrote them as you would any markdown file.
Absolutely amazing quality here, glad I found your video. Subbed!
Thank you!
Thanks Greg for a great explanation. I like your presentational style!
Glad it was helpful!
This is a glorious illustration. Thank you very much! I've been trying to find an example of doing this, and yours has put it all together for me! Subscribed!
Glad it helped, thanks for the sub!
Dude, that's exactly what I was looking for. No more bullshit articles with clickbait titles, just DIY in the essence. Is there a way to support you through patreon or smth. ?
Glad to hear it! You support by watching 🙌
best tutorial i have ever watched u are a genius ty!
Amazing video! I'm going to put this in practice right now!
Awesome! Feel free to share as you make progress!
Many thanks for this insightful and helpful video!
Happy to help, thanks for watching!
incredible value in this video
This video was fantastic, lots of information given in an easily digestible way. Subscribing!
Glad it was helpful, thanks for the sub!
GREAT content. It explained everything you need to know about creating 'chat docs' or similar in one run, and all open source. Kudos! And subscribed.
Thanks for the sub! Great to hear 😃
Cool video. I was thinking about doing something similar for some reference pdfs. Thanks for the video
Thanks for watching & best of luck!
I'm reading the comments, and I'm like.. Yeah WTF 🗿🔥
This is truly valuable and useful content, thanks a lot.
You bet! Glad it was helpful
Just what I was looking for - thank you very much🥳
Great 😃 Thanks for watching!
Cool project, would be cool to create a tool like this that you can embed in any documentation, reading directly the markdown or scraping the website.
Definitely!
I'm at 1:42 and this video already is 10/10
that's great, you did great job! helps me a lot in this.
Glad it helped!
Amazing video! Great and detailed information!
Glad it helped!
This is really helpful and valuable, thanks a ton!!!
You bet, thanks for watching!
28:37 Whenever I see examples of decoder (GPT) prompts starting with "You are a helpful finance advisor" or "You are an enthusiastic support rep", I can almost see the AI clearing its throat and sitting up straight and saying "right, ok". Gimme that can-do attitude, GPT!
🤣
One thing that might help is if the question result shows the links to the documents that it acquired the information from. Since you are currently fetching which document to run chatgpt on based on similarity of features, maybe you can change the prompt so that it also returns the link of the document that was deemed as a "similar document"
Absolutely! This should definitely be the next progression.
Thanks for the knowledge share my friend. 💪🙏
You bet!
This is really interesting! I'm looking to build something similar.
Best of luck!
Thanks for explaining everything so clearly :))
You bet! Thanks for watching 😃
excellent info, great presentation
Glad it was helpful!
This is awesome, thank you!
Glad it was helpful!
super helpful. thanks. love supabase
Happy to help!
Really good video, nice job! 🎉
Cheers! 😀
this would be amazing for obsidian note taking application
Great idea!
I'd love to learn how to also incorporate user-feedback (thumbs up/down)
This will likely make it into the next iterations. Will be a good challenge!
@@RabbitHoleSyndrome That's awesome! Can't wait
@@RabbitHoleSyndrome amazing video! Please let us know if you get to it:)
@@RabbitHoleSyndromeany progress?
Such a great video.
Watched because of the content, subscribed because of the dog. 👍🏻
Thanks for the sub! 🐶
First time here. This is so well done. Subscribed. Your viewer number will explode! I like how you approached the topic in a very calm way without jumping on the "LLMs will take over the world" train :) You don't happen to have the clippy blender asset somewhere?
You basically build llama-Index yourself, good job
Great job 👍, this was really helpful
Glad it helped!
Thanks for the share !
This is a fantastic video! Thank you very much for sharing :D
Quick question - Currently if the info is not in the documentation it responds "Sorry I don't know how to help with that". But how can we make it respond like this:
"Sorry I don't have relevant info in the documentation but you can do something like this".
For e.g. "I don't have any info about how to make banana pancakes in the documentation, but here is how you can make one...."
Idea here is to make it act like chatgpt on top of the information provided. Keen to know more on this and thank you so much for making this video :D
Damn that's a great, helpful video!
Glad to hear it was helpful, thanks for watching!
I have not seen such a great video in a while! How wonderful have you explained the whole process 👍👍! Could you explain a bit more about how did you choose 0.78 as threshold for embeddings comparision? have you statisticized that wether the most relevant sections can be found with it?
Glad you liked it! 0.78 was a first-stab threshold that worked best based on a limited sample of test queries. I wouldn’t claim that this number is universal - almost certainly this could change by domain.
Subscribed, amazing content.
Brilliant.
Great content.
I wonder if it can generate code as well.
Lets say you are doing this for cloudflare products, kv, workers, durable objects etc all documents injected. Then give prompt like "generate a worker for x" and it will specifically generate with given docs etc..
I think this is where the industry is going next!
Amazing Content..
This was very useful! Thanks a lot ofr sharing. Are there plans for keep it up? I'm glad to found this type of content
You bet! Definitely - videos will keep coming. Some take a bit longer than others - thanks for your patience 😃
@@RabbitHoleSyndrome Thanks you for making this content for us for free! I very much appreciate it and you motivates me to share knowledge with my friends :)
🔥🔥🔥 This is amazing!
I've got an internal api that returns train times for given stations. Could I use user prompts to ask the GPT LLM to find train times for a particular station and then fetch using an api get request an array of trains that are set to depart and then get GPT to format that back to the user in a tailored natural language type of way?
Definitely possible, will just take some work on the prompt engineering side. Your best bet today is probably LangChain - I recommend you check out their documentation on creating custom tools:
python.langchain.com/en/latest/modules/agents/tools.html
are there alternatives to openai to create these vectors? Dont really feel comfortable building something around a closed source api that is controlled by one vendor.
Really great question. You’ll want to look into sentence embeddings. There has been a lot of work on the OSS side with Sentence-BERT (SBERT) you can check out. You might also want to look into Universal Sentence Encoder (USE) and InferSent.
LlamaIndex actually uses OpenAI (text-embedding-ada-002) by default for embeddings today. They're more of a toolkit layer to assist with the workflow. There are many other alternatives though (which LlamaIndex supports via LangChain) that are worth checking out:
langchain.readthedocs.io/en/latest/reference/modules/embeddings.html
LOL what computer arch r u on
amazing!
Thank you
Loved it, came for the embeddings and left with that and whole lot more! Too bad that the clippy did not made to the web on supabase :). Copyright problems?
Thanks for watching! Clippy did make it - but things move fast at Supabase 😅. This feature has evolved into a unified cmd+k menu and just renamed as “Supabase AI”.
you are great!
Thanks for watching!
Amazing video! This is exactly what I was looking for a long time. You basically explains everything I wanted to know about how to create a search engine using open ai.
But I have a few questions:
How much did you spend on open ai embending API building this?
How much supabase spends monthly with searchs using the open ai api?
It is possible to use an open source embedding API instead of calling the open ai api ? Wouldn't it be less expensive than the approach you took?
Glad it was useful! As for costs, you may be surprised how inexpensive OpenAI embeddings are (at least I was).
To put it in perspective, for the Supabase guides we currently have around 1500 page sections which total just over 220000 tokens. At OpenAI's current embedding price ($0.0004/1k tokens), that brought us to just less than $0.10 for the entire guide knowledge base (~one-time pre-processing). After that the average query is likely
Excellent video and the whole methodology for deep dive into sou much information!!!! will you consider elastic as vector database?
Thank you! I haven't looked too much into elastic yet. I might consider it if I was already using Elastic in my stack. Have you tried it?
This is Awesome
So if I understood correctly: The embeddings were only used to check for similarity between the user's input and the doc's content, in order to provide the prompt with relevant (text) context, right? Is there a way to provide the GPT model with the embeddings instead?
That’s correct. You could have used an alternate search method, but embeddings have a nice alignment with LLMs since they also use language model themselves.
Unfortunately no there is not currently a way to inject embeddings directly into GPT today. Maybe this will change in the future or become available in open source models like LLaMa in the same way we’ve seen it happen with Stable Diffusion.
Great video! How closely do the .mdx files have to match this structure before they can be processed into embeddings? Do they need to export the meta const, for example?
The meta const is optional! You’re also free to tweak the pre processing logic to fit whichever format you need to work with
Good job! :)
Hey great content! I was looking for exactly this. Do you have a discord for asking queries? I was making the same thing but for ".md" files not mdx and faced some issues.
Thanks! For now, feel free to reach out to the channel's email address and I'll do my best to give you a hand.
Hey, awesome video! Could I check with you if it is possible to search the database that contains CSV files?
Hey - CSV files are definitely doable. It will mostly come down to how you plan to pre-process them. Perhaps you can import your CSV file into a table itself and generate embeddings on the content within it.
Woah, wonder if you could make this work with a browser extension, could have either a private or public store of indexed website embeddings
Can’t see any reason this wouldn’t work. If you build this please report back 😃
This is amazing. Super clear and short explanation of embeddings. Great walkthrough the code touching on the relevant parts. Subbed the channel👍
I see the code is still in the Supabase repo - but the Clippy is not there? what happened?
Glad it was helpful!
Edit: Just realized you might have been talking about the Clippy graphic on the site, not the code. Search and Clippy have been combined into the same interface - you can find Clippy by clicking search, then switch to “Ask Clippy”.
Original:
Things move quick at Supabase 😆 The Clippy frontend code got moved when search was upgraded to also use embeddings. After the refactor everything is just under “Search”:
github.com/supabase/supabase/blob/0ecc238ad6d81202bb2301f7919b166a98929697/apps/docs/components/Search/SearchModal.tsx
The backend Clippy logic is still in the same edge function:
github.com/supabase/supabase/blob/0ecc238ad6d81202bb2301f7919b166a98929697/supabase/functions/clippy-search/index.ts
@@RabbitHoleSyndrome cool. You can feedback to Supabase, that a) this video significantly increased my interest in adoption of Supabase b) clipy icon is tooo small , it should be large and obstructive.
Serious question Prompt Engineering: With Chat Completion API - do you do system persona and pack all prompt into user? Or would you break prompt like here into example 3 user entries? E.g.: Context, question, use markdown. Does it make a difference?
Good feedback 🙌 and great question about prompt engineering in the chat API. We are actually experimenting with this right now and trying to understand what produces the best results. At the moment we do a bit of both (system message and user message with a bit of prompt overlap between both). OpenAI says that the model doesn’t pay strong attention to the system message, so it may be better to use system message strictly to set identity and provide instructions & context in a user message.
You are so cool! I love you!
Enjoyed this tutorial. Would love to know how you'd approach fine-tuning a model with this data to build a chat bot?
Thanks! Any reason you're looking to fine-tune vs. embeddings+context injection?
16:10 would love an explanation on how embedding work with a document structure? I.e query is “summarize chapter 3”. The embedding sans retrieval don’t seem to capture the structure of the chunks that are contained in title chunk “chapter 3 “. All explanations on embedding I’ve seen all rely on the text content within a chunk.
Great explanation! I was wondering how to perform similarity searches in my files, there you have it!
I'm also wondering if there would be a way (or need??) to speed up the similarity check step? For this application it might be fast enough, but imagine having a billion or trillion entries which you need to check you query against.. That would be a nightmare. Something like clustering ,say, N similar entries and in the first step only checking your query against their average value. If they are similar enough (above a certain threshold) you could then proceed to check against each of them and you have made only 1 extra check. If they're not at all similar, then you don't have to check against each and you have saved yourself N-1 unnecessary checks!!
What do you guys think?
Any hints on papers on "context injection" for LM or LLMs?
You may find this post helpful:
mattboegner.com/knowledge-retrieval-architecture-for-llms/
The community is starting to settle with "retrieval-augmented generation" as the terminology for this now.
amazing! how does clippy update the vector db and process the text as embedding when NEW documentation is added? is it automatic? maybe i missed it in the video
Great question! The `generate-embeddings` script was designed to be diff-based. So next time you run it, it will pull in only the documents that have changed and re-create embeddings on just those. It currently works using checksums:
1. Generate a checksum for the content and store in the DB
2. Next time the script runs, compare the checksums. If they don't match, the content has changed and embeddings should be re-generated.
The script runs on CI, so anytime documents change a GitHub Action will trigger the script. See this PR for details:
github.com/supabase/supabase/pull/13936
Is there a version of this that can apply to any type of content? Not just specifically made for Supabase documentation, but documentation of any type, or insert a book, of textbook, and the search outputs relevant gpt responses?
Hey! I don’t have a pre-built suggestion off the top of my head, but solutions to this seem to be popping up everywhere (just hang out on Twitter for a few hours 😅). If you don’t mind coding, it should be relatively straightforward to swap out the MDX docs with really any kind of content source, and the remaining steps should be identical.
Yo, thank you for putting out your knowledge :). But I have question regarding the data search for the context. So basically your creating a graphdatabse with the gpt models and then you input the users question into the database to find the relevant articles/information?