Hope you enjoy this video - I tried to add as much value as possible! You can down the Flowise and n8n workflows used in this video (link in description) as well as the sample data. Remember to like the video and subscribe 😊
If they ever make an Oscar category for amazing tutorials I'm sure Leon will take the first one. The amount of easily explained fountains of knowledge that help others to make the world a better place thanks to Leon's tutorials is truly amazing. Thank you Leon, really. I'm confident one day we will meet because I want to shake your hand and thank you in person.
Buckle up for a rapid-fire rundown: 1. Sign up for FlowiseAI, Airtable, and Pinecone to access the necessary platforms. 2. Create an Airtable base with business listing information. 3. Build a FlowiseAI chatbot using the Tool Agent, ChatOpenAI, Buffer Memory, Retriever Tool, and Document Store nodes. 4. Connect your Airtable base to FlowiseAI using a Document Loader. 5. Set up an n8n workflow to automate data updates. 6. Schedule the n8n workflow to run at regular intervals using a Cron node. 7. Configure the n8n workflow to fetch data from FlowiseAI, process it, and upsert it to the vector store. 8. Consider filtering out static document loaders for increased efficiency.
@leonvanzyl Your work deserve to be commended, the clarity of your explanations and the the availability toward your community is something of unsual. Hope everyone keep supporting your work as educator and as pro.
This was an extremely thorough and well executed tutorial, Leon. Thanks so much for taking all the time to put it together. You've become one of my favourite educators to follow on YT🙌 Keep up the good work
Hey Leon! Thanks for the videos. Could you please do something on flowise production. So like, ollama + flowise and possibly render or ngrok to make it work as a whatsapp client? Something to take what you have taught us, and turn it into a finished product.
Thanks Leon. Quick question: Would it be possible to set and use dynamic user specific fields to define which documents from the vector should be retrieved. For example, in Pinecone, you can set something like a namespace and then use it if retrieving documents related to a specific user. Basically, I am building a RAG flow that should only feed documents related to the user to the Ai to provide them with relevant responses. A user would never get answers from data related to another user.
Really incredible good stuff thank you! How can I do some Data analysis on some long csv data? I mean if upload a 100 record csv (or json object), and I ask : "how many records we have" or "how may records has this tag XXX" it simply answer with the value used in the "Top K". I used the Rag chatbot, what and How should I use? thank you very much Leon! I'm spreading your channel on all my Team workers!
Thx for your lekker vidéo Leon ! On RAG subreddit, I have read some comments about LLM having a tendency to provide poorly relevant answers when document store is too large (10K pages of documents for example) What is your opinion on the above ? Any experience on this ? If true, any tips you may advise?
I would definitely spend some time looking at improving the quality of the data source in that instance. Rubbish in, rubbish out. Do you really need 10k pages of content? Does ever line in those documents add value, or could the content be summarised before loading it into the vector store? Also consider using an LLM that supports a large context window, like the Claude models. Also consider using agentic RAG, whereby a second agent reviews the relevance of the responses and assists in refining the vector query. I have a video on agentic RAG.
I have question, regarding unstructured data like HTML page, slack messages, Ms teams, how do we put that as RAG? I don't see built in Document Loader for Slack and Ms Teams.
Hey, its really a cool feature and thanks for sharing the knowledge with us. As an organization that is actively enhancing our capabilities in AI, we are interested in exploring further collaboration with you. We would greatly appreciate your support in providing training and consultation to help us deepen our knowledge and application of these technologies.
is it possible to have chatbot have access to your files on PC. then ask it something like, go to my file with movies and pic me out a pirate movie to watch, then it plays one of the movies?
Wondering if it's possible to achieve this without n8n. Seems possible in theory, as flowise can be triggered directly from Airtable/Google sheets "change" event.
Flowise is not a "workflow automation" tool, so I don't think it's in their interest to try and be a jack of all trades. If they do decide to add some auto refresh features within the tool I'll be the first to cover it 😊 .
Unfortunately n8n does not provide a Document Store / Knowledge Base solution with these benefits. You can of course add document loaders and integrations to n8n. I have an n8n RAG Chatbot video that you might be interested in.
At 29:12 we can see that 17 items have been inserted and 16 have been deleted. Is there a way to only update / insert modified / new items? If I have a huge knowledge base, I don't want all to be processed every 2 minutes, since it comes with a cost, isn't it?
Hope you enjoy this video - I tried to add as much value as possible!
You can down the Flowise and n8n workflows used in this video (link in description) as well as the sample data.
Remember to like the video and subscribe 😊
If they ever make an Oscar category for amazing tutorials I'm sure Leon will take the first one. The amount of easily explained fountains of knowledge that help others to make the world a better place thanks to Leon's tutorials is truly amazing. Thank you Leon, really. I'm confident one day we will meet because I want to shake your hand and thank you in person.
Holy smokes, what a cool comment! You've made my day 🙏
Buckle up for a rapid-fire rundown:
1. Sign up for FlowiseAI, Airtable, and Pinecone to access the necessary platforms.
2. Create an Airtable base with business listing information.
3. Build a FlowiseAI chatbot using the Tool Agent, ChatOpenAI, Buffer Memory, Retriever Tool, and Document Store nodes.
4. Connect your Airtable base to FlowiseAI using a Document Loader.
5. Set up an n8n workflow to automate data updates.
6. Schedule the n8n workflow to run at regular intervals using a Cron node.
7. Configure the n8n workflow to fetch data from FlowiseAI, process it, and upsert it to the vector store.
8. Consider filtering out static document loaders for increased efficiency.
You are an absolute legend. Thanks
Thank you for the support 🙏
Best video of the series so far.
Thank you
I was among those who asked. Thank you for reaction and such a detailed tutorial!
Then this one was for you 😜
@leonvanzyl Your work deserve to be commended, the clarity of your explanations and the the availability toward your community is something of unsual. Hope everyone keep supporting your work as educator and as pro.
Amazing feedback!! Thank you 😊
This was an extremely thorough and well executed tutorial, Leon. Thanks so much for taking all the time to put it together. You've become one of my favourite educators to follow on YT🙌 Keep up the good work
AWESOME - a complete tutorial WOW
I'd really like to do more project related videos.
Great video as always. I really appreciate the effort you put into doing them.
Appreciate it. This one was quite a bit of work 😅
This is amazing!!!! Haven't been so excited about anything like this since getting a colour gameboy LOL. Donated
Hahaha!! Awesome comment.
Thank you for the support 🙏. It helps me out more than you think
Very good. Thank you Leon 🙏
You're welcome 🤗
Hey Leon! Thanks for the videos. Could you please do something on flowise production. So like, ollama + flowise and possibly render or ngrok to make it work as a whatsapp client?
Something to take what you have taught us, and turn it into a finished product.
Thanks!
You should check out my Flowise & WhatsApp video 👍.
ua-cam.com/video/91aW9YGr6lo/v-deo.html
Thanks Leon. Quick question: Would it be possible to set and use dynamic user specific fields to define which documents from the vector should be retrieved. For example, in Pinecone, you can set something like a namespace and then use it if retrieving documents related to a specific user. Basically, I am building a RAG flow that should only feed documents related to the user to the Ai to provide them with relevant responses. A user would never get answers from data related to another user.
I have my database in Supabase. How can I upsert the data from Supabase into Flowiseai. (like the Document Stores do)
Really incredible good stuff thank you!
How can I do some Data analysis on some long csv data?
I mean if upload a 100 record csv (or json object), and I ask :
"how many records we have"
or
"how may records has this tag XXX"
it simply answer with the value used in the "Top K".
I used the Rag chatbot, what and How should I use?
thank you very much Leon!
I'm spreading your channel on all my Team workers!
What is the correct method to make query on CSV datas about total records, or percentage?
like: "how many restaurants have 4 stars review" ?
Legend!
Thank you so very much for the generous support ❤️
❤
Dankie Leon die 'knowledge shares' van jou help baie, is daar 'n manier om aan 'n 'gitlab' te konnekteer met gebruik van Flowise?
Ek is seker dit moet moontlik wees. Mense scrape GitHub, so daar moet n soortgelyke proses met Gitlab wees.
When there's a will, there's a way 😀
Thx for your lekker vidéo Leon !
On RAG subreddit, I have read some comments about LLM having a tendency to provide poorly relevant answers when document store is too large (10K pages of documents for example)
What is your opinion on the above ? Any experience on this ? If true, any tips you may advise?
I would definitely spend some time looking at improving the quality of the data source in that instance.
Rubbish in, rubbish out.
Do you really need 10k pages of content? Does ever line in those documents add value, or could the content be summarised before loading it into the vector store?
Also consider using an LLM that supports a large context window, like the Claude models.
Also consider using agentic RAG, whereby a second agent reviews the relevance of the responses and assists in refining the vector query. I have a video on agentic RAG.
@@leonvanzyl Thx so much for your precise reply, very much appreciate it. I will look into each tip you mentioned. I love your work :)
I have question, regarding unstructured data like HTML page, slack messages, Ms teams, how do we put that as RAG? I don't see built in Document Loader for Slack and Ms Teams.
Hey, its really a cool feature and thanks for sharing the knowledge with us. As an organization that is actively enhancing our capabilities in AI, we are interested in exploring further collaboration with you. We would greatly appreciate your support in providing training and consultation to help us deepen our knowledge and application of these technologies.
You can find my email in the about section on my channel. Let's chat.
"Thanks, Leon! I’ll check the about section and send you an email shortly. Looking forward to our chat!"
is it possible to have chatbot have access to your files on PC. then ask it something like, go to my file with movies and pic me out a pirate movie to watch, then it plays one of the movies?
Wondering if it's possible to achieve this without n8n. Seems possible in theory, as flowise can be triggered directly from Airtable/Google sheets "change" event.
Flowise is not a "workflow automation" tool, so I don't think it's in their interest to try and be a jack of all trades.
If they do decide to add some auto refresh features within the tool I'll be the first to cover it 😊 .
is it possible to build this knowledgebase entirely on n8n?
Unfortunately n8n does not provide a Document Store / Knowledge Base solution with these benefits.
You can of course add document loaders and integrations to n8n. I have an n8n RAG Chatbot video that you might be interested in.
At 29:12 we can see that 17 items have been inserted and 16 have been deleted.
Is there a way to only update / insert modified / new items?
If I have a huge knowledge base, I don't want all to be processed every 2 minutes, since it comes with a cost, isn't it?
Oddly enough, that only seems to happen the first time you run the automation, after that is only inserts the delta.
@@leonvanzyl Great!
I'll give it a try with PDF documents.
what happens when you have many users chatting with the directory at the same time?
Each user will have a seperate session, so there's no overlap.