"How to 10x chatbot UX? 🤖 🖼️ " - Add Image Responses to GPT knowledge retrieval apps

A step by step tutorial of how to enhance your chat with PDF/website app user experience by enabling it to respond with not just text, but also images 🔥🚀
🔗 Links
- Follow me on twitter: / jasonzhou1993
- Join my AI email list: www.ai-jason.com/
- AI Researcher Github: github.com/JayZeeDesign/knowl...
- My discord: / discord
⏱️ Timestamps
0:00 Intro
0:43 Why GPT chatbots don't return imgs
1:23 Case study overview
2:20 Step 1: Scrape HTML
4:03 Step 2: Convert HTML to Markdown
6:22 Step 3: Create vector index with llama index
7:10 Step4: Retrieval Augmented Generation (RAG)
8:43 Handle PDF docs
👋🏻 About Me
My name is Jason Zhou, a product designer who shares interesting AI experiments & products. Email me if you need help building AI apps! [e-mail]
#gpt #chatgpt #ai #artificialintelligence #tutorial #stepbystep #openai #llm #langchain #largelanguagemodels #largelanguagemodel #chatwithpdf #autogpt #chatgpt4 #gpt4 #gpt3 #aiautomation #aiautomation #aiagents #nocode

[@jaysonp9426]

Dude, your content is fire. You keep finding use cases and angles no one else on UA-cam is covering

[@zidviziouz]

Couldn't have said it better myself!

[@user-vk5fj5mj8s]

Yes, I agree. Unique content. I'm looking forward to new video posts. Jason is cool 😎

[@PhoebusG]

Agreed, gained a loyal fan here.. keep up the good work ^_^

[@MarkRambotech]

agree

[@AhmedIbrahim555]

9:26 Yes please, 💯%open source, that what I love your channel for.

[@mrdbourke]

Jason your channel is outstanding for interesting and helpful use cases of LLMs, I've been experimenting with your techniques and loving them. Thank you and please keep it up!

[@kchapadia]

Incorporating images in responses definitely improves the UX. Would love to try this out with PDFs, an open source library to convert PDF to Markdown would be great!

[@fire17102]

Foss Pdf2Markdown sounds incredible! Please explore the idea further ❤🙏
When you do, please remember to design it in a way where we can easily jump back to (and hopefully highlight) the original source at the right place. Either on the Markdown (easier) or to the PDF (more complete)
Thanks a lot for everything,
You run one of the coolest channels for sure

[@MeanGeneHacks]

❤ 100% Open PDF2 markdown is needed

[@jameslibby5215]

I would love an open source pdf to MD library. I'm working on creating a "chat with your documents" program for my company so we can use our confidential data and that would be so useful. I'm a young CE, so I'm still so ignorant on so many things. Subscribed. Great content.

[@jasonfinance]

This is soo good, having rich media as part of response really improve the UX of the GPT apps!

[@half_way_expert]

Another great tutorial Jason! Keep it up and keep generating these new ideas!

[@garmeeh]

Keep up the good work! Content is always on point 🚀

[@rickylafleur4378]

I subscribed to couple of newsletters about ai but your videos are just simoly better than all of them

[@onangarodney7746]

I'd love to see the Pdf to markdown. You are doing a great job 👍🏾

[@adrianwhite100]

Awesome work Jason. Straight up smarts period.

[@pratiksingh3432]

Straight up, best content. Keep it up and we sure do need an opensource pdf to markdown converter.

[@asithakoralage628]

Fantastic video Jason, thanks a lot, I’m learning a lot from your channel.. thanks a lot

[@user-lv7ft2zu7k]

Excellent job of explaining the material. You make learning this stuff enjoyable. Keep up the good work and hell yeah, deep dive. Thanks

[@user-vk5fj5mj8s]

Good job Jason. Thx 🙌🏾

[@roshghadamian733]

This is excellent! Loving your content, would love to have an open source pdf to markdown package!

[@ajitdmonte]

Very nice content😊

[@Afzal3000]

Thank you so much

[@user-iv9om2ld2l]

9:28 great job! I’d love to try an open source pdf to markdown library.

[@seanxjin]

That’s some real value!

[@RussellLundstrom]

super impressive Jason, do you do any custom dev?

[@micbab-vg2mu]

thank you for the video

[@A28480]

Pdf to markdown would be fantastic!!

[@stepkurniawan]

awesome vid man! btw, an idea, what about using AI to update your notes automatically? :D

[@OryginTech]

Bro I’ll make a shrine of you in my house if you make an open source version of that converter 😭

[@briangrover1625]

🔥

[@bahamu]

I would love PDF to markdown!

[@albertalbert5785]

PLS MAKE A PDF TO MARKDOWN LIBRARY

[@sel3460]

Jason, brilliant work, please continue on pdf2markdown and come out UX with steamlit or gradio

[@chivesltd]

I'd love to see the Pdf to markdown!

[@musumo1908]

This rocks. Yes to an opensource PDF markdown. How do you suggest creating an API for this and any of the coded langchains ? I want to test dropping the API into a bot builder…the builder probably won’t support img response….will also drop into an appbuilder. Thx

[@sanchaythalnerkar9736]

How is LLama Index different from using the traditional emebddings and vector store?

[@robinwang6399]

I am currently working on a tool to extract tables from pdf as data frames and images as image references from pdf. I was wondering how I can display them, and happened on your videos. This is very helpful.
I am currently having trouble extracting PDFs with images of mathematical formulas and expressions, OCRs from what I can see tends to give nonsense. Due to how messy PDFs are, extracting image and table contexts and dynamically inserting them into documents that doesn’t have these data becomes a pain in the backside. (There was this PDF which had 5 to 6 images in a cluster, and displayed as a single image with texts and math around them, there was only one description for the cluster, my tool only managed to extract the first image and group it with the description.) I am thinking of finding AI models that generated texts from image, to dynamically generate the contexts alongside using OCR, but that can become really slow.
If anyone have any ideas, please help.

[@gawrysiak100]

thx

[@ithaca2076]

please we need a PDF to MD library that would be incredible

[@betun130]

I would be very interested in a pdf to markdown library version!

[@yixuanyang8516]

a quick question: does gpt 3.5 understand and analyze the img/video that's embedded in the html? because if so, then it's kinda doing what llava does

[@yashsrivastava677]

please create an open source version of pdf to markdown library.

[@jacobisknight]

i want that, would also be interested in building that for open source.

[@srh1034]

@AI Jason
Amazing video! Can you please teach how a CPU-based chatbot can be built without needing OPENAI and any other API key in case a user like me want to host the solution and not transfer sensitive data to cloud? And also minimize the cost of having a local CPU-based chatbot?

[@dharmendra_kumar_yadav]

Bro thanks I am also finding this can you make an ai chatbot that can give image and give output and also talk using model chatbot 😢❤❤❤

[@hocklintai3391]

I tried it it, but the rendered markdown's images are not perfect. I applied it on some other sites, and the images may failed to appear in some cases.

[@boypotus]

Yes many regurgitated tutorial | lacking in richness of experience agreed

[@lilspyguy]

How would you add chat history to this?

[@WN1917]

CAN you make a video where you train llm with pdf document containing exam questions and answers. Then the user could use that llm do generate questions and answer to question. Kind of an ai tutoring platform

[@xavierf2229]

Pdf to mark down library please bro

[@AhmedIbrahim555]

I,m just software engineer, what the exact necessary roadmap should I take to do what you doing.please😊

[@AIJasonZ]

Thanks mate! Id start with the prompt engineer :)

[@AhmedIbrahim555]

@@AIJasonZ I appreciate your response,thx 😊

[@python_handler]

create one video on research GPT using open source model ...not open AI. thanks in Advance..

[@tijendersingh5363]

Nice.. can we have the code file

[@emmanueldomains2829]

The most impactfull youtube channel I have come across. @AIJasonZ I sent you an email