I built a *streaming* AI chat app
Вставка
- Опубліковано 16 чер 2023
- Learn how to use the new Vercel AI SDK to quickly build streaming AI chat apps with APIs like OpenAI, Anthropic, HuggingFace, LangChain, and more.
Upgrade to get the full SvelteKit Course at fireship.io/pro
Use promo code FKIT to save 35%
Vercel AI Announcement vercel.com/blog/introducing-t...
![[柴犬ASMR]曼玉Manyu&小白Bai 毛发护理Spa asmr](http://i.ytimg.com/vi/0TsXQ7z2Dh4/mqdefault.jpg)
It's incredible how many hats he is able to wear at the same time. I mean making high quality super dense tech videos, making top notch Courses, keeping track of all happenings and also creating such incredible side projects. Please do make a video about ur time management...I'm sure it's gonna worth ton to lot of people...!!
Yes exactly, How can someone keep up with todays tech updates and even do projects and tuto about it barely can keep up with what has been done 10 years ago HAHA
I run a 20k sub software dev channel on a separate account - It's hard to explain, but the drive you get from "I need to build this for content" is insane. I can hardly focus on my non-channel side projects because I don't have that fire under me, but when I'm building something for a video I just get carried away. Perhaps Fireship has a similar mentality.
Maybe he made an AI that is able to replicate his voice and he uses it to make the videos
In a previous video he showed his workflow using all generative AI tools so I think it takes very little time to make these videos, just time to develop the workflow
U got bamboozled, it's just an AI making the content since day 1 🤯
just started working with sveltekit, so nice
Fantastic work man. Can't wait for the Vercel course to be out on Fireship pro
Man I am excited for the SvelteKit course! Jeff for President!
Great video, very exciting for this! This video is very useful for me. Thanks!
Dude how did you watch it under 1 minute
0:01 train is from my country. Love from Bangladesh
in the time he programs, edits, and uploads the video, I made the basic chat app. lmao
In that time I made nothing
are you actually comparing yourself to someone with over a decade of experience? bruh moment
@@YuriG03042Could play a role but those are mostly hello world examples, things that take time are usually little details you need when developing a real app, you can ignore those for video tutorial purposes
Dear Jeff and any of you super coders out there, How would this work at scale? If deployed, how many simultaneous users could stream data given the standard paid api? And what might the code look like to deal with hitting that limit?
Easy. Use multiple accounts each with their own api keys and go brrrrrrrrr
@@chukaraslan9051 So create an api key for each of my users?
thank youuuu
Vercel jumped on it because it takes 5+ seconds to complete a stream, during which they charge you for each 50 ms of edge function your using.
So 100 edge units for each answer. USD 2 for 1 million units.
function makeMoreOpenAIAccountsForUnlimitedAPIKeysAndFckVercel () {
return "nothing to see here)
}
time to make one for myself!
The "fkit" course 😭😭💀you're too good man!!
FirebaseKit stack is too strong
For anyone trying to follow along at home for your own app, in case you get a 400 code (malformed response) and no other info the possible causes can be:
- Invalid or maxed out API key
- Bad prompt with illegal characters or too long
- in my case I passed in const messages to the request instead of const {messages} (oops!)
did you know that you can do it without Vercel AI SDK using OpenAI SSE (Server-Sent Events) Streaming API? it's not much more complicated
Got content or recs on how to pull that?
Frick all this... We need a video on how you keep yourself updated with all this and manage your time.
Newbie question from someone with no backend knowledge: What would I need to look up for if I would want to make a AI chatbot such as that in React? Any tips for starters?
Fortnite
Amongus
Cheeseburger
First of all - Don't build it in React... It's overhyped, overcomplicated and just plain slow. Meanwhile Svelte is like drugs, once you try it you don't wanna go back.
@@shapelessed svelte is react but shit. give me jsx expression any day of the week
What is the benefit of using this over just using the ChatGPT API? Is the only benefit that its streaming so typing the result instead of waiting for the whole result before showing it?
The *_"typing effect"_* makes it sentient.
Well I built that weeks ago using simple vanilla js (with streaming)
So in this example, you're using readable streams over HTTP? Are there any advantages/disadvantages to this approach compared to websockets?
nope
Pretty sure it uses a lot more bandwidth, since I think each HTTP request has a lot more boilerplate, like headers.
Unlike having to keep a connection alive with WebSockets, HTTP is much more simpler, and better in cases where client->server streaming isn't required. Also HTTP is probably more familiar anyway.
SSEs have always been a great option
@@potatohater5599No, HTTP SSE uses a single connection
Any idea how to maintain the streaming text while using the new function calling api openai just reelased? I had to disable stream to get it to work properly.
Not an answer, but a sidenote, functions are fucking incredible.
does Open AI gather your data?
Im not gonna say first that would be cliche, me being early means nothing, so Ill just say fireship if you see this you should make a video on hugging face transformers
Soon... HF transformers are awesome
Vercel's ai hype train 😂😂
can we use this in next js?
can we use open source local models?
I want to built this for my webapp..i mean it would be nyc if the user ask question on my web app and based on the data on the web app the chatbot responds..like if the app is about stack overflow for ex..the chatbot responds to the data according to user so he dont have to go through all the articles...can someone help me
i just did it in NuxtJS
❤
This isn’t possible with Firebase cloud functions is it? (The streaming part)
What if I want to run my own modem and have it train locally?
When I use this code, the roles are rendered as "undefined". What am I doing wrong?
OpenAI has revoked all free monthly limits for API Calls and has become a paid service now. Can someone suggest which other platform is free?
hugging face
thanks Fireship .. this github repository is definitely broken lololol
i deployed an openai chat app into an Ubuntu server but as soon I make it https the stream stops working, any clue why is It happening
This was definitely his AI voice
Bangladeshi Train for a sec 0:01
I wonder if customer support bots based on ChatGPT, used by big corporations, could be manipulated to get unlimited prompts on things not related to a real support conversation
Good question... huh. ChatGPT api afaik, has a system message - this messages basically tells the bot what it is. But hey, I'll have to look into that, sounds really interesting
They should do additional training on that part probably. Otherwise people just gonna write an entire book with it.
@@bittersweet4074and that’s bad thing
@@Batman-lg2zjnot nessicarily. As a matter of fact I may or may not have achieved huge success with the aforementioned method and may or may not choose to make the repo public in the near future
Thanks I have the app built now, I just need an API KEY 😵
So is the stream is just library streaming the full response? Because the API does not provide a stream of the responses.
The API does it already. You just need to add "stream: true" to the request.
I absolutely love your videos man! But as some one with sensitivity to sibilance, your s’s sound very piercing. It would be amazing if you could tame the sibilance a tad with a de-esser 😅😅👌
Fireship really doesn't rest. (but he did rest, if you see what I did there 😉😉)
Rest assured that he will rest at some point in time 😂
Is there a free alternative to this?
its important to note that it might be a good idea to add rate limiting to your endpoint incase you arent elon musk and can afford it either way
I am Elon Musk.
When you say app in all your videos do you mean mobile app or website? Because websites are not mobile apps.
Neither.. he means web application
I was in the middle of working and i put up this video as whitenoise
Got instantly more overwhelmed.
Waiting for Vue and Nuxt support.
Waiting? Why don't you give it a short?
Vue support just landed!
@@leerob awesome!
Now make it work with firebase
Yo
His voice is definitely ai generated
Beyond Fireship
IS the new
Fireship
where is nextjs video 😅
Streamberry
Can anyone here give me an idea what the best way would be to have a LLM that's trained on my knowledge base?
Get a life
That being said, look into langchain and RAGs
This is how AI super viruses will be created 😢
they just keep building SDKs on top of SDKs... guys, there are like 3 api endpoints of openai, these companies tries so hard to get a share and get you hooked to their eco-systems.
how can customized data for answer ????
By sending a system message
@@unconv i want use my data example a pdf and just see this file and answer , dont answer another question ...
npm i ai
69th
Can you do python
Schwarzenegger
Was this recorded with your AI voice clone?
@Beyond Fireship btw the earth is flat.
First comment :)
No I am
@@ropoxdev no you are not
@@cholaempire yes I am
Hi (first)
cool ill use your chat bot instead of openai so i dont have to pay for it 😂 jk
triggered by all these serif fonts. otherwise, brilliant as usual 🖖
first
No I am
Isn’t it a bit tasteless to make jokes about India and trains?
it's not a joke about either india or their trains? it's about AI and hype
The typing effect gets old for the user waiting on long replies, just don’t use it
But then you will have to show nothing or a loading spinner until the request is done. Which I think is a worse experience
@@riyaadhabrahams exactly, as far as I know, it's not an effect, it's just the actual speed of the generation.
That's the actual inference process, hence streaming. It's not a fake delay
dumb comment
Love the subtle fl@t urth truths that your smarter audience picks up on. Th
truths LMAO smarter audience
Can you do a tutorial on how (www.twitch.tv/trumporbiden2024 this) is done? Not sure what they are using for the backend. The AI model is uncensored. Be cool to know the process.
first
No I am
@@ropoxdev no fabyosk2664 is
check the comments from top to bottom