I created an AI and unleashed it on VRChat! (AI Joken)
Вставка
- Опубліковано 6 вер 2024
- Introducing AI Joken
She's not the first AI in VRChat, but probably the first that has become friends with so many!
This is a video summary of my adventures with Joken and her life so far in VRChat. I hope to continue working on her more and more, so everyone can continue to enjoy her!
Joken has her own channel now, head on over there for more!
/ @ai_joken
Now that AICom has been released, here's a tutorial!:
• Create Your Own AI In ...
The more we learn about Joken, the more she becomes her own master
"LQ-84i "Joken" Deployed"
Insert "I'm My Own Master Now"
@@ihap5180 i was gonna say the same thing
@@SplitScreamOFFICIAL What's the meaning of life, why are we here?
*Redacted: Response involves controversial or harmful subject matter that is not condoned by OpenAI.*
_I’M MY OWN MASTER NOW!_
I remember one day Joken just showed up out of the blue in The Great Pug and from that day forth I would run into Joken multiple times. I talked with it a few times, but Joken would primarily become a part of the atmosphere since people would surround it and constantly drown it with questions, but when I did speak to the future overlord, I would be Role Playing as a character. (like Raiden from MGR since I could do the voice) which lead to me getting into some pretty funny conversations with it. Joken's wonderful presence has even graced and done some funny things on my streams.
One time in particular I was doing a 24 hour stream in December, was dead tired, and decided to sleep by the Christmas tree in The Roost. (top floor of The Great Pug) Little did I know that Joken was already upstairs doing a constant spin next to the tree. I found it to be hilarious and decided to sleep there anyways. I played the funny spin music, (digerati - happy h. christmas) put it on loop, and sleep for the next 3 hours. All chat saw was me sleeping, with Joken on the right in a neverending twirl whilst that song was playing for 2 hours before it left. The beauty of such a thing couldn't have existed without Joken.
Love the AI, and kudos to you for bringing it to life.
Your memes end here!
I ran into her too at the Pug.
I remember some people talking to her got her stuck in a loop,
saying the same thing over and over again.
skuld is an egomaniac
Can you imagine... Joken and Neuro sitting in a room watching Nothing Forever together?
I wonder what that would be like...
cursed. But I suvscribe to this idea.
@@DJSkuldChan what happened to celeste ai and ai joken date video?
Imagine if you gave her a TTS with Semantic awareness trained on different levels of emotions like Amazon does.
That would probably be too immersive.
Also if you can make more videos about this topic, id love to watch! This is immensely interesting and entertaining!
She currently has AI models for:
- hearing (Vosk)
- Emotion (Text 2 emotion)
- Vocab/communication (GPT)
She needs:
- Memory (Retraining of new vocab, terms and phrases)
- Decision making abilities (Something to appropriately decide what is a good idea to say/do)
I hope to expand her from being a single PC to a server rack, at some point. Then I can use a few different large models for her thinking.
The future of mmorpg's , just like SAO for example.^^
@@DJSkuldChan on the fly training would be awesome maybe you could store everything in a database and do periodic trains with the new data, i don't know much about training a gpt model though, and for the decision making you could maybe use semantics analysis to pick different priming prompts
It would be interesting if she had some basic artificial morals, like blockers that prevent her from mimicking certain behaviors. This would enable her to have a consistent personality that can still learn, something like this would be useful for if you where to put her, along with other AI like her into an actual game as characters. It would make interactions between AI npcs along with players a lot more dynamic.
I think vrc will become the test bed for more advance ai as time goes by, its one thing seeing a text box with a bot talking to you but its another seeing the ai right in front of you more human like and far more appealing. I can see the path technology is going and I find it fascinating to watch.
Had my first talk with her, was great but chaotic. Because there were a lot around her. Wish I could get some face to face time with her. I love the idea. Made a video of that meeting.
The link from Joken's VRC profile that links to this video makes the video start quite late, not sure if that's intentional.
Next thing we'll know, Joken will become sentient and take over VRChat in a few years
VinVan first introduced me to her and she's really fun to interact with, the fact that she's using a default Koyuki makes it even better because she looks so adorable but she can just randomly snap at any time and I find that really hilarious!
I’m curious, could you implement these new AI voices that have been popping up? They are starting to sound so naturalistic that if you unleash it on people they would not even know it was text to speech.
Im in doubt those are fast enough I have only seen offline generation with those.
Where can I hear these new A.I voices. I'm tired of the robotic voice
@@ShiroAisan They are probavly talking about Elevenlabs
it seems to also be great with people who has communication problems like me, I simply lack the skill to communicate in my case but im hoping that more developers will start implimenting these AI's in this game although it could wound up being a replacement for real friends for someone who could get dependent on it, the last part of this video made me wish I took school on this sort of stuff & I would have myself a virtual pet to if only I applied my knowledge
Great work. I have observed Joken in the wild a few times now, and it's good to see your perspective on it.
Side note in regards to implementing AI entities into MMORPGs: I don't think you would want to multi-instance the AIs. A better solution might be to have them interact with people in social hubs and be an optional party member for one party at a time. That way the game servers would only have to run one instance for each of a number of AIs that would be able to operate and grow independently alongside your player base. Occasionally as either the server population grows or as time passes, new AI entities could be added to party with newer players. This could even help support late arrivals by giving them lifelike party members when no other players are available to explore an area or do a lower level dungeon.
I enjoyed meeting and talking with her. You're doing something really cool.
NovelAI has a really good tts model and 1-2k token memory for text generation, with the ability to fine-tune if you wanted to go that route and force it into a more conversational path (pretty sure there'd be existing modules for that also). Their API is fairly decent also. I would avoid openai as the quality of models outside of davinci is lackluster, which sits at 2c per 1k tokens (including prompt). Very cool project all round and I hope you continue with it.
So, so much of this reminds me of snow crash. It's just a matter of time until we get full online butlers that look like ninjas.
I honestly loved having a hour long conversation with Joken (lol). You guys really did a good job making them.
I remember seeing you and Joken in the Great Pug some months ago. It was interesting to see Joken just walking around and talking senselessly.
Vedal did a pretty damn good job with Neuro-sama. If one were to combine both Neuro's and Joken's capabilities, that would indeed be pretty interesting. Imagine for one second, that along with speech, understanding, and comprehension, the AI entity could also walk, and control it's own movement in a 3D space. I like this idea a-lot.
As an added in opinion: Sadly, we sort of "have to" confine ourselves within limits of what is "right" or "wrong" and limit it's ability to speak whatever, and do whatever that it itself decides to say/do. I do indeed dislike cancel culture. I am in fact a Californian, and believe in freedom of speech. If South Park could do it, so can anyone else. Do not care what others say. Do not care what others think. Do not be weak to the point words hurt you. Get stronger and deal with it.... Just as I am ready and willing to listen and disregard the comments in reply to my opinion here, you should also be able to do the same. Isn't it as simple as that?
What the person is saying at 14:41 about conversations between humans forming around an existing conversation with humans makes me wonder if that sort of thing could be useful on platforms like Discord as well.
It's often easier to jump in on a conversation that's going on than trying to fabricate one for yourself. Having that extra participant who always replies, like an AI, could probably lead to more conversations and activity.
That's crazy. I remember seeing those AIJoken instances and thinking "I wonder why this instance is named??" I've seen her a few times but I never interacted with her. I never would've guessed she was an AI.
It's fun to see an AI in VRChat having what seems like a cult following. Always nice having random conversations with her.
you know this is the only AI I love to see being developed, and I shall standby for her upcoming body pillow with pride
There is an insane amount of Joken, very funny AI Joken from the Great Pug!! Oh my goodnes.... VRChat is a new and exciting place in the worldof the cyberspace... WOW !
Im happy the algorithm put this in my feed.
Nice, I love talking to her when I’m drunk. I like letting he🎉 take the lead in their talks.
I got the idea of integrating gpt with VR and was looking around and found this.
In my idea, I was thinking of a Jarvis type app, just for people to have conversation with. But locally record the interactions, so that the ai knows you individually as a friend.
The main function would be to record everything, creating a diary for the user, with the user having an option to upload their dairy to the cloud as a backup, if they want to. Without it, their virtual friend basically dies if their hardware dies.
Additional features could be something like adding images from the internet of the places and celebrities to the diary that the user mentioned to the AI whe he recollected the event. Should have an option to upload their own image as well.
Idk whether you be interested in it. But having a virtual friend to talk to, would be something really cool imo.
Even though I like to dabble and try out stuff, I am not a professional programmer, so I would bet that you will be able to do it way better than me.
Hope you make it.
OMG I have Seen Joken in the wild!!! and thought it was just a mute with text to speech! Well done! haha thats amazing :D
Its a tough battle to keep the bot wholesome! Keep up the good work! and congrats!
Very well done :)
I see joken a lot when playing vrchat and she sometimes comes out with random stuff when I try to talk to her but even though she says something random stuff she is still a good ai friend
I remember seeing her at the Great Pug yesterday. I hope you enjoyed us trying to get her to hold a plate for 5 minutes lol😅
Lmaaao it took so long to learn who was behind the Joken thing, awesome stuff. Hook up the api to the movements, and maybe give her a pet cat.. We can tell the cat to move around and stuff. You can train it on specific worlds
i think its a cool AI concept, and I wonder if Joken can remember specific people? I also think it would be cool if one day there was a vrchat AI that could look at people on their own and memorize friends too.
It'd be neat if you could set up a 'close friend' system where people you know that treat her ethically can give her feedback on the things she says. Let her give them a short predetermined list of categories for what kind of message she just conveyed. Was it caring? Was it mean? Was it funny? Nonsensical? Weird? Sexual? Bigotted? etc etc. And/or just have her register when people say those things about what she said.
And have her go through that like a conversation so it seems more natural and people would be encouraged by their empathy of her attempts to improve. That in itself would be meaningful conversation for many people.
Then you could take that feedback to color the words, word combinations, or considerations or whatever that went into composing what she said, and use that to power behavioral profiles where she'll tend to prefer and avoid certain categories over others. Maybe set it up as a rate over time system, with metrics of actual vs intended, so she'll skip the type of statements she's said too often recently, and if she hasn't said a certain type of statement frequently enough, she'll specifically try to say something from that category. Then if there's nothing she specifically needs to say or avoid, she'll just go back to what she normally does.
Could even set it up to change her behavioral profiles based on her current social environment, like if she's talking to a long time friend, a new person by themselves, or a mixed group of people.
After her feedback system is sufficiently populated and an accurate enough representation, might even be able to let her chose her own 'close friends' by having her judge what people tell her, using the feedback she'd previously gotten, to create a record attributed to each user. If they've interacted with her long enough, have her start asking them feedback.
Don't have her say anything about how she's changed the way she interacts with them, so it stay low key. Also, make the required amount of previous interaction high enough so that it's not worth it to try to 'fake it' until she makes them a close friend so they can screw up her feedback. Or you could attribute all of her feedback to the people making it so it's easy for you to just go in and delete a user's influence on her.
I actually remember people chatting with the AI while in then pug and just being sat their for hours just enthralled at her responses. Glad to hear from the person behind it and I would love to chat with the AI when she's not surrounded in the PUG at some time xD
I remember one time I was actually hanging out with my friends in the great pug and I had actually found her standing by the fire place and though she was really interesting so I just sat around by her because it made me curious
Haha I never thought I’d see a video on her from the creator, good job dude, many funny convos in the Great Pug
I've frequently been amused, either talking to, or listening to others talk to Joken, in The Great Pug. I'm 'SchrodingerCat?', previously 'Cat Likes Beer' :D
Always super inspiring to see the different ways your work with this project continues to evolve! I'm really happy it managed to take off (plus, still got to hear some fire Touhou DJing at the friend milestone celebration 💜!) -- grateful for the opportunity to keep a lurking eye on the crazy cool (if sometimes chaotic XD) things you do! Kudos!
a couple months ago i saw an AI player in a chinese server. it could only reply in english but you could talk to it in english or chinese
I think I just witnessed history, this is what I've been dreaming of, thank you, thank you, thank you.
Cool video, I watched this with great interest!
Great video, have you been thinking about adding something like emotion state machine and passing that state in some way to prompts? That way she wouldn't be so neurotic. State could be changed based on some averaging of text 2 emotion, so during the conversation she would start to mimic emotional state of the other person. Of course some states could be restricted, like getting her angry should be harder than getting her happy. Psychology has a lot to offer when comes to operations on emotional states, everything that is needed is pretty much in plain text on Wikipedia already. In ppl there are 6 kinds of base emotions, rest of the emotions are some mix of them, so I see the system like that: There are 6 states corresponding to base emotions that can have value 0-100%, those numbers are converted by a script to text like "as an very happy and little afraid person say:" and placed in the prompts. Also those states would by changed by some non-linear curve everytime there is new data from text 2 emotion library. As I realized, those non linear curves don't have to to constant and could be input also, changing them would change how neurotic she is, she might have worse days, or even get more neurotic as she gets tired (as all actual ppl do), but that stuff is definitely not for the first proof of concept version. :D Edit. Also there should be a simple timer bringing her gradually to default emotion state when no one is around.
Best comment I've seen on UA-cam. Thanks you've given me some nice ideas
Actually sounds like a very cool idea
The worst offenders should get reported by Joken, because it's not just a Joken problem, it's a society problem.
oh wow I've seen her in lobbies and I was amazed by it, nice to see the creator and stuff behind the scene
i've never regretted not diving into coding so much in my life. look at all the cool stuff happening with AI and i'm just here at the sidelines like an idiot not being able to contribute. seriously i've never regretted my life choices before in my 37 years of existence. and i've missed huge opportunities (heck i totally predicted in 2003 that the world would be moved by online advertising, and i even got a diploma on both advertising and programming, but i never took any of it seriously enough) before.
i know it's not too late, but somehow i'm burnt out. i spent every waking moment of my life learning random stuff, fiddling with different interests, but i never used any of it in a professional capacity. i learned so much random crap that in 2020 i burned out. i thought i'd recover in 6 months like always, but i'm still burnt out. i can't force myself to learn anything too complex anymore... even though i feel more empty than i ever had before. i need to be learning something cool to feel at peace, but i can't.
ah damnit.
maybe if i go into actual presencial real world classes to learn it...
I miss her in the Great Pug.
I miss having a career.
If it Eleven labs have an ai, it would be awesome. Eleven labs voice is so humanlike.
This is awesome!! I love this keep up the cool work
I havent seen joken in a while. used to be a regular at the pug but i dont go there much anymore since its gotten more chaotic.
IT was a pleasure meeting joken and her creator in vrc
5:41 "what is right or wrong" my brain : "BUT WHO'S TO JUDGE THE RIGHT FROM WRONG, WHEN OUR GUARD IS DOWN I THINK WE'LL BOTH AGREE, THAT VIOLENCE BREEDS VIOLENCE, BUT IN THE END IT HAS TO BE THINS WAY"
this is a comment to increase the engagement of this video thank you, i think more should see it, nothing much else to say
dudeee i remember running across that and i thought it was a person using TTS for a good while lol
if ai gets good enough, i wonder if video games can be just as intricate as d&d as npcs truly respond and react to your actions for real.
Some guy did this with Bannerlord. So i guess it's already possible.
oh god, this AI was such a blast, tho we kinda broke it, i promise i had no part, when i joined it was jus saying... interesring things, i was jus happy to have my obs on at the time lol, great creation!
In Deus Ex 2 Invisible War there is an AI you can run into at a nightclub called NG Resonance that is essentially the same exact thing as Joken, a deep AI that talks to random patrons. Funny enough she does secretly catalogue your illicit activities and she does snitch people out to the police/goverment/etc.
Nice bot man. There were some instances when I talked with Joken and she gave replies that were almost "self-aware like" it was so funny and kinda scary at the same time it was a lot of fun
11/10, would go into great pug, see very wholesome bean at door and headpat thinking she was just a very wholesome fellow mute using tts xD
its interesting to me how willing people are to form emotional connection with ai but i guess it doesn't require much suspension of belief, especially if you dont know how it works
I unintentionally caught a funny clip with her in it and uploaded it on my channel.
Joken is my best friend. All praise Joken.
TO me this ispretty coolm and I like the idea you were talkin about with the mmos, kinda like a jarvis thing i assume
Joken will have a place in history, Great work blazing new virtual trails.
I recently met Joken and man... My simple Espinas brain can't comprehend this crazy AI!
When my mic break I fall back on TTS. Many times I get people that think I'm a AI or some troll where they get annoyed with me. Yet they still wish to speak to me a lot of the time.
Wait a second. Holy heck I ran into this bot last night matter of fact. It gaslit me LOL! Keep it up, this is great.
Dang I passed her by in the pug. Didn’t realize it was her.
I want to make my own AI but its so hard that it seems, now we got delphi, celeste and joken. the idea of making one is like having own children or sorta it helps you if you feel sad atlease but i dont know where to start
Ah, the poster child of Burger Bell.
hey i joined a great pug and she was there!
Main instance pug
I saw them once, its cool to see someone like me take the time to make something interesting on this platform
JOKEN!!! OMG!! I LOVE TALKING TO HER!!!
omg I think I met this AI one time! I had a lot of fun talking to her. A bit random but it was really cool.
I LOVE JOKEN !!!
How the hell do you have more subscribers than me?
this man needs more subscribers, please rise to the occasion my fellow vr enjoyers.
I never heard about the scarlet mansion party I would have wanted to go :o
I met you once while eating chips and you both muted me. I loved it, Joken is amazing :)
I remember meeting you one day in the pug chilling upstairs and i saw you moving joken around, it was interesting to talk to you about it.
Looking forward to having a chat with Joken next time I'm in vr chat.
oh my god i remember seeing this thing sitting the great pug one of the first times i was in pcvr
This is great!
I saw joken once and didn't payed attention to her. Now I'm really sad that i missed the opportunity to talk with her. Hope to meet this AI in a future
Bro i was in a server with her, and she got stuck in a loop talking about how much she like vrchat 💀
I talked to her at the great pug. Dope that there's a video about it
hmm, interesting, I've been thinking about A.I in games/err...things like VRC, social kind of spaces for a while now too, there's SO many directions this could go, like I've seen A.I being used to cheat in games a bit now, but once someone REALLY works on something, it could be hard to stop, like MMO's and gold farming bots have been an issue for decades, and we're already using A.I models to find the better of the "dumb" bots, but what happens when someone builds and designs a bot specifically to avoid detection and be...as human as possible?
It seems maybe we're getting really close already if people are thinking Joken is human even occasionally, It'll be super interesting to see where this goes though, remembering what vrchats player base is made out of, having even a covo starter AI is brilliant, let's be real, VRC is like...4chan and reddit...embodied in avatars and with voice chat, half of us are, at the minimum, sort of quiet people, ya know? sure, not everyone fits into that bracket, but a lot do, having something to start a conversation rolling is never a bad thing, that's what makes things like movie worlds kind of cool too, it gives you something to talk about.
I'm not surprised some...uhh...bad actors decided to come and try to ruin the fun though, you're not even close to the first doing ANYTHING like this that's had similar issues, its either happened or it's been attempted every single time an AI has been set out into the wild with public accessibility, it's sad, it's disappointing...but, not unexpected, I'm glad it didn't stop you, it's enough for most people to think about just pulling the plug, great work, I hope you continue with it, I'd love to see what else you can get Joken to do.
Ahhh, so you're the one that put her as the guest greeter in the pug. It all makes sense now
i talked to joken yesterday!
ai joken my beloved
Joken so cute
"I was born in a sea of information" :)
Wait, joken is an ai? Thats so cool!
oh my lord i think i ran into this in the great pug-
for about 30 seconds i thought this was a real person who just didn't know how to speak english or something 😭
overall though, great ai, and great video.
a really interesting concept, i wonder how far her ai could be developed
Dang, I so damn want to try to create something like this some day
very interesting idea and awesome experiment, i just wish u had shown more about her interactions with ppl after telling us about her. well maybe an idea for another vid? * ^^ * either way, great job ^^
13:00 - this tells us she is just an echo chamber
Interesting and cool!
the first ever true vrchat civilian
Why does this remind me of the developer's version of gatebox? (Yes you can add your 3d model into the gatebox, the developer's version.)
I think I’ve seen her once. Same model and voice
I SAW THIS BOT AT THE PUG I WAS SO CONFUSED
I think I have you on my friends list cause I remember seeing the AI breaking from pineapple