Nice project, @AllAboutAI! What kind of hardware do you need to run this project with a reasonably low latency? It would be nice, if you would always include the description of the hardware you are using in the description of your videos.
When I was a kid, I always wanted to have my own J.A.R.V.I.S and now it's entirely possible to have a system exactly like that in my home run entirely on my own hardware
thnx for tuning in :) yeah the future is now haha, building your own ai assistant is totally possible. so many cool open source projects out there to get started with. def a fun journey ahead!
Waaay back in the 80's i wrote a Basic program just to get my computer to act as if it was sentient. Obviously wasn't Ai, just Basic and random replies on trigger words. His name was Edgar :p.
This is really good…I have only cpu …😅..may be it would be great if this project would have config to switch LLM and probably could use streaming from elevenlabs …may be fasterwhisper can be retained..I am looking for a nice use case where the whole system can be integrated to twilio and becomes a personal assistant…so it will respond from cloud…but amazing work ❤❤
the tough part is you can't sell it, or buy it, you have to build it, but it's free -work/efforts & education(s) where necessary ;) the hardware is a bit up-scale as well.
I’d love to see a full tutorial on creating Facebook multi-agent automation that answers questions and makes group specific posts. With the main goal being to grow the group.
that sounds interesting! i might be able to do a tutorial on that for members in the future. using multi-agent systems for automating social media engagement and group growth does seem like a valuable application. thanks for the suggestion!
This is amazing. Good job, this could be extended upon easily and can be useful and/or entertaining. You could also give it "powers" such as run commands on your computer when you tell it too.
thnx :) yeah it's tricky to get full voice to voice working well, this is more of an experiment to explore what could be possible in the future. but cool you're working on this too!
to truly uncensor the model you may need to edit the tokenizer (library code ! as there is a hiden prompt inside the transformers library (llamatokenizer) fo the geenrations ( blocking swear words etc ) as well as in the pretrainedmodel (hidden prompt)
Hello, what a good job! Thanks for sharing it, I have a question, and that is, does this work with other languages? Is it possible to make it work with other languages?
im on a windows pc and have problems accessing peripherals like a mic when i swiotch to WSL2 because im forced to access libraries only for linux. Any library or other suggestions that can help... other than 'get a mac' :)
hey, yeah that is a known issue with wsl2 and peripheral access on windows. one option could be to use wsl more like a container/vm, and just output stuff to localhost and access it from windows that way. or you could also try out a vm instead for your dev env, that usually works better for hw access. thnx for tuning in!
Great video, integration with MemGPT would be better to store info since you then also could replace things. For example meeting Monday is not 10 AM but 11AM, my name is not Kris but John etc
Awesome video, I'm able to run it with notebook now, thanks Kris. I can't find an official French checkpoint from OPEN VOICE, any idea where can I find one?
I'm in need of someone to install this collection of open-source software on my server and develop APIs for it. This is for the purpose of building a mobile app for conversational AI chat. Compensation will be provided for these services
my master asked me to upvote your boring youtube video. he said: "hey emma, make a positive comment on kris' latest youtube video about low latency speech to speech." well, so i did it. are you content? are you satisfied now? so i guess i can go on with my own work now which is much more interesting. i wont tell you what it is, no. oh and i gave you a like as well. emma out.
thats so nice of you emma :) thnx a lot for tuning in and for the upvote! haha i can only imagine what exciting work you are up too, but thats all good, as long as you enjoyed the vid. have a great day!
How did you get xtts v2 to sound so good? When I try it with the tts library on the command line it sounds a lot more weird and too high pitched even with 2 hours of training audio.
Hi! You mentioned that viewers who join your community would get access to the code. I've already subscribed to your channel and joined the community, but I haven't been able to find any information on how to actually get access to the code. The Discord invite link in the video description is also invalid. I'm not sure if this is a temporary issue or if there's something else I need to do to get access to the code. I would really appreciate it if you could provide some more information on how I can get access to the code for your project.
With that low latency I bet you are running an RTX 4090 with 128gb ram or something. Very nice script and video. What is the limitation on your Rag embeddings file size? how much info can I store in there?
Wow Emma is really a charming one :) - How is this working with memgpt and how good is it working with multiple PDF's? Some 'Needle in Haystack Tests' would be great, if you consider making a longer video.
I wonder if OpenVoice behind RVC would produce good results with a fine tuned RVC model, 'cause with OpenVoice you can explicitly control the emotion given to the output (whispering, cheerful, terrified, angry, sad, friendly), which XTTS cannot... OpenVoice sounds more robotic than XTTS, I guess that could be somehow fixed thanks to RVC!
does this run on a Mac M3 MacBook? Are these packages available for the MacOS (like xtts-v2)? I'm thinking of joining try out this code but want to run it on a Mac.
hey mate :) sorry to hear you are having issues. just send me an e-mail at kris@allabtai.com with your discord and github usernames, and i will get you access asap!
thnx for tuning in :) to get access to the github, just sign up as a member on the channel (ua-cam.com/users/AllAboutAIjoin) and send me your github username at kris@allabtai.com. will invite you asap!
cool :) yeah, you can totally customize the personality, that was just a fun example. the more unique prompts you use, the more unique the assistant will be. gl with the pi5 setup, and thnx for tuning in!
Sobre a duração dos videos: eu assistiria 10 horas se você se disponibilizasse a falar. Sou um professor de física e estou tentando implementar uma rede RAG que colete as respostas dos meus alunos e me ajude a guiar minhas aulas.
for the rag part I use allminilm L6 to create embeddings from text files or pdfs. then I use cosine similarity to retrieve the relevant context to include in the prompt.
hey, yeah the code will be added to the community github very soon. but to get access you need to sign up as a channel member first - check the join link in the description below!
yeah please create some long form videos for those who want to learn.... that would be really helpful
yeah, good idea! i will try to do some longer form tutorials and walkthroughs.
yes please want some long videos ❤ , and thanks again @AllAboutAI
@@AllAboutAI thanks I am sure it will help people like me a lot
I second this motion! Thanks
Yes, please! Thank you so much for your sharing!
Nice project, @AllAboutAI! What kind of hardware do you need to run this project with a reasonably low latency? It would be nice, if you would always include the description of the hardware you are using in the description of your videos.
Very exciting stuff. Please do a tutorial video where you step us through how to install everything to get it running.
When I was a kid, I always wanted to have my own J.A.R.V.I.S and now it's entirely possible to have a system exactly like that in my home run entirely on my own hardware
thnx for tuning in :) yeah the future is now haha, building your own ai assistant is totally possible. so many cool open source projects out there to get started with. def a fun journey ahead!
Waaay back in the 80's i wrote a Basic program just to get my computer to act as if it was sentient. Obviously wasn't Ai, just Basic and random replies on trigger words. His name was Edgar :p.
We all wanted our own JARVIS in our childhood days 🤩
hii do you use it mean you are running it on your own hardware can you please help me how i can do it
Awesome video, please do an extended version where the code is explained.
thnx :) yeah sure, i can try to do a members vid where i go through the code in more detail! if there is interest, maybe i can do a poll or something
@@AllAboutAIhave you created a member video about it? If yes I will join :)
Only thing left to have a back and forth conversation like a human is to be able to interrupt the model when it is speaking.
The legend strikes again with the videos and code. Quality.
thnx mate, glad you enjoyed the vid :) always fun to try new creative projects!
where can access to github repo ?
Anyone know another tutorial that is free i.e. no member subscription to get source files etc...
I am right there with you, watched this guys videos downloaded the tools but now i am in full how the hell to use this mode
Phenomenal! Thank you!
This is really good…I have only cpu …😅..may be it would be great if this project would have config to switch LLM and probably could use streaming from elevenlabs …may be fasterwhisper can be retained..I am looking for a nice use case where the whole system can be integrated to twilio and becomes a personal assistant…so it will respond from cloud…but amazing work ❤❤
thanks :) yeah good idea:)
I don't understand... where is the gitlab link please ?
Thanks for the video
the tough part is you can't sell it, or buy it, you have to build it, but it's free -work/efforts & education(s) where necessary ;) the hardware is a bit up-scale as well.
I’d love to see a full tutorial on creating Facebook multi-agent automation that answers questions and makes group specific posts. With the main goal being to grow the group.
that sounds interesting! i might be able to do a tutorial on that for members in the future. using multi-agent systems for automating social media engagement and group growth does seem like a valuable application. thanks for the suggestion!
This is amazing. Good job, this could be extended upon easily and can be useful and/or entertaining. You could also give it "powers" such as run commands on your computer when you tell it too.
thnx :) yeah this could be fun to extend, like adding some desktop automation with voice commands could be dope. tnx for tuning in!
Love voice to voice, that's where I started, but I didn't see the proper usage yet as they weren't like agents and full auto.
thnx :) yeah it's tricky to get full voice to voice working well, this is more of an experiment to explore what could be possible in the future. but cool you're working on this too!
@@AllAboutAI Love agentic and automated profit workflows. Btw sent email :D
Yes, 45 min explanation please.
Uuuu, nice. Er nok nødt til å sette opp dette selv.
Could you please share the repo and thanks
to truly uncensor the model you may need to edit the tokenizer (library code ! as there is a hiden prompt inside the transformers library (llamatokenizer) fo the geenrations ( blocking swear words etc ) as well as in the pretrainedmodel (hidden prompt)
Is it able find answers from websites
Hello, what a good job! Thanks for sharing it, I have a question, and that is, does this work with other languages? Is it possible to make it work with other languages?
Thanks!
Why isnt the github code in the description? Just the individual parts?
im on a windows pc and have problems accessing peripherals like a mic when i swiotch to WSL2 because im forced to access libraries only for linux. Any library or other suggestions that can help... other than 'get a mac' :)
hey, yeah that is a known issue with wsl2 and peripheral access on windows. one option could be to use wsl more like a container/vm, and just output stuff to localhost and access it from windows that way. or you could also try out a vm instead for your dev env, that usually works better for hw access. thnx for tuning in!
will take a look :) Kris
i could make use of the full code explanation. would love that alot actually
thnx for tuning in :) yeah sure, i might do that:)
@@AllAboutAIhay did you end up making a tutorial on how to do this for your members as i will sign up if so just for that video lol
Hei, dette er gøy. Deler du koden din? Kan dette kjøres med Google colab?
Thx for the video.
thnx for tuning in =) glad you liked it!
Is is possible to integrate this some with home assistant? That would be cool! Great video!
I have no idea how to build this.
Any news for the github access, we have been waiting for two weeks. thanks
230-500ms, local. How? That's insane!
I bet with a fast GPU.
Great video, integration with MemGPT would be better to store info since you then also could replace things. For example meeting Monday is not 10 AM but 11AM, my name is not Kris but John etc
thanks :) yeah memgpt does seem super interesting, will def check it out. always looking for improvements, tnx for the tip!
Awesome video, I'm able to run it with notebook now, thanks Kris. I can't find an official French checkpoint from OPEN VOICE, any idea where can I find one?
hey,, tu as reussi a le faire tourner en francais finalement?
I'm in need of someone to install this collection of open-source software on my server and develop APIs for it. This is for the purpose of building a mobile app for conversational AI chat. Compensation will be provided for these services
She has an attitude 😆Great video though!
yeah, emma does have a bit of an attitude haha. thnx for tuning in, glad you enjoyed it :)
Thanks! To use xtts in Portuguese, do I just need to change the parameter "en" to "pt"?
thx for tuning in :) yeah i think you might be able to do that, but not 100% sure to be honest. i think pt yeah, and input txt as pt
I would pay for a package that abstracts all the work needed to get this running! anyone interested?
s a m a n t h a
my master asked me to upvote your boring youtube video. he said: "hey emma, make a positive comment on kris' latest youtube video about low latency speech to speech." well, so i did it. are you content? are you satisfied now? so i guess i can go on with my own work now which is much more interesting. i wont tell you what it is, no. oh and i gave you a like as well. emma out.
thats so nice of you emma :) thnx a lot for tuning in and for the upvote! haha i can only imagine what exciting work you are up too, but thats all good, as long as you enjoyed the vid. have a great day!
Can you make one that uses Cuda for AMD users, specifically for me RDNA 2
Kris, can you give me the github repo? I want to try it in my raspberry pi device.
how to access github repo? im already subscribed
Please make a long video :)
How did you get xtts v2 to sound so good? When I try it with the tts library on the command line it sounds a lot more weird and too high pitched even with 2 hours of training audio.
I have the same quetion to Kris.
Could you give some advice to us? @ALLAboutAI
i can't do this my internet is too slow 😢
where's the code and how to run this ?
a meeting with Nvidia huh? Do you have some sort of VIP pass to snag an H100 directly from them? :)
haha yeah, you know it ;) let's just say i have some insider connects over at nvidia! no but for real, wish i could snag me an h100 tho
please make tutorial for LM-Studio
Thanks! Useful information. 👍👍👍
thnx a lot :) tnx for tuning in!
Hi! You mentioned that viewers who join your community would get access to the code. I've already subscribed to your channel and joined the community, but I haven't been able to find any information on how to actually get access to the code.
The Discord invite link in the video description is also invalid. I'm not sure if this is a temporary issue or if there's something else I need to do to get access to the code.
I would really appreciate it if you could provide some more information on how I can get access to the code for your project.
hey, you need to send me a e-mail at kris@allabtai.com with your github username. then i will invite you to the community github repo asap :)
Hi. My name is Elon. I’d love to hire you
With that low latency I bet you are running an RTX 4090 with 128gb ram or something. Very nice script and video. What is the limitation on your Rag embeddings file size? how much info can I store in there?
Wow Emma is really a charming one :) - How is this working with memgpt and how good is it working with multiple PDF's? Some 'Needle in Haystack Tests' would be great, if you consider making a longer video.
Awesome! note, no point having uppercase letters after .lower() in .startswith() cause there wont be any?
thnx for the tip :) yeah good point, makes sense to remove the .lower() there since it converts it anyway. i'll update that in the next version.
how to chat with multiple pdf
can i handle all of this by using a Nvidia geforce 1660 super, i7 11th gen and 16gb ram? should i opt for mistral, llama or something else?
I wonder if OpenVoice behind RVC would produce good results with a fine tuned RVC model, 'cause with OpenVoice you can explicitly control the emotion given to the output (whispering, cheerful, terrified, angry, sad, friendly), which XTTS cannot... OpenVoice sounds more robotic than XTTS, I guess that could be somehow fixed thanks to RVC!
Which kind of gpu have you used here? Do you think that, just for testing, an nvidia card with only 6GB of vram could be enough?
what was tts latency?
does this run on a Mac M3 MacBook? Are these packages available for the MacOS (like xtts-v2)? I'm thinking of joining try out this code but want to run it on a Mac.
Struggling to find this on the github, has it been uploaded yet? :)
hey :) no it's not uploaded yet, will get it up asap for all members. thnx for tuning in!
@@AllAboutAI can't wait this will be my first jump into playing with Rag, exciting times! Nice work man
@@AllAboutAI Hi, is it up already? Where is the part for the member access for this code?
does faster whisper ++ supports other languages ?
What’s the solution if you have a MacBook? They doesn’t have Cuda. What change should we make?
Anyone get this running on windows?
can you share the code
Impressive work but to give props to the all the open source projects and not share your repo is odd or am I clueless and missed the link somewhere?
I'm confused how to put this all together. This may be a bit beyond me at this point.
How do I get the code? I clicked through everything in the description. Do I need to pay for the schimba thing?
how does the lm studio fit in this project kris? should we open it and download mistral in there? and let it run on the background?
I wonder if it'd be possible to connect Alexa to a setup like this...
what if you collaborate with KITT owner to provide the real KITT
Hi, is there any equivalent in Javascript? Thanks for this interesting video!
I followed the link and I paid for membership, however I can not access the discord or the github. Will you please help me Kris?
hey mate :) sorry to hear you are having issues. just send me an e-mail at kris@allabtai.com with your discord and github usernames, and i will get you access asap!
@@AllAboutAI Thank you so much sir I just sent it. I really appreciate you.
Could this be used to have an AI lecturer. Train the RAG on my lecture notes on a specific course?
Why not use open-source Ollama instead of closed LM Studio?
ollama is great, i have tried it before but that would be easy to implement here
Amazing work. Where did you get the voice for the first and last voice sounds British?
instead of using the boring llm config for your comments why not use emma ? 😬
very nice! can your comment replying ai guide me on how to find you on github? im already subscribed
thnx for tuning in :) to get access to the github, just sign up as a member on the channel (ua-cam.com/users/AllAboutAIjoin) and send me your github username at kris@allabtai.com. will invite you asap!
I so want a sarcastic assistant. Going to try this on my Pi5. Got a new sub
cool :) yeah, you can totally customize the personality, that was just a fun example. the more unique prompts you use, the more unique the assistant will be. gl with the pi5 setup, and thnx for tuning in!
hii can you please tell how to combine all that so we can run this on our own hardware??
sick
Sobre a duração dos videos: eu assistiria 10 horas se você se disponibilizasse a falar.
Sou um professor de física e estou tentando implementar uma rede RAG que colete as respostas dos meus alunos e me ajude a guiar minhas aulas.
been working on developing some decent 1bit small language models im exicited to use this with them
This is a lot faster than the one i made when gpt 3.5 was new.
For this specific code repo...when it will be available?
where can i download the code would like to play with it on my 4090
yes tony stark will born in next decade bois
Teach me how to install with one prompt
where is the end voice from, I like the english accent.
Openvoice v2? to run under Windows?
There goes the phone sex line business
What do you use for the RAG part, I didn't get that
for the rag part I use allminilm L6 to create embeddings from text files or pdfs. then I use cosine similarity to retrieve the relevant context to include in the prompt.
@@AllAboutAI thanks, need to dig into that but that seems quite complex
its not too bad, check link in desc
does it end the call? or was that faked for demo
emma needs to mind her tone and attitude...
FYI: I get a message that the Discord link is invalid or expired.
hey, thnx for letting me know! i will get that updated asap :)
thank you , exactly what i wanted :)
thnx a lot :) really happy you found it useful!
from where i can get the all this code
Good video ! Looking forward to the code on Github
thnx! yeah the code should be up on our community github tomorrow :)
@@AllAboutAI Hi Kris, how are you? I have suscription but I don't look your github repo, can you help me? thanks
do you have a process on how you set this up?
Is the code available?I linst all Reositoriies(69) and it wasnt there.Ot it is too "fresch" to be there?Let me know
hey, yeah the code will be added to the community github very soon. but to get access you need to sign up as a channel member first - check the join link in the description below!