Generative AI on Raspberry Pi 4 and Beyond: Running Language Model Transformer on Embedded

Can we run a (not so Large) Language Model similar to GPT on Raspberry Pi 4 and still get it to produce sensible results? Let's find out!
Github:
WIP
Media Credits:
Epic Jenga fall
• Epic Jenga fall
MEET HK-47 - TATOOINE - KOTOR 1 - Star Wars Knights of the Old Republic
• MEET HK-47 - TATOOINE ...
This video features materials protected by the Fair Use guidelines of Section 107 of the Copyright Act. All rights reserved to the copyright owners.

[@user-er6ck2yu7b]

The great UA-camr returns!Awesome video, Dima!

[@Hardwareai]

Thank you!

[@user-qq4hm9ov3o]

Nice demo of Tensorflow, ChatGPT and Raspberry Pi!

[@Hardwareai]

Thank you! Glad you enjoyed the video.

[@gavinknight8560]

This really is great.

[@Hardwareai]

Thanks!

[@danilovaz9839]

Truly promising! Congrats! Certainly the best implementation of Generative AI on Raspberry PI that I have seen. Do you offer tutoring services by any chance? There's a similar project I'd like to implement! It consists of a digital avatar for a river, working with realtime sensory data.

[@danilovaz9839]

Oh, I just saw your 1-1 consultation. That's pretty cool. I have a question about it. Is the suggested value of $99 equivalent to a one-time consultation or is it more like until the project is implemented? Thanks!

[@Hardwareai]

Hello and thanks for watching!
About consultation, contacting me on LinkedIn would be the best :)

[@allanboughen]

Finally machines helping humans rather than humans constantly pushing buttons & reading manuals to help the machine.

[@Hardwareai]

I did read some manuals to make that machine xD

[@blakebullwinkel1619]

How much memory did you need to run the models on your Pi 4?

[@Hardwareai]

I used 2 Gb version. I did not measure the memory, but that was definitely not a bottleneck here, since the models are rather small.

[@SinanAkkoyun]

Could you please link the model weights? :)

[@Hardwareai]

Ah, yes. I was busy with my other project, check it out here ua-cam.com/video/Kj05IWS1hx4/v-deo.htmlsi=WQ_aVjOlZIbQpzM6. I'll see how to publish what I had for the toaster.