PlayStation 6 AI, Nvidia, AMD Hawk Point, Intel Meteor Lake | Game AI Developer | Broken Silicon 235
Вставка
- Опубліковано 29 лип 2024
- A Gaming AI Dev joins to discuss what hardware you’ll need to power next gen games!
[SPON: Use ''brokensilicon30'' for $30 OFF $500+ Flexispot Orders: bit.ly/3RcyPla ]
[SPON: “brokensilicon” at CDKeyOffer Black Friday: www.cdkeyoffer.com/cko/Moore10 ]
[SPON: Get 10% off Tasty Vite Ramen with code BROKENSILICON: bit.ly/3wKx6v1 ]
#blackfriday #windows11
0:00 Getting to know our guest, how to get into AI
5:15 What is Pygmalion building to change gaming?
11:31 The Next 2D - 3D Moment for Gaming could be Neural Engine AI
20:44 AMD Hawk Point and the Importance of TOPs in APUs
27:30 Intel Meteor Lake’s NPU - Does it matter if it’s weaker than AMD?
33:03 AMD vs Qualcomm Snapdragon Elite X
40:45 Intel's AVX-512 & NPU Adoption Problem with AI...
53:01 Predicting how soon we'll get Next Gen AI in Games
1:00:45 Can the PS5 run Next Gen AI? …what about the XSS?
1:16:26 How might the PlayStation 6 do AI?
1:27:19 AMD’s Advancing AI Event & ROCm, Nvidia’s AI Advantage
1:50:20 Intel AI - Are they behind? Will RDNA 4 be big for AI?
2:03:19 Will future APUs be as strong as H100? When will the AI bubble pop?
2:15:22 Will AI hurt Gaming long term?
2:33:14 AI Ethics and AI's impact on Artists
MAJOR SPONSORS:
Support MLID by getting a FREE SSD from Microcenter: micro.center/brokensilicon
Cheap Windows & Gaming Keys:
25% software discount code: brokensilicon
3% OFF EVERYTHING discount code: dieshrink
Win 10 Pro ($16): www.cdkeyoffer.com/cko/Moore10
Win 11 Pro ($22): www.cdkeyoffer.com/cko/Moore11
Win10 Home ($14): www.cdkeyoffer.com/cko/Moore10h
Office 2016 ($45): www.cdkeyoffer.com/cko/Moore16
Office 2019 ($53): www.cdkeyoffer.com/cko/Moore19
Get 10% off Vite Ramen w/ "brokensilicon" at the following link: bit.ly/3oyv4tR
Patreon: / mooreslawisdead
US Amazon Affiliate Link: amzn.to/3Xzf1eE
Address Fan Mail to:
Moore's Law Is Dead,
PO Box 60632
Nashville, TN 37206
Broken Silicon [A PC Hardware Podcast] is available on ALL platforms.
/ mooreslawisdead
www.stitcher.com/podcast/moor...
podcasts.apple.com/us/podcast...
play.google.com/music/listen?...
open.spotify.com/show/6cdnW7Y...
RSS: / sounds.rss
Twitter: / mooreslawisdead
All Music by SAHARA:
/ saharasuck
/ @saharasuck
Broken Silicon Logo by Tom.
Broken Silicon burning graphic by Tom & Sulphurous:
/ sulphurous11
2021 Video Intro by Tom & abstractrealism on the MLID Discord
LINKS!!!!!!:
$400 RX 6800: amzn.to/3uRsLqX
Main Domain (pygmalion.ai is not them): pygmalion.chat/
Discord with Active Devs: / discord
AI engine github: github.com/PygmalionAI/aphrod...
Guest’s github (very new): github.com/IsaiahGossner
Main github for the project: github.com/PygmalionAI
Their hugging face, where actual models are stored: huggingface.co/PygmalionAI
www.amd.com/en/newsroom/press...
www.servethehome.com/wp-conte...
www.servethehome.com/wp-conte...
Bryan Heemskerk AI Episode: • Optimizing RDNA 3, Int...
• Intel going Fabless, G...
en.wikipedia.org/wiki/Von_Neu...
www.trendhunter.com/trends/pl... - Ігри
[SPON: Use ''brokensilicon30'' for $30 OFF $500+ Flexispot Orders: bit.ly/3RcyPla ]
[SPON: “brokensilicon” at CDKeyOffer Black Friday: www.cdkeyoffer.com/cko/Moore10 ]
[SPON: Get 10% off Tasty Vite Ramen with code BROKENSILICON: bit.ly/3wKx6v1 ]
#blackfriday #windows11
can't wait to have AI hallucinate plot points and gaslight me in a video game
🤣
still better than what most AAA publishers churn out these days tbh
The first time I ever got to chat with a character powered by AI they walked "away from the road into the forest" and they AI kept commenting on how abandoned the road was after that. Good thing local models have come quite far, since!
It'll be fascinating to see what bizarre inputs and results the speedrunning community will come up with
Really LOVED to hear about AI from the guests point of view. Many interesting points brought up got me thinking.
Would love to see him come back sometime in the future to hear about how AI has changed.
Spread the word!
02:13:00 And they were right. One of the biggest problems with the AI craze is that it's called "AI", when it's actually just ML - machine learning. Calling it "AI" is not accurate, but that's the buzzword that has caught on.
completely agreed. Artificial intelligence in its basic state is something that does not need a prompt... it can just act on its own volition, which current machine learning doesn't really do. On top of just plainly 'not thinking' but that is a deeper rabbit hole...
Hello everyone, I'm the guest that showed up on this episode, and I was a bit nervous to be talking on the podcast, but if anyone had any questions that I didn't get to, feel free to throw them here and I'll try and get to them when I can.
I didn't get to answer fully what laptop I would buy personally, or recommend for that matter, but I think Strix Point is a great middle ground for getting good AI performance, future proofing, and the like, without waiting, like, three years to upgrade. Certainly, if I was buying less than that I wouldn't pay a lot, personally.
Also worth noting that I calculated out an AGX Orin to be around the same as a 3050/3050TI in performance at one point, and that thing has almost 300 TOPs of performance, so for really big models that's a huge reason I stressed that we might need to make use of more than *just* the neural engine in upcoming APUs, because I think that alone won't be a great experience for everyone out there. Then again, some people don't mind 1080p gaming, so some people might not mind the smaller faster models that are very responsive.
Hey just a quick question. E cores on intel are slow etc. How bad are they for AI? If you had 16 E cores only doing ai when a game is running would it be useful or just a waste of time?
@@skilletpan5674 That's a great question and I meant to get into it in the podcast, but we already had so much other stuff to talk about that I literally forgot to mention it.
In short CPU cores are a pretty great way to run smaller AI models, even little ones.
To expand on that a bit, CPUs aren't as well suited for AI as such but they're a lot better than you think, and my Ryzen 5900X was able to run a 13B (4bit) model pretty well, and that's kind of in the area of what I would want for AI in games.
My 5900X in that case was memory limited using around 10 cores, so I think even 8 little cores would be able to run AI fast enough to generate responses in slow scenes (ie: You're walking on a mountain and your companion is taking some time to think about stuff), and you might even be memory bottlenecked at that point anyway.
I certainly think 16 or 32 little cores on a future platform would be kind of a cool way to use little cores in gaming (because everyone's asking for little cores to be used all the time), and they don't have a lot of the issues that come with using little cores in other applications. I think 8 little cores would feel like, 3 - 9 TOPs if used in a video game.
When do you think AI will be able to just generate the entire game for you from a prompt? Like MidJourney or something, you just prompt it and it makes an entire game. Of course will be gimmicky first, but what about a real temporally stable game that could have been made by a real human team. BTW you were a great guest!
@@IsaiahGossner yeah thanks for the reply. So in the future we might see that in games. It's encouraging to hear that E cores might have a real usecase in the future. They could be used for non real time ai tasks as well like evolving storylines for example (AI dungeon master) or other game balance areas.
Anyway i can see ai being usful for stuff like 'make me 100 generic NPCs from the victorian era'. Artists might be angry but this is the same thing as happenend in the 1700s and 1800s with machines. No one seemed to really care when cars became cheap and easy to buy. The same will happen with AI.
I just hope we can get more affordable cards in the next few years. AMD is in the right direction with more RAM though.
@@MermaidTyrone Thanks very much for the question and the kind words!
text-to-video-game is kind of a category already, but it's mostly for simple games that could be found in something like an arcade cabinet 40 years ago. Space Invaders is an alright example.
One of the issues is that game design is multi-discipline, performance intensive, and has fairly large interconnected codebases.
Each of these can be solved in time, but today's models are generally mono-modal (just text, or just images), and don't handle complex projects fairly well (code with multiple files, etc). People are working on it, but today's models problem can't handle it, and probably not for a long time. I think you'll see prompt-to-video-game for simple 2D games in a real way perhaps in 2027 or so, but I wouldn't really expect all the tooling and the like to be ready before 2030 (text to 3d model, advanced agents, etc), but then I wouldn't have expected the performance of GPT 4 5 years down the line in 2017, so I suppose we'll have to wait and see!
Do keep in mind that even if it's possible it might not be practical (ie: The AI does literally do what you ask it to, but not what you want it to because it's hard to fully convey what you want the end game to be).
With that said, even today's models can provide a lot of assistance in learning to program and handle a variety of assets, so it's possible that an AI could help you get up to speed in game development in as little as a year or two if you wanted to start today!
I doubt it's even deep learning but on 2023 LG oled tvs there's an "ai visual setting" where you select 2/6 pictures you like and at the end the tv sets itself up according to what you like this took way less time than my usual manual tweaking and was spot on max sharpness and bright picture
I have some settings on LG TV about AI sound and it works pretty good in terms of balancing talking vs sounds effects (explosions, music etc) when it's often unbalanced in movies by default
There's a lot of products out there with "AI" on the name that are a lot of the non-machine learning things we've been doing for a few years now. It's definitely a trend at the moment to put "machine learning" and "AI" into everything, but I'm glad that regardless of the internals that the feature's working for you!
I think my TV from… 2015 or something… did that already. Just wasnt called AI Feature.
A long time ago, Microsoft claimed the cloud was going to be a big factor in advancing games, with Microsoft having an advantage as a leader in the field. Didn't really happen, as games that really utilize the cloud for single or few player campaigns are very few and far between. AI, with it's voracious appetite for memory and processing, could be where the cloud and gaming really connect.
The biggest mistake a company can make is protecting their existing product rather than innovating. Another company is always willing to do that innovation and grab market from you. A classic example is RCA and the creation of LCD display technology. It was perceived by a VP as a threat to their very successful CRT TV business and buried until the patents expired. Then, smarter companies took the tech and brought it to myriad products, improving until it became the standard for the majority of displays. RCA is now the answer to trivia questions with little influence on any product space today, after they squandered many years of R&D to potentially bring LCD to commercial viability much sooner and reap massive revenue.
I agree with you. I think AI is going to enable the cloud.
The most Canadian guest to ever Canadian.
Thought he was an AI for much of that
@@5lipperysausage No promises about if I'm an AI or not, eh?
@IsaiahGossner still not sure 😂. Great interview dude!
it dont matter if PlayStation 6 can run 8k, 400fps
if it has always online drm , that sony wants to do its a useless paper weight
Well, in a word, at Pygmalion AI that's kind of what we're doing. We want you to be able to run these AI models locally, and we've built out a lot of software to help people do that, and we have no intentions of stopping now. Part of the philosophy of a lot of the team is that you really should be able to own these models and use them personally on your own device no matter the circumstances, even if you don't have a connection to the internet. As a lot of people run more of their life with AI, it's going to be more important to have control of things like your chat history, and what you're using these AI models for.
@IsaiahGossner can the AI trick Sony servers into thinking I'm always online?
As I listen to your guest suggesting Intel could sell a card with 32GB to Ai developers, I'm like "Welp, it's too fucking bad Intel shutdown their Optane division because OMG could imagine a GPU with a massive onboard Optane cache?" Sounds like a game charger to me.
Not only games Ai will change game development.
GTA 6 must use a bunch of AI just for realistic twerking from the sugar babies for the trailer alone.😂
@@tringuyen7519I need to watch it, but I’m scared 😱 😂
I don't see the a.i. revolution gaining real speed as long as companies try to vendor lock in their customers by their proprietary apis. I think we may see a lot of fragmentation in the a.i. market without acceptable standards.
And i think we may see Game Streaming services gaining bigger share, because A.I. may need lots of data. Depending on the a.i. model it may be to big for a game download at some point in future.
@@aladdin8623 That is a risk, I have seen a lot of impressive open source ai so I think this will help with that problem. A recent 7 billion parameter mistral llm was able to perform better than gpt 3.5 on many tests while being much smaller.
AMDs phoenix ai support sucks, still no drivers for the ai-module, and the rocm 'hsa-overwrite' for the igpu is unstable (even though its nice when it works).
Seems to me most major AI stuff would still run in the cloud and not locally. I'm curious what types of things developers are thinking about offloading locally. I guess I could see local photo library analysis (facial recognition, tagging, etc). Maybe better for background blurring, noise suppression, etc as well.
It looks like UA-cam may have eaten my first comment, so I apologize if this is a repost, but I'm the guest who showed up in this episode!
It's worth noting that 7B and 13B models actually aren't *that* hard to run locally, especially with future computers (like the ones we might see in...Oh...I don't know, 2025), and they won't even take that long to answer, really, so it might be that there's a lot of local uses of these models which might be considered "abusive" of an API, or a website, for instance. Things that come to mind are annotating your whole 2000 entry long photo collection, or having an AI model go searching through them for an individual detail. Writing assistants for really long stories, or note taking models that say, write down a whole meeting (or your friends DnD session) and so on are also places that are very processing intensive and you don't want it to cut out at the most important part because you went over your API limit.
Beyond that, there's reasons a person might want a local assistant that isn't in the cloud. What do you do when the internet goes out? There's been plenty of times I couldn't connect to ChatGPT and so on because their resources are split pretty thin at the moment, and having that reliability of always being able to contact your assistant is a pretty big deal, especially to the right person.
There's also a lot of concerns with data privacy that might lead to people running models locally, as well. You might want to be able to have a model that can search through your financial documents quickly when you're busy, for instance, and you might not want those leaking if they're in your OpenAI account.
A major topic we didn't even cover on the podcast is that some of the best translation apps going forward won't be translation apps, they'll be assistants trained in both languages, with an ability to dynamically translate the meaning of important types (not just the individual words) and being able to run those constantly might be a big deal in the right situation. Or it just might be that you want to watch a move before the official English subtitles come out.
@@IsaiahGossner It's definitely interesting. I have an M1 Max 16" macbook that I thought i'd keep like 7yrs, but big AI changes may change that plan depending what comes out. Just watching for now.
@@neilquinn sadly Apple is still very conservative when it comes to AI, even their recently released Apple MLX is just a waste why would they want to make MLX instead of working with Pytorch is beyond me
@@Eleganttf2 Ya seems like it. Maybe will change. My daily driver is the 16" M1 Max but I do have a mini PC with the 7840HS right now. Will watch for AI updates soon.
training happens in the cloud, application may happen locally
You went out with a bang, in a good way. I hope to see your guest again in 2024.
Outside of making NPC's and world's more interactive, I'm curious what the gaming applications will be exactly. Mind you, I fully suspect we'll be finding new applications for years, once we start using AI in gaming in earnest. That doesn't mean I can't start thinking about it now though!
There is actually an entire graphics pipeline that uses machine learning for the entire process. There is no rasterization at all. Nvidia is working on this. They see it as the future.
The AI just makes the game for you, you will prompt it with something "Espionage shooter, 1960's", and then it will make it for you.
Well, it's hard to say. There probably are some aspects of machine learning that could be used for things like graphics and such, but I really do want to stress that more dynamic game worlds will really change how you experience games. I guess it's the difference between having experienced what it's like to use AI and feel like you have no limits on what you can do, versus looking in on it from the outside, but I really do think that alone will be revolutionary.
With that said, here's a few things that come to mind:
Better animations. The Unreal devs brought this up with the Unreal Engine 5 demo, but basically as you improve graphics poor animations are more noticable, so having really dynamic animations that can really express what your character is feeling in the moment could be a really subtle and immersive benefit of generative AI / modern AI pipelines.
Better collision detection. I'm not 100% sure about this, but I think somebody is probably going to try an AI driven approach to this some time in the next two to three years, and it may turn out better than current ones. Have you seen a cutscene in a game where it just ends up being super immersion breaking when two characters try to touch hands or something and it doesn't fully connect? I'm pretty confident even a fairly simple neural network could solve that, or if devs even want to implement that in their engines, but it could be possible!
Transparency. I think it's possible to add transparency to an image with AI instead of traditional rendering, which might be preferable to doing that with raw GPU (it's actually been mentioned a few times on Broken Silicon but that's extremely expensive on raw GPU), and it could work similarly to how fluid motion frames / DLSS 3 literally generate completely new frames, I think you could post process an image in that way with AI. Not sure if Nvidia or AMD will actually do it though, but it would be interesting.
@@IsaiahGossner
I would fully expect a more dynamic experience would be the main benefit. I mean, people talk about rendering games with AI and all, but while I don't know much about that, I can't see too many studios being eager to abandon all the tools and engines they've built up for rasterization. Especially for a 1st gen AI capable console.
But yeah, better animation and collision detection I can definitely see. If I recall, there's also that recent AMD patent about using machine learning to aid in ray-tracing reconstruction. Again, I don't know too much about that except in the abstract, but stuff like that is probably something they can use that part of the APU for, when it isn't busy making games more dynamic. (You don't want to leave too many resources sitting untapped for long periods, after all.)
i love watching Moore's Law flip flop
I'm pretty sure that Qualcomm is using directML. Plus Qualcomm has been in pc's and had an npu since the first surface X.
This was one of the most interesting shows I've ever watched. AI is really going to change the landscape of the hardware world. Seems like RAM and bandwidth is going to be really important going forward.
AI is going to replace the majority of humans.where will those humans go? Chemtrails.. COVID.. byebye. Hello robots that need nothing and take care of the elite.
Agenda 2030.
This comment is going to get deleted isn't it...
Ai in general will also be literally the downfall of us all!
@@BamaMike7187 that's what I'm saying. Also.. chem trail, floride, low quality food, covid and on and on are the tools of depopulation of the elite. They're literally killing us off. Open your mind and start hoarding food, water and guns or you'll be taken advantage of when shit goes down. I can't put into words just how serious I am. As AI take our jobs.. we will be killed off so the economy will continue with robots and automation and the rich own the world. The leaders of the world are so corrupt and evil. 😭
The 2 ends of the spectrum in these 2 comments
Anyone remotely sceptical of AI: AIs will ruin us all!
Me, abusing ChatGPT for spitting out very, VERY autistic things like Genshin fanfics, talking about my feelings toward Albedo or AI Joken, or pasting in IRC chatlogs: haha 'Wacky Alchemy Adventures' go brr
1:56:14 About Intel doing a 32 GB card, AMD's strategy seems to be to release just enough to spoil all Intel's launches (wich makes sense since they are competing with a huge complany)
This was really interesting to listen from someone who got not much idea about ai yet
I’m wondering if a dedicated NPU will be included for AMD’s upcoming desktop CPU architecture or if this will be limited to APUs?
One.can only hope 🙏
May or may not be. This channel hasn't leaked any, specifically, but it's worth noting a big reason you want an NPU isn't necessarily for performance. If you look at Hawk Point I think they hit 39 TOPs or so, but 26 of that was the iGPU I'm pretty sure. They do it on mobile SOCs because it uses very little power and gives you a lot of performance quickly, but the issue is that only goes so far in something like a CPU where you're usually memory bottlenecked anyway.
Usually on a desktop CPU you'll have more cores (and an iGPU you're generally not using for gaming), and the cores will be clocked faster. In my opinion, and from what I've seen of other people running locally you probably won't be limited by an AI application on desktop unless a developer doesn't optimize well for the CPU and so on. The number one thing you can do is pay a lot more attention to the memory in the system you're building when it comes to AI applications because they really do need the bandwidth (I think AM5 maxes out at 128GB/s with two channels of DDR5 8000 but I might be misremembering. A lot of GPUs are around 1TB/s, so...)
@@IsaiahGossner Thank you for your reply and your time Isaiah! I'm very much a novice and just starting to learn about AI, machine learning, and some of its applications. I have mostly heard about AI applications using specifically GPUs, like the Nvidia GPUs that use Turing Cores or AMD Radeon's application with AI accelerators that seem to use more multipurpose hardware. I'd only recently heard about NPU's and dedicated AI processors , and mostly in the context of mobile APU/SOCs. Thank you for helping to clarify!
I really enjoyed this episode! You were a great guest!
@@J_Erik yes they do uses those Nvidia's Tensor cores as for AMD only recently did they implemented an AI accelerator in RDNA3 which is the WMMA instruction which supports fp16 and fp8 but even then its still much inferior compared to Nvidia's tensor cores but i guess its still the right small steps for AMD
If I remember correctly there was an AI chiplet, but I think that was servers. I would think it's either a part of the IO die on desktop or a separate chiplet on all CPUs.
Great guest and interesting listening.
Man I just bought the flexi spot chair a week and a half ago. Wish I knew you were going to get sponsored by them. Most comfortable chair I've ever sat in.
Try emailing their support. If you're in the return window (if there is one) they might help you out.
Another great show. Thanks!
I appear to be outside of the loop, what is atops?
Great podcast as well as the guest.I would like to listen to him more than once.Thanks Tom :)
Good interview. Got me thinking about how these NPUs/AI engines which are just marketing fluff right now can actually be applied going forward.
I tend to lean more towards the conservative end of things where it won't be until 2026-27 before gamers even need to think about factoring in these engines into their desktop upgrades, but I'm happy to be wrong there.
To me, these NPUs are like raytracing during the RTX 20 series. There's not really going to be much available on that hardware generation but it's important to seed an early version of the technology to developers. Significant AI stuff will come into its own in the next generation or two.
I'd lean into longer than that. AI stuff is more complex than ray tracing and is not nearly as universally applicable to games. This will slow development and adoption significantly in comparison to what is already a slow adoption of ray tracing (inspite of industry efforts). This doesn't even consider the growing crowd of people who actively REJECT AI development and implementation, which in sufficient numbers can throw a lot of wrenches.
Using AI for 'nocode' development sounds a lot to me like the 'nosql' fad where we ended up with a bunch of projects made by people who had no clue about data structures or maintainability and frankly had no business pretending they were devs
Thanks!
Thank you for your support 🙏.
Happy Holidays! 🌲
1:37:05 Me too. I just bought a 3080 10G instead of the 12G model because I couldn't spend so much more for the same card only w/ 2 more gigs of vram while the Ti model has a really bad perf/cost ratio.
How much you paid for it? I got a used 3090 for $830 even though I saw them for 750. I wanted a certain model.
why does this guest sound so much like huggbees
Mind blown at 2:32:00
A thing you guys missed is that smartphones already beat PCs in NPU performance.. Snapdragon 865 has 15 TOPS (2020), SD888 26 TOPS, SD 8 Gen 1 52 TOPS (2022) while Apple's A14 has 11 TOPS (2020), A15 & A16 have 16 TOPS and A17 has 35 TOPS. I think Qualcomm is well suited for PC AI because they have all these developers already using their software on android of course depending on how that translates to windows.
TOPS literally means Trillions of Operations Per Second. Nothing more.
Those operations are usually multiply and addition (needed for matrix multiplication), but the precision and format varies wildly from vendor to vendor. Some state 4-bit integer operations, others state 16-bit Floating Point operations.
I believe the Apple A-17 included an 8-bit block-floating-point mode which runs at double the 16-bit speed of previous generations (hence the > 2x increase in TOPS and why it apparently beats the M3)
Another thing they talk about with these Neural Processing Units is that most of them are starved for memory bandwidth.
For instance, the SD 8 Gen 1 has a memory bandwidth of 50 GB/s to feed 52 TOPS. Qualcomm doesn't state what precision that peak number represents which usually means it's 8-bit integer or lower.
Let's say it's 8-bit integer. That means it has to perform at least 1,000 operations on every operand it loads otherwise it's bottlenecked by the memory bandwidth. (Assuming no other systems are using the memory like the CPU and GPU)
The M3 has 100 GB/s of memory bandwidth and does 18 TOPS of 16-bit Floating Point. That means it has to perform about 350 operations on every 16-bit operand it loads before memory becomes the limiting factor.
The other problem is software. Android makes lousy use of GPUs for compute. I doubt the NPU utilization is a much better story.
Sadly, while these NPUs are extremely impressive on paper; they're most likely so-called "dark silicon" on most chips. AFAIK: the software support is almost non-existent.
@@AbeDillon TOPS means tera operations per second. Or do you think GFLOPS is for gillion floating point operations per second?
About to recline into my chair like the ad and listen to this. Have a great day everyone.
25:10 - how would the memory bottleneck be influenced by cache size? Would v-cache chips have a real benefit here? Might big L4 caches be making a comeback to boost ai performance?
I'm far from an expert on the latest in AI software, but from what I know I'd say no, unlikely that caches will help much. Because caches mostly help with somewhat random access, in particular if you have smaller chunks of hot data that is accessed often and some stuff that only gets fetched when needed. That is true for a lot of software and games in particular, but AI calculations mostly involve blasting through a lot of data fairly linearly. It really comes done to how fast you can feed a ton of data, not how quickly you can access parts of it. I.e bandwidth is the main limits, not access latency. Caches mostly help with the latter.
Cache doesn't help much atm. Currently, batch=1 inference has to read through basically all the memory for each layer, so you're literally maxing out/limited by your total memory bandwidth to do a pass.
@@lhl Ah okay, thanks c:
AMD really needs to catch up in the AI space with ROCM. It's really annoying to see NVIDIA have all the best features.
Intel doing great with their open framework (oneAPI) receiving lots of support since devs have been wishing for an open CUDA alternative .
They recently did with rocm6 over rocm 5 by a factor of 8 times being faster. Watch the latest ai presentation of amd.
Being a copycat will never gets you very far
Cuda is a proprietary api to vendor lock in customers just as directx does. Nothing more and nothing less
Thanks for asking the last question, I've been concerned about that too as it's not just programming that could potentially suffer. It's IT support as well, if people don't have a knowledge base of not only the fundamentals but the odd qwerks of software and work arounds. How are they supposed to support others let alone teach them?
Also without knowing the code basics, how do people distinguish malicious code?
I don't want to think about what would happen if a nefarious individual or group got into the servers of AI generative services that are being utlised by people to get a quick short cut on their workflows.
on AI responding to speach in a game, it's already happening with Skyrim modding check, GingasVR on youtube. pretty much any vid that has "AI" in the title is her playing with how chatGPT(I think) responds to her questions. I would add a link but more often than not YT will not post the comment.
if you want a specific vid check "I had a REAL Ai NPC Control my LIFE in VR" on her channel.
I also wish Google's Tensor can be as competitive in AI at least like the Snapdragon too.
Awesome conversation! And since i share isaiahs perspective a very reassuring one 🙏
I've said before, on a different channel, when I think about all the advancements in the AI field, all I can think about is a very large man with a deep Austrian accent asking, " Sarah Conner ? "
Cool video. I wonder if, for a demonstration of capabilities, your guest is actually ai? Well, if not, it could happen. Watch out for that...
No promises :P
One thing about Intel's cards is that they're in a similar position as Vega was. They have a lot more compute than their relatively weak gaming would indicate. They no doubt make a much better compute/ML card than gaming card. Put another way, the non-gaming performance tier of those cards is notably higher than their gaming performance tier.
Yeah but Vega successfully completed with the 1080, and the liquid cooled model (though a space heater) actually touched the 1080 Ti in a few games. Furthermore, Low Power Vega products were REALLY efficient in laptops. So overall Vega was great at compute, efficient at lower clocks, and ok at gaming efficiency until you pushed it super hard.
ARC isn't like that at all. It wastes die space, is always innefficient, and struggles to compete with products vastly cheaper to manufacture than it. I see the comparison to Vega, but it really is a different circumstance.
I think that zen5 may end up having an NPU.
This guest sounds just like Brett from PC Perspective.
This whole conversion of games using AI hardware, seems to ignore how games continue to be developed year after year. We get 1-2 major games every couple of years, that actually make an effort to target PC primarily. Never mind targeting cutting edge features. Cyperpunk is like the most prominent RTX showcase game, and the only other "big" game that tried to make it a big deal was Metro. That really came down to Nvidia just paying them to help push RTX GPUs. Gamers have the option of just not using it. Most devs are going to ignore the AI stuff until it's in the consoles. Especially if it going to be an important element to fleshing out the actual game play. I'd be amazed if we see it used inside of 6 years in a meaningful capacity.
Thanks for you'r incredible podcasts and information!! really helpful
Sounds like ai will need a special processor IC block like the one in AMD apu. Leaving the gpu to do graphic is the right move? Next generation consoles will have ai processors ic block. With 3D stacking ai, graphic are limitless!
I wish AMD Makes ROCM available as a flatpak Install, All the stable linux distros have it , Snap is viable but many people in the linux community don't like using it
Have it as a flatpak and it'll be an AMAZING oneclick install experience
Imagine a random NPC crying like Haley Joel Osment in the abandonment scene in A. I., begging you not to turn off the computer you're playing on because she's aware of herself and doesn't want to be erased
Turn off the game, uninstall, get a refund, leave a negative review about making such jokes. And if the devs actually responded and said "we didn't... program that in," you know you made the right call. 😬
2:11:08 they had arc lighting back in the day... messed up peoples eyes... they would have known what it was.
The SkyrimVR AI mods are great
Im hearing that Meteor Lake is going to be a big upgrade
Yeah, upgrading the crater sized holes in Intel's financials 🤣. Joking aside, objectively speaking for Intel's sake it better be.
it ended up not being that (compared to phoenix).
@VideogamesAsArt it wasn't suppose to be. It is a slight upgrade from Phoenix in the GPU.
then what kind of big upgrade did you mean in your original comment?@@ehenningsen
@VideogamesAsArt from Iris XE. Meteor Lake's iGPU already beats Phoenix and future APUs will be much better as Intel is quickly shrinking die
That's not to say that AMD will be beaten. I had to qualify this as we all know how AMD fanbois are
Here’s a crazy idea, minus path tracing how about maximising the potential of a 3060ti before we starting thinking about anything else
This looks like it will be informative, thanks for finding such diverse guests!!
Interesting point about the physics engine and how for decades now we think nothing of a corpse spontaneously ragdoll breakdancing or a car doing a somersault off some tiny pebble, guess a lot of game devs will have to write in plot components about schizoid tourettes syndrome game characters
1:18:30
That's cheaper than RX 7600 XT here where I live
1:18:10 I think an Apple Silicon Mac might be a better choice than Zen5 for running CPU based inference. Their SOC memory is blazing fast.
That is a good point; the M3 Pro chips have 150GB/s of bandwidth if I remember correctly, whereas Zen 5 with quite fast memory should cap out at 128GB/s. Plus Apple has fairly good support for their AI accelerators and a lot of developers like using Apple products for running AI.
With that said this was a podcast about gaming hardware and PC gamers, so while I feel a bit silly for forgetting to mention that, a lot of people are looking for advice on how to build a gaming rig that won't be made obsolete by AI. But it's also good to remember that depending on the application it may be able to make use of CPU *and* GPU which may throw the balance off, as many GPUs can have around 1 TB/s of bandwidth.
good stuff.
My biggest terror with AI is its ability to impact the economy - by mopping up many lower-skill jobs that automation and current software had not quite reached yet. I feel like this concept wasn't touched upon quite enough. It seemed like it was kept within the confines of that one example (illustrators and art).
I don't really need this guy to lobby against the bigger concerns but I'm surprised that the conversation didn't go there. AI scares the shit out of me for that reason. Forget customer service, editors, social media managers, writers of any kind, entry-level programmers ect ect. And I just don't know how far that will go. It will erase jobs while only creating a few technically skilled positions.
Bytes are way easier to move around than physical objects are. For all the promising reliable hardware is not cheap. That fact keeps biting a lot of business managers who believe automation will replace their pesky employees in the ass.
Basic flowchart logic also seems to be very hard for deep learning to grasp. Largely because flowcharts come from a top down design methodology while all the current AI stuff are designed around bottom up brute force.
Intel has the MAX series of professional cards! Also, Arc pro.
That guy was very inspirational to me. Going to learn AI/Python. I have an understanding of Python. Haven't felt this motivate or inspired in awhile.
Hey there! I'm the guest that showed up on this episode, and I've seen a few people get started with AI recently. If you start now you can probably be building a full AI app in two to six months depending on how far you want to go. If you're more interested in the training / fine tuning side of things there's a free online course called FastAI that a lot of our devs started with internally, but if you're more into building AI apps you may want to do some basic prompting in Python / a Python notebook. Wes Roth is a great channel with a lot of details on how to run some of this stuff yourself.
Once you learn the basic format for prompting models the sky really is the limit right now and there's a lot you can do to embed these models into really useful pieces of software that people might start using every day in the next year or so.
Could you link yours github?, i can't find it@@IsaiahGossner
@@IsaiahGossner Thanks for the reply back. Lots of info shared between you and Tom. Full AI app in 2-6 months time frame seems reasonable. You mentioned FastAI. I researched it an came across a website I assume you're talking about. TONS of info course. I'll also take the time to learn basic format for prompting models mentioned as well. Where can I go once learning just what we mentioned thus far? Building Full AI API, Jupyter Notebook, Basic Format Prompting Models or just focus on those things for now? Thanks for your time and reply.
This guy has such a nice voice
1:43:54 ROCm is believed to be a volunteer effort? No wonder support is growing as slowly as it is.
2:35:36 I wonder if the solution is to give artists individual models they train on their own art. That way, they'd still learn the skills, but it would still help them once they're there.
While I don't think it was a really "volunteer" effort, ROCm has finally been the #1 priority this year - it was previously severely under-resourced. The main thing though is that w/ the CDNA/RDNA split, AMD decided to basically abandon RDNA support *and* also stop all support for legacy architectures. Compare to Nvidia, which has support for all their GPUs (most importantly, including all consumer and mobile GPUs going back to the GeForce GT 430 (2010)) and you can see why ROCm has been completely uncompetitive.
A.I won't ever just generate anything, just like humans it will just hold all this early data from others in its deep storage and wont need to see actual data sets because it will have all the baseline data it needs, but that original data has to be memorized to even know anything, it can't magically pull solutions out of thin air.
ROCm has 477 open issues non resolved and latest 6.0 instruction doesn’t even work, lol
Skyrim already has AI modded in it and it isn't "terrible".
Just sayin it might be closer than you would think.
AMD needs to get their ROCm backend for TensorFlow upstreamed.
I want ai in more software like using simulation for things like cancer research such as folding home boinc network traffic assist software etc. I want there to be integrate chat gpts to download that help windows or help the my computer research things! Ai is amazing! Also allow us to overclock the ai tops engine would be nice! Why not just combine the ai engine with the cloud engine... Also why not use people ai processor to pool processing for a game server we join so we have more combined tops to process things...
Still only partway through but this already has me thinking about Deus Ex and the concept of "emergent narrative" and gameplay. I wonder if we'll see a reemergence of the immersive sim since AI would open that genre up in a way that was never possible before. Or RPGs like Baldur's Gate 3 would be possible without needing to write a thousand different outcomes to match any possible player decision.
Deus Ex with AI-generated NPCs would be pretty batshit.
I'm actually really looking forward to a lot of the things we could see with AI driven emergent narratives. Imagine in Skyrim if it was possible that the war would literally happen in the backround and you could actually change the outcomes of real battles that were happening, or if you could break into someone's home in Cyberpunk and get their full life story out of them at gunpoint, and so on.
Some of that stuff is further, while some of those basic implementations will actually come faster than you think, and I really hope we're able to be the ones implementing them!
AI can't generate anything new and creative. It's takes the middle average of all gathered data.
I use an AI in WOW that simply gives voices to all quest givers
In terms of using AI in games, it really depends on what he means by it. You might want to define that alot more before discussing it.
For example, in terms of NPC dialogue and maybe even questing, we see some of that already, and it could ramp up next year. Since this is LLM based.
But in terms of producing 3d assets and systems, his 2027 timeframe sounds more like it. Although I wouldn't rule out 2026 if the rumors from OpenAI checks out on every box. Since this required more math.
Finally, I think you both might be underestimating the media part of Meteor Lake. There is alot of users out there who mainly use their laptop for running video. Then again, if OEMs don't want to use Meteor Lake, the point is moot. But I think more will than what is disclosed already.
If you think AI generated 3D assets will be in games by 2027, I have a bridge to sell you. We would need to have proof of concepts of that NOW for games to have it that soon. Game development takes time and they aren't developing for things that don't yet exist. If anything they develop for things that have been out for 5 years as it increases their market cap.
@@ReivecSDo you really think game development will take 5 years once AI does 98% of the work?
He said only the RX 7000 series with enough VRAM are supported for ROCm, but even a 6900 XT works for running stable diffusion locally (Automatic1111). Xformers will not work with AMD and low VRAM-mode is recommended.
I ran stable difusion on a RX 6500 XT 4GB back in january this year, but it was a pain in the ass to set everything to work using ROCm and Linux OS, too much bugs and settings that I had to manually fix... I was also able to run some LLMs on that card few months later. Now i have a Nvidia card, things are much easier to install and use when it comes to AI.
@@MCA0090Out of curiosity, was the unnamed Nvidia card you went with as kneecapped as the 6500 XT?
@@benjaminoechsli1941 No, lol... But still, I remember to see people with beter AMD cards (RX 6700 and RX 6800) having the same problems as I had to make things work with ROCm, the problem wasn't with the RX 6500 itself, the problem was AMD software...
Sorry, just to clarify, I wasn't trying to say that they were the only ones that ran, but they were the ones that came to mind off the top of my head, and I was pretty confident they would be really good for AI going forward. I was trying to recommend to the average person who might be future proofing for AI going forward, and wanted to get on it early. I think a lot of old cards will definitely get you through more than you think, though!
ROCm is still not in a great state, for consumer prosumer level tinkering CUDA is still king. My 7900XT with 20GB goes OOM and crashes a lot as it won’t reclaim memory properly. Olive has the same problem.
The big cloud providers are rolling their own Silicon focussed entirely on MATMUL operations - if you need a certain scale of operation then rather than spend millions on nvidia hardware you’ll just hire some specialist engineers and make it run on actuall dedicated hardware.
We’re on the same trajectory as Bitcoin mining, it works on GPUs currently but Azure/GCP/AWS are going to eat NVidias lunch. CUDA is sticky but it ain’t 10 million dollars of CapEx sticky.
I think PS5 will run AI on cloud. But PS5 pro will run AI natively on device because of new AI engines.
Persistence of character and narrative structure interacting with the player. I play with AI dungeon and these concepts are in a lot of the lit RPG books I read where they are asking what will future games and AI in them be like. Also the normally have brain interfaces.
Yay we should have memory and computation near and distributed.
Persistence of character's a tricky one. That's basically linked to a model's "context" which has to be trained into it. Various models will have different levels of context (number of words that they can remember), and still recall information from earlier in the conversation. We're actually looking at a few ways internally of extending this to some pretty impressive lengths (think around the size of a novel, if we get a crazy good solution working), but there's a lot of ways to do it, and I can only hope one of them works!
With regards to AI dungeon I don't remember which model is running it, but if it's GPT 2 I'm pretty sure most modern local models should be better able to recall things over a long period of time and we've implemented Rope scaling (or a similar technique) for extending certain models in Aphrodite-Engine if you wanted to play with it and had a Linux PC on hand. RWKV is also a really solid model that runs alright on CPU and doesn't take a ton of RAM, and has in theory infinite context (there's a few practical issues with the earlier versions, though).
a healthy food with lots of Glutamat 🤣
1:21:30 AMD has compute in memory?!!! I didn't realize that. You need to get Stacy Rasgon back on the show to talk about stuff like this and what's holding back AMD from putting it in products
What an excellent, informative episode.
Great guest!
I think another major factor that Ai POTENTIALLY offers is the "democratization" of Game Development. You could develop a user friendly AI tool that can generate maps based on a prompt. Those prompts could potentially even be audio prompts. So you start the program, hook up your mic and then you open a development environment and tell the AI "generate a 4Km² RPG map with 3 small medieval style towns. Make 60% of it european forest and 40 of it a thinly forested landscape mainly dominated by meadows. Randomly add small rovers, ponds and waterfalls" and so on and so on and then BOOM, it generated it. You could make it so that areas/objects you select can be hand picked and changed like "Change this houses walls from stone to wood" and all that stuff. Basically everyone could craft their own games and they could even be decently sized. And I mean it has happened before. In the late 70s/early 80s you had to be a real computer expert to use a standard desktop PC. Now literally the greatest dumbass can carry around a smartphone and use it. Hell, even 3 year olds can use phones. It's not like such stuff has never happened before.
And with the increase in computational performance of AI hardware that we see every generation and the improvements in AI software this really isn't too far fetched. it could also massively speed up game development of big AAA titles and cut costs massively.
I feel like that's just going to degrade quality make games worthless. Just because you can make a half-assed game in 20 minutes doesn't mean we'll do anything good.
@@AndrewMellor-darkphoton You're making the assumption tho that there will constantly be half assed games and you also automatically assume that all of these games will indeed be ass. There's no reason for that tho. Sure there will be games that are ass. But that's also the case now and eventually things would sort themselves out. Like initially everyone might try to do it but as they see 99% of people fail MOST people will realize that you still have to put some thought into it.
I just find your comment masisvely negative. There's some truth in it but 95% I disagree.
AAA developers are preparing to fire all their artists because they thought AI art was better. Companies are starting to fire all their customer support people because AI can be trained to be psychopathic quicker. The problem isn't the technology is great The problem is the tech CEOs are psychopaths.
@@kaystephan2610 Part of the concern I think the first reply has is that if AI democratizes game development to the point you don't need any particular training or time investment or budget, then there's going to be an explosion of half-assed titles that define "success" as just making more than the electricity cost to run the GPU they were going to own anyway.
While there could be some interesting concepts that get made that otherwise wouldn't, finding them under the sea of AI-powered asset flips is gonna be a challenge.
And then there's the issue with art direction, or lack thereof. If you've got some kind of unique and interesting plot, but you're using the same AI toolkit as everyone else to build your world, your game is presumably going to look an awful lot like everyone else's. Sci-fi and fantasy in particular are known for very distinctive and memorable worlds that are a part of the narrative themselves, so that's gonna present a challenge (unless you've got the time and resources to generate tons of concept art to train your own AI). And that's assuming you're not fighting with issues of consistency and uncannyness in the generic world it generates...
Then there's mechanics. If you've got some new and novel mechanic for a puzzle game or shooter or something, the AI game engine isn't going to have much relevant training data to work with, which could be a problem. Or you've got a unique aesthetic where you made enough concept art to train your personal AI on, but the jumping/shooting/driving/etc. all feels just like every other game made on that AI toolkit.
Heavily, *heavily* AI generated games might work for an early concept test or something, but it's going to make it a lot, LOT easier for games with poor vision, art direction, design, script, QA, etc., to be shoved out the door quickly and cheaply. And when the market is beyond saturated, it's gonna make it a lot harder to convince people that your one-person passion project built in the same toolkit isn't just as worthless as all the others.
Historically humans have always pushed back against machines/AI replacing jobs but after a few years they stop and progress is made. You can’t stop the push for the future.
AI and machines don't buy stuff. Good luck having machines run/create everything but no one to consume anything. How's that going to work out for capitalism?
People giving up on art so the worst people in the world can make billions isn't progress.
@@chinogambino9375 tons of 70+ year olds would say the exact same thing but we wouldn’t even blink an eye to them. You really think our grandkids are gonna give a shit about AI??? It’s unfortunate but it’s the future adapt or get pushed to the past just like them.
@@Entertainment-ev6ob There's nothing to adapt to. You seem to think just existing is an accomplishment.
@@chinogambino9375 ???
How did you two meet ?
the guest reached out to Tom in the discord
I've been watching MLiD for quite a while, and it was what really got me to start thinking about hardware back in maybe 2019 or so. I've been making career moves since, and now that I'm working in a field relevant to a lot of what goes on in the show, I thought it would be fun to contribute back, so I reached out and it just happened to be a really good time to have a podcast about AI hardware, lol.
@@IsaiahGossner Thanks for talking about ai hardware.
AI is "guessing and MOAR POWER". No technical knowledge, skill, honour, advancement...
Tom don’t you use ai for your thumbnails?
First of all Photoshop is 10$ a month , your guest actually didnt touch upon any ethical concerns of creatives (be it image art , voice acting , 3d art , animators ,writers etcetc) and kinda just told them to suck it , but then he said if theres reset because of copyright data in the training of the models that only big companies will be able to compete , do you know why he said that , he said it because theyre the only ones who could buy copyright data at scale to actually pay for the people whos work they are scraping, so hes saying , if we cant steal , we cant compete , kinda said you didnt catch that and push him on it. Anyway sad times coming for creatives without regulation.
As for him saying their roleplaying chatbot is mostly trained by users , abd that exempts them , well those users had to interact with a chatbot that was already trained on copyright material , because without it , theres nothing for users to interact with in the first place. So its still data gained from someone using algorithm trained on copyright material , its just been "laundered". While the inputs would be from users that consent to giving the data ,chatbots responses still come from Ai trained on copyright material you have no rights to monetize. And we come back to the first point , only big companies would be able to compete because theyre only ones that can pay for data , so unless you steal your business doesnt work. Great stuff ,0 pushback for some reason.
you can either just cry about it now and be out of a job later; or you can figure out how you're gonna adapt to the future so that you're not out of a job when the future eventually comes. no amount of crying has ever stopped the future from arriving. never has, never will.
@@EdR640 im not denying no ones stopping the future , AI can and SHOULD exist , just pay for copyright , its really simple
@@bingbong3084 as pointed out in the video - you need to realize instead of 'pay for copyright it's really that simple' they're just gonna use in house data sets to train on and cut you, the artist, and paying for the copyright, out of the equation entirely. It's really that simple.
Hey, I'm the guest that showed up on this episode, and I thought I'd chime in to add a few things on that I didn't quite get to in the podcast. To be honest I'm not a great public speaker and I was actually a bit nervous throughout the whole thing (listen to all the "uh"s at the start, lol), so there were some parts I sped through and didn't answer fully to a level that satisfied me, so I thought I'd add some stuff on, not necessarily to change anyone's mind but to clarify my position. I understand you probably won't like what I'm doing even with clarification, but I do hope that I can at least be informative and constructive for you.
1) Yeah, I literally didn't know off the top of my head what Photoshop costs. My bad.
2) It wasn't my intent to tell creatives to "suck it" I actually had a bigger point that I was building towards but I got lost because I had been working for around 9 hours that day before going on to the podcast which was three hours long. I'll expand a bit on that below.
3) Getting data if you're a startup is a bit tricky. I'm not saying "It's okay" but I am saying that it's really hard to start out, and especially when a lot of open source projects started during the silicon shortage and so on, nobody really had any idea what was going on and just kind of grabbed what they needed to get started because nobody knew how big it would get. There's a pretty big precedence for allowances for smaller startups (Unity and Unreal Engine are free for smaller startups) so I actually do think that especially for projects that haven't made any money yet asking them to front $100, $200, $300 thousand for data is a pretty big ask, for instance. Again, I'm not saying it's okay as it stands, but I do hopefully have something constructive to offer.
4) Specifically with regards to our roleplay logs, it gets pretty tricky. I'm not saying they're perfectly clean, or that they're not bad because we "laundered" information from another "dirty" AI or such. I brought it up in the interest of transparency, and data sources like that are going to be a tricky one for everyone to sort out. A lot of them are very human edited (meaning that not a lot of the AI's original response may even remain), and in language models specifically it's a bit tricky because they will adapt to the style of the person speaking to them to a great extent, meaning that after the first three to four responses they may look more like something written by the human interacting with them than the natural way they will appear when "prompted raw". Again, I'm not saying this is a "silver bullet" for getting a clean dataset, but I do think it's important for everyone to understand how this data was gathered for not just us, but future projects that may use similar strategies. I've been looking at a lot of ways we can get access to alternative forms of data, and it's possible our datasets may look completely different in a year (once we acquire some income and can make moves in this space. Again, we're basically operating on personal funds at the moment).
Anyway, those weren't my main points, just a few clarifications. With regards to artist attribution, there's actually a few different ways to figure out how much an AI model should re-compensate an artist, and I'm looking into this personally. There are attention maps that let you judge how much impact a specific token (ie: An artist's name, or a word associated with an author's style, for instance) has on the impact of the model, there's the % of the dataset made up of a given creative's data, and there's the loss when you show the model a given creative's work. I think a combination of these things can be used to get a rough idea of how much any individual artist contributed to an AI model, and I think you could do something with a website like HuggingFace (they host AI models), where they could measure metrics like these to get a rough idea of what an artist contributed to a given model.
With that in mind, PygmalionAI is kind of hoping to become an open source standard in the industry. I don't know if we'll be able to do it or not, but if we can get a lot of things going with AI applications, it's possible that'll give us some degree of weight when dealing with companies like HuggingFace and so on, and it's possible we could discuss an API that gives us a method to re-compensate individual artists that contributed to elements of our datasets. The issue with negotiating with artists right now (on an individual basis) is they tend not to understand how AI works very well, and they might ask for something that's not economically feasible if you ask them "Hey, could I use your art as part of my dataset? It'll make up 0.01% of it and not contribute strongly to the overall style of the model" you'll sometimes get back responses like "$40,000" or something because they think we're going to totally steal their artstyle and let people compete with them using Stable Diffusion. In the above example, imagine if you had a bunch of people who wanted to make a good open source, ethical model, but the people they talked to for data gave responses like that, and they had to just give up because it didn't work?
Another thing, is I didn't have the presence of mind to go into this in the podcast, but one of the reasons I got into math and machine-generated content is because of the movie Klaus, where they were able to animate the lighting pass with a computer assisted process. I can't remember the paper nowadays, but the math looked a lot like some of the math you see in machine learning, and it inspired me to build tools that could help out my creative friends. Unfortunately in the modern state of affairs, this has obviously started to feel a bit backwards from the outside.
I can't promise what PygmalionAI will do going forward, but personally as long as I'm in the space I'm going to be looking at ways that a compromise can be reached where artists can get compensated for their work, and AI developers in the open source community can compete with major corporations so that AI doesn't get limited to closed source models behind a subscription fee. I could totally see a future where artists don't have to deal with individual clients anymore (I've heard a lot of horror stories about artists getting screwed over because a client asked for a last minute change like, 20 times or something), and can pursue art that they're passionate about, while being paid a passive income based on their overall artstyle rather than any one individual piece, but this isn't the sort of thing a single guy at a small AI startup can do alone.
With that in mind, there's a few constructive questions that could be asked to creative people.
A) What would you charge for an individual art piece that didn't contribute to the overall feel of an AI model?
A1) What if that piece was now a central part of the style of it?
B) Have you looked at any features that you currently like in open source AI tools? Are there any you'd like to see? I talk with a lot of people in the industry quite regularly, and I can't completely change their direction, but I can definitely pass these things along, and I really do want art to be for everyone (not just AI bros, everyone includes artists).
Personally, where I stand, is that some sort of revenue share seems most amicable, where you could take the total revenue - training costs of the model, and pay artists some % based on the profit and their individual contribution, but again, that's a really hard sell when talking to artists on an individual scale, and it will depend on negotiations between sites like Hugging Face, Deviantart, Reddit, Artstation, and so on.
Regardless, again, I know I'm not convincing any creative people that "hey, AI is cool now!" or anything, but my point wasn't "artists can just suck it" and more "Hey, I want to work with you, but if artists don't talk to me about what they want and come in swinging without listening to what I say, my hands are tied and I can't do anything to help".
My personal commitment is that I will do what I can to push forward standards that let everyone profit. The weather's nice, and I think everyone can profit from this. If you read this, thank you. If you did not, I don't blame you, and for what it's worth, I understand, and will do my best to help out creatives where I can.
Damn, this dude is using like at least two likes in every sentence. It is really grating to listen to when you notice.
I really hope that is just a nervous tick and he does not sound like that IRL.
@2:19:00 Wow... that was a sorry excuse of a response! "Hey, sorry we basically took everyone's IP and used it to make crappy versions of what you people created. Hu-huh!
I guess I can understand that you are angry because your skillset is worthless because we lacked morals in our quest for our own money. It sucks to be you!"
It's basically, 1) I'm not sorry. 2) we don't consider what we're doing wrong because it makes us money and we have no morals... (ps, it wasn't me directly!)
Just.... horrid.
It's a bit silly listening to your anti Intel rethorics though. And clearly you both have invested money in AMD shares.
I could write you a book on this, but Nvidia is about to become the Intel of the past. Plus they don't have their own foundry. While AMD will always be the little 3rd brother who lacks software support.
Apple, Tesla and Microsoft now have their own AI chips. Soon we can add Google, Amazon and Meta to that list.
Then the real gold will be the foundry business. And AI robotics will be a booming new part of it's growth
My problems with ML are basically two-fold.
One is that it's a black box. When it makes mistakes, you can't figure out why. You can just try to adjust the training. You cannot prevent future mistakes, either - at best, you can prevent mistakes of the same kind as you've already encountered.
Second is that it's far too trusted. We've already seen people actually arrested based solely on incorrect ML-based facial recognition, with no attempt to investigate beforehand. Sure, you can create laws to prevent that specific thing (something which is far too slowly coming to pass), but that's just one symptom. People place an entirely unreasonable amount of trust in ML results. ML is going to give wrong answers that won't be caught for a very long time in a lot of situations. And as our reliance on and unwarranted trust of ML increases, the number of people with the training and experience required to spot ML mistakes will dwindle, exacerbating the problem further.
Actually, there are some ways to "figure out why" - there's the whole sub-field of mechanistic interpretability that's exploring the black box aspect (and making lots of interesting discoveries). Beyond that though, there are lots of real world techniques emerging like using constraints (eg CFGs), logit-biasing, self-consistency checks, etc.
I do agree that some people are entirely too trusting of ML systems. Often it's a case of garbage in garbage out on the data front, but at the most basic level, it's important to remember that ML output is probabilistic and non-deterministic...
Wait for strixs seems reasonable 58:30
Arrow Lake
This is brought to you by amazing UA-cam service.
My God. That analogy comparing an artist's life to those of slavery.
AI threatening the financial livelihood of an artist is in no way comparable to slavery. A mind who can even think that in my opinion is f'ed up.
Slavery wasn't bad because it took other people's jobs away.
It was bad because people were forcing people to be servants in the most detestable fashion.
There was nothing good about that comparison.
Yeah, that was the point dude. The point was it could end up being a terrible thing for mankind that at the time people argue is ok because "it's necessary". Nobody said we agreed with the horrible arguments people made in the past.
@@MooresLawIsDead I still stick with my statement. Slavery is not the same as Ai.
Ai is going to be essential in the secular world. Like the internet. It can be used for good or evil. I was listening to the whole video. I know what you all said.
Good video by the way. No hate here. :)
Being a copycat will never gets you very far. That is the difference between master and amateur
I've been saying this for a while...Jensen said Nvidia is an "AI Company" and he's not wrong... the GPU might have reached the end of the line, with future generations mostly making just incremental improvements. The next frontier in enthusiast hardware to maximize games and productivity will be in AI, and you need memory and processing. Plenty of room to improve that, too... but you'll want 16GB+ for any decent "worldly" LLM - memory dedicated to the model, and there may be sub-AIs handling various tasks more efficiently (like speech-to-text, text-to-speech, imaging, and such)I think in 5 years, we won't even blink about having a GPU with 32GB (using its own AI capability for things like upscaling), and an AI card with another 32GB or more.
We really need some neuromorphic chips like the recent Northpole from IBM (at least to be used for inference), putting a chip like that in a dedicated PCI-E card would speed up AI inference by 5 to 10x easily and no need for expensive memory chips since the AI weights are loaded inside the chip distributed along each processing unit.
@@MCA0090 Yes, and you also don't NEED 16x PCI-e 5.0 lanes, since the cards act as a black box to the real world, mostly. They'll sip input and output (except for loading models, of course), but that will be quality output... Intel would be well served to pivot some of their ARC program into a standalone $100 "AI-only Card" for consumers. The next performance battle will be fought for faster and more power efficient inferencing.
Star Citizen has some game enthusiast that has created a companion for the game and is using AI for a large number of features including doing things with keyboard commands in game. Its called wingman AI and it is awesome!
I never had a laptop in my life. But if Meteor lake is trash, so can i asume, that Arrow lake also will be garbage ? i Just play games.
Bethesda needs AI to fix their engine.
Did You hear about analog computing’s?
I really want to run the best language models out there. The problem I run across is memory.
I’ve given up on getting the amount of vram required to run these llms. It’s way too expensive. I think either ddr5 or unified memory is the way to go. Either way an APU is necessary.
Apart from TOPs, the next bottleneck will be memory speed. The M1 Max ultra shows that it’s possible to get decent speeds even with lower memory speeds.
That might be my next computer unless someone else offers something better. I don’t like the closed approach that Apple ecosystem offers but I’ll bite if that’s the best for running AI algorithms.
Hey! I'm the guest that showed up here. With regards to AI, you can actually have a lot of fun with fairly small models (Pygmalion 6B, Pygmalion 2 7B, etc.), even if you don't have super great hardware. It won't run super fast but in KoboldAI you can move some of the layers off to CPU / system RAM to take the worst off, and you can also run 4bit quantization (basically a technique of making the model smaller), in, as I noted, relatively modest hardware.
I think I saw someone running a 7B model on an android smartphone, lol.
If you're offloading a lot of layers to get it to work (you have an 8GB GPU, etc.) you might have better luck with KoboldCPP which lets you run a quantized model on CPU (Pygmalion 2 7B GGUF, etc.), and I want to say when I was doing something similar it was around 16GB of system RAM to get a good experience, but it's been a while, and I've been mostly using GPU, so my memory's a bit foggy.
There are a ton of resources in the PygmalionAI discord in the how-to channel including a wiki of the basic steps to get operational, and you might be surprised at how far an existing PC can take you, especially if you're willing to tinker a bit!
If you get the chance I look forward to seeing what you can do with AI!
@@IsaiahGossner Thanks for responding to my comment and thank you for being part of this podcast. It was really interesting to listen to.
In regards to smaller llms. You are right in that they are quite performative and are getting better. I also have to say that the 70b llms are getting better as well. I don't know where the sweet spot is to get the best performance for specialized llms. I guess people like you are still figuring that out (Thank you so much by the way)
I used to have a powerful computer but then a flood happened and damaged it. I now just have a 3090 and AMD 2700x while I save up for a new computer haha. I tried ggufs and it's really slow. I'm getting 1 token per second running 70B llms when I used to get 5-7 tokens per seconds. These days I'm using 13B to 20B llms to run on my 3090's vram. One of the perks of running smaller llms is that I've been increasing the context window by changing the alpha value to take more advantage of the vram. Instead of 4096 max context window, I have 6600-8000 instead.
Of course I should also pay more attention to langchain. I haven't seen the tutorials for it. I was more interested in QLoRA. I actually finetuned a language model on my own data so that was pretty fun. I want to create a chatbot of tax purposes. Can't release it out in the wild because of the copyrighted dataset but for personal use, it's pretty cool. Right now I'm just cleaning the data.
I'll join the PygmalionAI discord group. Thanks for recommending that to me. I mostly use r/locallama myself.
In regards to open source general AI, I'm quite excited for Minstral's MoE approach. Even though it needs a lot of memory from my understanding it just needs the processing power required for 13b llms. Even a powerful enough CPU should get decent tokens/seconds let alone one of Strix point and Qualcomm Snapdragon Elite X.
I'm not an expert but I reckon Minstral's MoE approach should make multimodal llms much better. I picture 8 or so 34B llms merged together. It would need a lot of memory but DDR5 is cheap enough to provide that. I don't know how viable it is but it would be pretty cool if we can get stable diffusion as one of the experts as well.
I wonder how easy it would be to add my own finetuned llm as one of the experts.
@@theworddoner Whoah, that's quite a bit, lol. I think I've heard people say similar things about GGUF before, and a lot of people prefer EXL2 I think it was with TabbyAPI / Ooba's WebUI, but people have also gotten great results with I think AWQ and GTPQ quantization in terms of speed.
As far as a sweet spot I would say the 20B -33B range is about where models start becoming a bit more rigid and don't improve as quickly per increase in parameters from what I've seen but it's a bit subjective and some people do prefer 70B (or our own 120B Goliath, lol).
I don't get to play as much with quants and settings, etc. as I'd like to, so I don't have as many opinions to offer on that front, but plenty of people in the Discord have had a chance to play with them and have sometimes very strong opinions, lol.
ROCm is horrible. AMD supports 7900s or CDNA cards, even those only recent ones. If they keep their record, in a year 7900 support will be deprecated. AMD has recognized this is the wrong approach but only improved support on Windows.
People have complained directly to AMD for 5+ years to no avail. No wonder Nvidia won...
Worse, despite good graphics drivers, AMD OpenCL support in Linux APUs is non-existant.
At least the Windows version of ROCm seems to be better but all the software is on Linux. This seems silly given directML but at least things seem to be improving.
I'm here!