At 22:29 you say Private Cloud Compute helps by allowing us to pass more info to get a better answer. Do we actually know that Private Cloud Compute is operating on a larger Personal context? That would require a considerable amount of data from the device (likely the semantic index) to be sent over the internet to obtain some inference response. Would that be a good enough user experience? For example, it might require a large portion of my SMS, email, and contact data to fully answer a complex query. That sounds like it could take a while to upload (nevermind a data plan cap), which would result in more response latency. Another possibility (perhaps more likely) is that the main advantage of Private Cloud Compute is that the model itself is much larger (> 100GB) and needs higher performance CPUs or GPUs to perform inference within a reasonable amount of time. That can all be true while the amount of Personal context used on device and in Private Cloud Compute is identical (or even less in Private Cloud Compute).
this is a great point, and you're right - this is not confirmed anywhere, but was the most convincing explanation i've heard about why Private Cloud Compute is necessary at all. all said, i'd bet that you have much more LLM expertise than i do - which makes me curious, what does a larger, higher performance model on the server afford Apple in this case?
Great break down. I recently found your videos and I've really been enjoying your take on things. From some of your videos it looks like you're in Spokane which is pretty cool. As a Spokane native myself It's been fun to see the different on location shots you use and I dig how often you use wide shots. It adds a unique style and makes the videos a little more visually interesting. Great work man, your channel is one the best hidden gems on UA-cam in my opinion.
I'm glad you're enjoying the videos! I'm not sure (but can't perfectly remember) if any of the content here was filmed in Spokane, though I have visited many times :) are you still in the Spokane area now? regardless, I'm glad you like the style & and I'm even more glad to have you as a viewer - see you in the comment section!
@@uncoverage Yeah, Spokane's my home town. I might be mistaken but the first part of your 'Steve Jobs would have never made Apple Vision Pro this way' video looks a lot like an area in Downtown Spokane.
How does apple intelligence work across all the apple devices you have - does it use Apple ID and treat them as one system or does each device have its own “AI” if so how does it communicate to complete a task ?
the easiest way to think of it is that each device has its own AI, but has shared data (if you use iCloud to sync data between devices). the only ways that servers are used is when iCloud syncs/stores data, and when Private Cloud Compute handles a particular request
I wonder if the server component is going to be something that we could run on an always-on Mac at home to keep everything under our control. It seems like a special vpn could be created that connects our phones to the private server running on our Mac mini/studio to get similar benefits of the private cloud compute
that would be very cool! i do wonder about the value of that, given the pre-existing security guarantees here, though - possibly that it takes server compute costs away from Apple?
@@uncoverage It could also be a new product category, I suppose, like an AppleTV, but it’s an at-home ‘AI module’ that every device tied to your appleid can use to boost it’s capabilities. I don’t know, I’m just kinda spitballing here. 🤣
I love the idea, and you can actually *kind of* see it implemented in Apple Vision Pro! People have talked about making a virtual "macOS app" built into the system, but the other day I was realizing that it's kind of nice to have my headset's processing power decoupled from my Mac's, which can then run heavy software like Final Cut Pro on its own. not exactly what you're talking about, but similar in spirit!
It mostly seems like the creator just did a screen record video of their iPad and used any semi professional video editing software to superimpose the screen record into the video with some basic masking to edit around the backpack in the foreground. This could probably be easily achieved with the free Davinci Resolve.
Hey. I know this is off topic, but i've been thinking lately about healthy usage of computers, and that made me wonder about the iMac on Apple's lineup today, as it seems less addictive and more communal, as a family computer. Do you have any opinion on it and if maybe that's the reason Apple keeps it around?
What is confusing me is how come AI can't be run locally on processor? I believe Apple's latest phone chips are nearing computer level power, so it would mean that even the most high end PCs today still can't run AI on device? Is AI really that powerful of a technology that it needs a room of servers?
It's a matter of GPU's really. If you try putting an RTX 4090 into your iPhone it still won't be enough to run an LLM like ChatGPT locally (atleast not to the seemless degree you can get just by doing it over the internet)
At 22:29 you say Private Cloud Compute helps by allowing us to pass more info to get a better answer. Do we actually know that Private Cloud Compute is operating on a larger Personal context? That would require a considerable amount of data from the device (likely the semantic index) to be sent over the internet to obtain some inference response. Would that be a good enough user experience? For example, it might require a large portion of my SMS, email, and contact data to fully answer a complex query. That sounds like it could take a while to upload (nevermind a data plan cap), which would result in more response latency.
Another possibility (perhaps more likely) is that the main advantage of Private Cloud Compute is that the model itself is much larger (> 100GB) and needs higher performance CPUs or GPUs to perform inference within a reasonable amount of time. That can all be true while the amount of Personal context used on device and in Private Cloud Compute is identical (or even less in Private Cloud Compute).
Only one way to find out
this is a great point, and you're right - this is not confirmed anywhere, but was the most convincing explanation i've heard about why Private Cloud Compute is necessary at all.
all said, i'd bet that you have much more LLM expertise than i do - which makes me curious, what does a larger, higher performance model on the server afford Apple in this case?
Great content, dog going crazy at the end of the video 😂😂😂
LOL it was perfect timing too, Miso is glad you noticed her performance!
Great break down. I recently found your videos and I've really been enjoying your take on things. From some of your videos it looks like you're in Spokane which is pretty cool. As a Spokane native myself It's been fun to see the different on location shots you use and I dig how often you use wide shots. It adds a unique style and makes the videos a little more visually interesting. Great work man, your channel is one the best hidden gems on UA-cam in my opinion.
I'm glad you're enjoying the videos! I'm not sure (but can't perfectly remember) if any of the content here was filmed in Spokane, though I have visited many times :) are you still in the Spokane area now?
regardless, I'm glad you like the style & and I'm even more glad to have you as a viewer - see you in the comment section!
@@uncoverage Yeah, Spokane's my home town. I might be mistaken but the first part of your 'Steve Jobs would have never made Apple Vision Pro this way' video looks a lot like an area in Downtown Spokane.
Choice or Convenience. Love it. 🍎
How does apple intelligence work across all the apple devices you have - does it use Apple ID and treat them as one system or does each device have its own “AI” if so how does it communicate to complete a task ?
the easiest way to think of it is that each device has its own AI, but has shared data (if you use iCloud to sync data between devices). the only ways that servers are used is when iCloud syncs/stores data, and when Private Cloud Compute handles a particular request
I wonder if the server component is going to be something that we could run on an always-on Mac at home to keep everything under our control. It seems like a special vpn could be created that connects our phones to the private server running on our Mac mini/studio to get similar benefits of the private cloud compute
that would be very cool! i do wonder about the value of that, given the pre-existing security guarantees here, though - possibly that it takes server compute costs away from Apple?
@@uncoverage It could also be a new product category, I suppose, like an AppleTV, but it’s an at-home ‘AI module’ that every device tied to your appleid can use to boost it’s capabilities. I don’t know, I’m just kinda spitballing here. 🤣
I love the idea, and you can actually *kind of* see it implemented in Apple Vision Pro! People have talked about making a virtual "macOS app" built into the system, but the other day I was realizing that it's kind of nice to have my headset's processing power decoupled from my Mac's, which can then run heavy software like Final Cut Pro on its own. not exactly what you're talking about, but similar in spirit!
😀Hey, I like your videos. May I ask what video editing software did you use to make the floating iPad window? It’s really cool!👍
I would like to know too -- I think it's some of the mac video presentation backgrounds, but not sure how the author did his setup
It mostly seems like the creator just did a screen record video of their iPad and used any semi professional video editing software to superimpose the screen record into the video with some basic masking to edit around the backpack in the foreground. This could probably be easily achieved with the free Davinci Resolve.
FCPX! all stock :)
Hey. I know this is off topic, but i've been thinking lately about healthy usage of computers, and that made me wonder about the iMac on Apple's lineup today, as it seems less addictive and more communal, as a family computer. Do you have any opinion on it and if maybe that's the reason Apple keeps it around?
What is confusing me is how come AI can't be run locally on processor? I believe Apple's latest phone chips are nearing computer level power, so it would mean that even the most high end PCs today still can't run AI on device?
Is AI really that powerful of a technology that it needs a room of servers?
It's a matter of GPU's really. If you try putting an RTX 4090 into your iPhone it still won't be enough to run an LLM like ChatGPT locally (atleast not to the seemless degree you can get just by doing it over the internet)